JP2019139372A - Storage device management system, storage device management device, storage device management method and program - Google Patents

Abstract

To improve access performance to a storage area secured using multiple storage devices.SOLUTION: A storage device management system according to the present invention includes multiple job execution devices each of which includes a storage device and executes a prescribed job, a job management device including a schedule extraction means analyzing a schedule of a job that the job execution device processes to extract performance of a storage area that a job in the schedule uses and a job management means setting a use schedule time at which storage areas are allocated to jobs, and a storage device management device including a storage device determination means determining multiple storage devices to be allocated to the jobs so that the performance of the storage areas is satisfied at the use schedule time.SELECTED DRAWING: Figure 1

Description

  The present invention relates to storage device management, and more particularly to a storage device management system that manages a plurality of distributed storage devices.

  An information processing system such as a High Performance Computing (HPC) system disperses and arranges various devices using a network or the like in relation to an installation location as the scale thereof increases.

  In information processing systems, performance requirements for storage devices (sometimes referred to as “storage” in the following description) are increasing. However, system procurement costs are limited. Therefore, in an information processing system, an Input and Output (IO) system including a storage device or the like is required to improve IO performance while suppressing costs (see, for example, Patent Documents 1 and 2).

  The distributed storage system described in Patent Document 1 realizes efficient data supply between a server and a storage using a cache in the distributed storage system.

  The storage system described in Patent Document 2 determines the capacity of a partition area to which a cache is allocated based on the buffer size.

  In order to increase the IO performance, the use of Solid State Drive (SSD), which has a higher access speed than the magnetic disk device, is increasingly used instead of the magnetic disk device. For example, an SSD is used as a storage device that is directly connected to a device that executes a job (hereinafter referred to as a “job execution device”).

  However, the number of storage devices to which each job execution device can be individually connected is limited. For example, when the job execution apparatuses use SSDs connected thereto, the capacity that can be used by the job execution apparatuses cannot exceed the capacity of the connected SSDs. That is, when using SSDs to which job execution devices are individually connected, there is a problem that the performance of the job execution device is limited.

  Further, each job execution apparatus has a different storage area capacity. For example, an SSD connected to a job execution apparatus with a small used capacity has a large unused area. In other words, there is a problem in that when using an SSD to which the job execution apparatus is individually connected, the use efficiency of the SSD is lowered.

  As described above, the information processing system using the SSD in which the job execution apparatuses are individually connected has a problem that the performance is limited and the use efficiency of the SSD is low.

  As a countermeasure for the above problem, an information processing system using a dedicated server (for example, a file server) including a plurality of SSDs is assumed. In this system, when realizing a storage area that satisfies the desired throughput and capacity for the job execution device, the dedicated server bundles the SDDs connected to the dedicated server and stores the necessary performance. The area is provided to the job execution device.

Japanese Unexamined Patent Publication No. 2016-212819 JP 2008-181243

  However, in the above case, all accesses from the job execution apparatus using the storage area are concentrated on the dedicated server. For this reason, when a plurality of SSDs are accessed from the job execution apparatus, an access conflict occurs. Access contention may reduce access performance in the information processing system.

  That is, in a general information processing system using a dedicated server, there is a problem that access performance is lowered when accessing a plurality of storage devices.

  The inventions described in Patent Documents 1 and 2 are inventions related to a cache, and do not improve contention for access to a storage area configured using a plurality of storage devices.

  An object of the present invention is to provide a storage device management system or the like that solves the above problems and improves the performance of access to a storage area secured using a plurality of storage devices.

  A storage device management system according to an embodiment of the present invention includes a storage device, each of which includes a storage device, executes a predetermined job, analyzes a job schedule processed by the job execution device, A job management apparatus including scheduled extraction means for extracting the performance of the storage area used by the job, and job management means for setting the scheduled use time for allocating the storage area to the job, so as to satisfy the performance in the storage area at the scheduled use time And a storage device management device including storage device determination means for determining a plurality of storage devices to be assigned to the job.

  According to one aspect of the present invention, a storage device management method includes a storage device, and a storage device management system including a plurality of job execution devices that execute a predetermined job analyzes a schedule of a job processed by the job execution device. Then, extract the performance of the storage area used by the job in the schedule, set the scheduled use time to allocate the storage area to the job, and assign multiple storages to the job to satisfy the performance in the storage area at the scheduled use time Determine the device.

  A storage device management apparatus according to an aspect of the present invention includes a storage device, and is connected to a storage device management system including a plurality of job execution devices that execute a predetermined job, and schedules jobs to be processed by the job execution device The schedule extraction means for extracting the performance of the storage area used by the scheduled job, the job management means for setting the scheduled use time for assigning the storage area to the job, and satisfying the performance in the storage area at the scheduled use time And a storage device determination means for determining a plurality of storage devices to be assigned to the job.

  According to one aspect of the present invention, there is provided a storage device management method including: a storage device management device connected to a storage device management system that includes a plurality of job execution devices each including a storage device; Analyzing the schedule of jobs to be processed, extracting the performance of the storage area used by the job in the schedule, setting the scheduled use time to allocate the storage device to the job, and satisfying the performance in the storage area at the scheduled use time Then, a plurality of storage devices to be assigned to the job are determined.

  According to an embodiment of the present invention, a program includes a storage device and schedules a job to be processed by the job execution device in a computer connected to a storage device management system including a plurality of job execution devices that execute a predetermined job. In order to satisfy the performance in the storage area in the scheduled use time, the process of extracting the performance of the storage area used by the job in the schedule, the process of setting the scheduled use time to allocate the storage area to the job, And a process for determining a plurality of storage devices to be assigned to the job.

  According to the present invention, it is possible to achieve an effect of improving the access performance for a storage area secured using a plurality of storage devices.

FIG. 1 is a block diagram showing an example of the configuration of a storage device management system according to the first embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a storage device management table according to the first embodiment. FIG. 3 is a diagram illustrating an example of a storage device allocation table according to the first embodiment. FIG. 4 is a flowchart showing an example of an operation for setting a storage area in the job management apparatus according to the first embodiment. FIG. 5 is a flowchart showing an example of the allocation operation in the storage device management apparatus according to the first embodiment. FIG. 6 is a flowchart showing an example of an operation at the start of a job in the storage device management apparatus according to the first embodiment. FIG. 7 is a flowchart showing an example of an operation of releasing the allocated storage device in the storage device management apparatus according to the first embodiment. FIG. 8 is a diagram illustrating an example of job request information. FIG. 9 is a block diagram illustrating an example of a schematic configuration of the storage device management system according to the first embodiment. FIG. 10 is a block diagram illustrating an example of another configuration of the outline according to the first embodiment. FIG. 11 is a block diagram illustrating an example of a hardware configuration in the storage device management apparatus according to the first embodiment.

  Next, embodiments of the present invention will be described with reference to the drawings.

  Each drawing is for explaining an embodiment in the present invention. However, the present invention is not limited to the description of each drawing. Moreover, the same number is attached | subjected to the same structure of each drawing, and the repeated description may be abbreviate | omitted. Further, in the drawings used for the following description, the description of the configuration of the part not related to the description of the present invention is omitted, and there are cases where it is not illustrated.

  First, terms used for describing the embodiment of the present invention will be summarized.

  In the following description, “job” is used as a unit of processing. However, this does not limit the embodiment in the present invention. The unit of processing may be a unit to which a storage area is allocated. For example, the unit of processing may be “task” or “thread”.

  “Job schedule” is a schedule of a job to be executed in the job execution apparatus. The job schedule is created based on a job request (request information) to the job execution apparatus. The content and format of the request information is not limited. For example, the request information may specify a calculation resource such as the amount of memory used in the job, the number of CPUs, an upper limit of elapsed time, and / or a batch queue. The request information may be described using, for example, a predetermined program language or a script language.

  Note that the job schedule used in the following description includes the performance related to the storage area that the job plans to use. However, the performance is not limited. In the following description, “capacity” and “throughput” are used as examples of performance. Further, the performance of the storage area is not limited to the access performance as described above, and may include other requirements such as functional contents (for example, a predetermined RAID (Redundant Arrays of Inexpensive Disks) function).

  Further, the unit of allocation of the physical configuration allocated to the storage area in each embodiment is not limited. Each embodiment may use a storage device as an allocation unit, or may use an area obtained by logically dividing the storage device (hereinafter referred to as “partition”). In the following description, a partition is used as an example of an allocation unit. When a storage device is used as an allocation unit, the partition in the following description may be changed to a storage device as appropriate.

<First Embodiment>
The first embodiment will be described below with reference to the drawings.

[Description of configuration]
First, the configuration of the storage device management system 100 according to the first embodiment will be described with reference to the drawings.

  FIG. 1 is a diagram showing an example of the configuration of a storage device management system 100 according to the first embodiment of the present invention.

  The storage device management system 100 includes a plurality of job execution devices 1, a job management device 2, a network 3, and a storage device management device 4.

  The network 3 is a communication network that connects each device. The network 3 only needs to be able to connect each device with a predetermined performance, and its configuration is not limited. For example, the network 3 is a fiber channel, a loacle area network (LAN), or a telephone line network.

  The job execution apparatuses 1 are distributed and arranged via the network 3. Each job execution apparatus 1 executes a predetermined process (job). For example, the job execution apparatus 1 is a server that provides a predetermined service. However, the job execution apparatus 1 is not limited to a server.

  Further, the job execution device 1 includes at least one storage device 11. That is, the storage devices 11 are distributed.

  The storage device 11 is optional. However, as will be described later, the storage device 11 constitutes a storage area 12 used by a job. Therefore, it is desirable that the storage device 11 has high performance (for example, access performance). For example, the storage device 11 is an SSD.

  Furthermore, the interface of the storage device 11 is arbitrary. The interface is, for example, a fiber channel.

  However, these do not limit the storage device 11.

  Any job execution apparatus 1 may execute a job using the storage area 12. Further, the storage area 12 may be configured using the storage device 11 in any job execution device 1. FIG. 1 shows, as an example, a case in which a storage area 12 configured using the storage device 11 of the upper job execution device 1 is allocated to the job of the job execution device 1 in the right center.

  The job management device 2 manages jobs executed by the job execution device 1. More specifically, the job management apparatus 2 executes the schedule setting of the storage area 12 used by the job, the allocation of the storage area 12 at the start of the job, and the processing related to the release of the storage area 12 at the end of the job. To do.

  The job management apparatus 2 includes a schedule extraction unit 21 and a job management unit 22.

  The schedule extraction unit 21 analyzes the schedule of the job and extracts the performance in the storage area 12 used by the job. For example, the schedule extraction unit 21 analyzes request information (for example, a job script) transmitted as a job schedule from the user or the like to the job execution apparatus 1. Then, the schedule extraction unit 21 extracts the performance of the storage area 12 used by the job based on a request (for example, a request instruction line) regarding the storage area 12 in the request information. The schedule extraction unit 21 transmits the performance of the storage area 12 used by the job to the job management unit 22.

  FIG. 8 is a diagram illustrating an example of job request information.

  In FIG. 8, a line surrounded by a broken line is a request instruction line regarding the storage area 12. This request instruction line requests a storage area 12 in which a job “JOB_NAME” has a capacity of M [GB] and a throughput of N [GB / s].

  The request instruction line is not limited to the above. The request instruction line may be described using a predetermined format set in the job management apparatus 2 in advance.

  Returning to the description with reference to FIG.

  The job management unit 22 sets a time for assigning the storage area 12 to the job (hereinafter referred to as “scheduled use time”). Then, the job management unit 22 transmits the performance and the scheduled use time to the storage device management device 4 and requests the storage device management device 4 to reserve the storage device 11 corresponding to the storage area 12.

  Further, the job management unit 22 transmits an “allocation request”, which is a request to allocate the storage device 11 to the job, to the storage device management device 4 at the start of the job in the job execution device 1.

  In addition, when the job in the job execution apparatus 1 is completed, the job management unit 22 transmits a “release request” that is a request to release the storage device 11 assigned to the job to the storage device management apparatus 4.

  Note that the job management unit 22 is not limited to a method of setting a time for allocating the storage area 12 to a job. The job management unit 22 may use a method of setting a job schedule used in a general information processing system as a method of setting the time for allocating the storage area 12 to the job.

  The storage device management device 4 manages the storage area 12 using the storage device 11 in the storage device management system 100. Further, the storage device management device 4 may manage processing for securing the storage area 12 (for example, sharing of partitions of the storage device 11).

  The storage device management device 4 includes a storage device determination unit 41 and a storage device allocation unit 43. The storage device management device 4 may further include a storage device sharing unit 42.

  The storage device determination unit 41 determines a partition allocation schedule in the storage device 11 so as to satisfy the performance of the storage area 12 received by the job management device 2 and the scheduled use time. In this state, the partition assigned to the storage area 12 is in a reserved state. That is, the storage device determination unit 41 reserves a partition based on the performance and the scheduled use time.

  The storage device determination unit 41 may consider the arrangement of the job execution device 1 (and the storage device 11) in determining a schedule (reservation). For example, the storage device determination unit 41 determines the storage device 11 based on the logical and / or physical distance between the job execution device 1 that executes a job and the job execution device 1 that includes the storage device 11. Also good. Specifically, as the storage device 11 that satisfies the performance, the storage device determination unit 41 stores the storage device 11 that minimizes the total number of hops (or the number of intervening devices) with the job execution device 1 that executes the job. A combination of these may be selected.

  When the storage device allocation unit 43 receives the “allocation request”, the storage device allocation unit 43 allocates the storage device 11 constituting the storage area 12 to the job. More specifically, the storage device allocation unit 43 allocates a partition of the storage device 11 constituting the storage area 12 reserved for allocation to the job.

  Further, when receiving the “release request”, the storage device allocation unit 43 releases the partition of the storage device 11 constituting the allocated storage area 12.

  When the storage device sharing unit 42 needs to share a partition in the construction of the storage area 12 using the partition of the storage device 11, the storage device sharing unit 42 sets the sharing. Further, when sharing the partition, the storage device sharing unit 42 cancels the sharing of the partition when the storage area 12 is released.

  However, the storage device management system 100 may not use the storage device sharing unit 42. For example, the storage device management system 100 may include a RAID controller (not shown), and the RAID controller may manage the partition of the storage device 11 assigned to the storage area 12 as a RAID. In this case, the partition constitutes a stripe in RAID.

  The storage device management device 4 manages information for realizing these operations. An example of this information will be described with reference to the drawings. However, the storage device management device 4 may use information different from the information described below.

  FIG. 2 is a diagram illustrating an example of the storage device management table 411 according to the first embodiment. The storage device management table 411 holds information related to the storage device 11 included in the job execution device 1. The storage device management table 411 illustrated in FIG. 2 includes a job execution device identifier, a storage device identifier, a storage capacity, and a throughput.

  The “job execution device identifier” is information for uniquely identifying the job execution device 1.

  The “storage device identifier” is information for uniquely identifying the storage device 11.

  “Storage capacity” is an available capacity in the storage device 11.

  “Throughput” is the processing capacity of the storage device 11 per unit time.

  For example, the first line in FIG. 2 shows the storage device identifier “1-sda” in which the job execution device 1 with the job execution device identifier “NODE1” has a storage capacity of 1 [TB] and a throughput of 1.2 [GB / s]. The storage device 11 is provided.

  The storage device management table 411 shown in FIG. 2 includes one storage device 11 in one job execution device 1. However, this is an example. The job execution device 1 may include a plurality of storage devices 11.

  FIG. 3 is a diagram illustrating an example of the storage device allocation table 412 according to the first embodiment. The storage device allocation table 412 is information for managing partitions provided in each storage device 11, and is created for each storage device 11. The storage device allocation table 412 includes a partition identifier, a partition capacity, a start time, an end time, an allocation identifier, and an allocation state.

  “Partition identifier” is information for uniquely identifying a partition.

  “Partition capacity” is the capacity of each partition.

  “Start time” and “End time” are examples of scheduled use time, and are a start time and an end time for assigning partitions.

  The “assignment identifier” is information for uniquely identifying the job to which the partition is assigned (specifically, the storage area 12 assigned to the job). A plurality of partitions may be assigned to the storage area 12. When a plurality of partitions are allocated to the same storage area 12, the same allocation identifier is set for these partitions.

  In the following description, it is assumed that partitions in a plurality of storage devices 11 are assigned to the storage area 12. However, as long as there are a plurality of storage devices 11, at least some of the partitions may be partitions of the same storage device 11. For example, when the number of partitions assigned to the storage area 12 is greater than the number of storage devices 11, a plurality of partitions are assigned to at least some of the storage devices 11.

  "Allocation status" indicates the current allocation status of the partition. In FIG. 3, “QUEUE” indicates reserved. “FREE” indicates an assignable state. “USE” indicates in use.

  For example, the first row in FIG. 3 indicates that the partition with the partition identifier “1” having the capacity of 0.1 [TB] is used from “H1: M1: S1” to “H2: M2: S2”. Further, the first line indicates that the allocation identifier of the partition is “ZZZ” and the allocation state is “QUEUE (reserved)”.

  Note that the storage device management apparatus 4 may manage the storage device 11 using information in which information related to the storage device 11 (storage device management table 411) and information related to partitions (storage device allocation table 412) are combined. Good.

[Description of operation]
Next, operations of the storage device management system 100 according to the first embodiment will be described with reference to the drawings.

  First, the operation of the job management apparatus 2 in the allocation of the storage area 12 will be described with reference to the drawings.

  FIG. 4 is a flowchart showing an example of an operation for setting the storage area 12 in the job management apparatus 2 according to the first embodiment.

  First, the job management apparatus 2 receives job request information in the job execution apparatus 1 (step A400).

  Next, the schedule extraction unit 21 of the job management apparatus 2 performs the performance of the storage area 12 scheduled to be used by the job included in the job request information (for example, “capacity” and “throughput” of the storage area 12 allocated to the job). ) Is extracted (step A402). For example, the schedule extraction unit 21 extracts a request instruction line included in the request information.

  The job management unit 22 sets a scheduled use time for allocating the storage area 12 to the job (step A403). The job management unit 22 is not limited to the method for setting the scheduled use time. For example, the job management unit 22 may set a scheduled use time for allocating the storage area 12 to a job using a job scheduling method in a general information processing system.

  Next, the job management unit 22 transmits the performance of the storage area 12 used by the job and the scheduled use time to the storage device management apparatus 4 (step A404).

  The job management apparatus 2 waits until an allocation result is received from the storage apparatus management apparatus 4 (No in step A415).

  When the result is received (Yes in Step A415), the job management unit 22 determines whether the allocation is successful (Step A405).

  If the allocation is successful (Yes in Step A405), the job management unit 22 adds the job to the processing queue (Step A406). At this time, the job management unit 22 stores the “assignment identifier” received together with “successful assignment” in relation to the job. For example, the job management unit 22 may include an “assignment identifier” in the job queue. Alternatively, the job management unit 22 may store an “assignment identifier” so as to relate to the job using a database or the like. Then, the job management apparatus 2 ends the process.

  If assignment is not possible (No in step A405), the job management unit 22 returns to step A403. Then, the job management unit 22 sets different scheduled use times for the job to use the storage area 12. Then, the job management unit 22 repeats the above operation. That is, the job management device 2 repeats the operation until the storage device 11 to be allocated in the storage device management device 4 is determined.

  Next, the operation of the storage device management device 4 will be described.

  First, the operation of the storage device management apparatus 4 in the allocation of the storage area 12 will be described with reference to the drawings.

  FIG. 5 is a flowchart showing an example of the allocation operation in the storage device management apparatus 4 according to the first embodiment.

  The storage device determination unit 41 checks whether there is a partition that satisfies the “scheduled use time” and “performance” of the storage area 12 used by the job transmitted from the job management device 2 (step B502).

  The storage device determination unit 41 refers to the storage device management table 411 and the storage device allocation table 412 for this confirmation. For example, the storage device determination unit 41 selects a combination of partitions so as to satisfy the capacity of the storage area 12 using the partition capacity of the storage device allocation table 412. Alternatively, the storage device determination unit 41 uses the throughput of the storage device management table 411 to select the storage device 11 including the partition so as to satisfy the throughput performance of the storage area 12.

  However, in the description of the present embodiment, the storage device determination unit 41 determines allocation using partitions of a plurality of storage devices 11.

  When allocation is possible (Yes in step B502), the storage device determination unit 41 sets partition allocation in the storage device allocation table 412 (step B503). Specifically, the storage device determination unit 41 assigns an assignment identifier (assignment identifier corresponding to a job), which is a unique value, to the partition to be assigned in the storage device assignment table 412. Then, the storage device determination unit 41 changes the allocation state of the partition to which the allocation identifier is allocated to reserved (for example, “QUEUE”). In this case, the partition to which “QUEUE” is assigned is in a reserved state to be used when the job is executed.

  When there are a plurality of combinations of partitions that satisfy the received performance, the storage device determination unit 41 selects a partition using a predetermined method. For example, the storage device determination unit 41 may select a partition so as to shorten the communication path. For example, the storage device determination unit 41 minimizes the total number of relay routes (the number of hops) from the job execution device 1 that executes a job to the job execution device 1 that includes the storage device 11 that constitutes the assigned partition. Alternatively, a partition may be selected. Alternatively, the storage device determination unit 41 may select a partition so that the number of storage devices 11 constituting the partition increases in order to improve throughput or improve parallelism.

  Note that the storage device determination unit 41 may determine the allocation in consideration of other information, for example, the processing time when switching the allocation operation or the like.

  Then, the storage device determination unit 41 transmits “allocation successful” together with the “allocation identifier” to the job management device 2 (step B505).

  If the assignment cannot be made (No in Step B502), the storage device determination unit 41 transmits “assignment impossible” to the job management device 2 (Step B504).

  In the case of transmitting “unassignable”, the storage device determination unit 41 may notify a partition combination that can be assigned. For example, based on the storage device management table 411 and the storage device allocation table 412, the storage device determination unit 41 extracts assignable partition combinations that can satisfy performance at times other than the received scheduled use time. Then, the storage device determination unit 41 may transmit the time to the job management device 2. In this case, the job management unit 22 may newly set a scheduled use time based on this time.

  When the storage device 11 is a processing unit, the storage device determination unit 41 allocates a reservation for the storage device 11 instead of the partition. Furthermore, the storage device determination unit 41 sets the state of the storage device 11 in predetermined information.

  Next, the operation of the storage device management apparatus 4 at the start of a job will be described with reference to the drawings.

  FIG. 6 is a flowchart showing an example of an operation at the start of a job in the storage device management apparatus 4 according to the first embodiment.

  Since the job uses the storage area 12, the job management apparatus 2 notifies the storage apparatus management apparatus 4 of “assignment of the storage apparatus 11 to the job” before the start of the job (a predetermined time before the scheduled start time). Send “Request”. “Request for allocation” is not limited. The job management device 2 and the storage device management device 4 may determine the “allocation request” specifications in advance. In the following description, as an example, “allocation request” is assumed to be “job start notification” and “allocation identifier” of the job.

  The storage device determination unit 41 of the storage device management device 4 receives the “assignment request” (“job start notification” and “assignment identifier”) from the job management device 2 (step C601).

  The storage device determination unit 41 uses the storage device allocation table 412 to extract the partition information corresponding to the “allocation identifier” (step C602). For example, the storage device determination unit 41 extracts the identifier of the job execution device 1 including the storage device 11 constituting the partition, the identifier of the storage device 11, the identifier of the partition, and the like as the partition information corresponding to the “allocation identifier”. .

  Next, the storage device allocation unit 43 creates the storage area 12 used by the job using the partition based on the extracted information (step C603). Note that the storage device allocation unit 43 preferably configures the partitions so that the degree of parallelism in the partitions of the plurality of storage devices 11 is high. For example, the storage device allocation unit 43 creates striping using a partition.

  After the creation of the storage area 12 is completed, the storage device assignment unit 43 saves the created assignment setting for the storage area 12 (step C604). For example, the storage device allocation unit 43 changes the partition allocation state in the storage device allocation table 412 to a state indicating that the partition is being used (for example, “USE”).

  When the storage device management device 4 includes the storage device sharing unit 42, the storage device determination unit 41 may realize sharing in the storage area 12 using the storage device sharing unit 42.

  Further, the storage device management system 100 may use the created storage area 12 as a predetermined area. For example, the storage device management system 100 may use the created storage area 12 as a burst buffer area.

  The burst buffer area is an area in which accesses with small processing blocks are collectively accessed as a large block unit. In general, the burst buffer area is used for high-speed access. For this reason, it is desirable that the device constituting the burst buffer area has high-speed access performance.

  For example, the storage device management device 4 creates a storage region 12 to be used as a burst buffer region using an SSD that is accessed faster than a magnetic storage device. Then, the storage device allocation unit 43 copies the file stored in the magnetic storage device to the created storage area 12. In this case, since the job can operate using a file stored in the burst buffer area configured by the SSD, high-speed processing can be realized. However, the storage area 12 may be used as an area other than the burst buffer area.

  When the storage device 11 is a processing unit, the storage device determination unit 41 assigns the storage device 11 instead of the partition. Furthermore, the storage device determination unit 41 sets the state of the storage device 11 in predetermined information.

  Next, the operation of deleting the storage area 12 (releasing the partition assignment) in the storage device management apparatus 4 will be described with reference to the drawings.

  FIG. 7 is a flowchart illustrating an example of an operation of releasing the allocated storage device 11 in the storage device management device 4 according to the first embodiment.

  When the job is completed, the job management unit 22 of the job management device 2 transmits to the storage device management device 4 a “release request” for the storage device 11 assigned to the job. The “request for opening” is not limited. The job management device 2 and the storage device management device 4 may determine the “request for release” specification in advance. In the following description, as an example, “release request” is assumed to be “job end notification” and “assignment identifier” of the job.

  The storage device determination unit 41 of the storage device management device 4 receives the “release request” (“job end notification” and “assignment identifier”) from the job management device 2 (step D701).

  The storage device determination unit 41 extracts the partition (partition to be released) allocated to the storage area 12 based on the “assignment identifier” (step D711).

  The storage device allocation unit 43 performs a necessary operation before releasing the partition (step D702). For example, if there is a copied file in the storage area 12, the storage device allocation unit 43 moves or writes the file back to the copy source device. Alternatively, when there is a newly created file in the storage area 12, the storage device allocation unit 43 copies the file to a predetermined device.

  Note that the operation of step D702 may be executed by the storage device determination unit 41, the storage device sharing unit 42, or another configuration (not shown) of the storage device management device 4.

  After executing the necessary operation, the storage device determination unit 41 releases the partition (deletes the storage area 12) (step D703). When sharing is realized using the storage device sharing unit 42, the storage device determination unit 41 requests the storage device sharing unit 42 to cancel sharing, and releases the partition after releasing sharing.

  Then, the storage device determination unit 41 updates the partition allocation state (step D704). Specifically, the storage device determination unit 41 changes the partition assignment state corresponding to the deleted storage area 12 in the storage device assignment table 412 to an assignable state (for example, “FREE”).

  When the storage device 11 is a processing unit, the storage device determination unit 41 opens the storage device 11 instead of the partition. Furthermore, the storage device determination unit 41 updates the state of the storage device 11 in the predetermined information.

[Description of effects]
Next, effects of the storage device management system 100 according to the first embodiment will be described.

  The storage device management system 100 according to the first embodiment has an effect of improving the access performance to the storage area 12 secured using the plurality of storage devices 11.

  The reason is as follows.

  The storage device management system 100 includes a plurality of job execution devices 1, a job management device 2, and a storage device management device 4. Each of the job execution apparatuses 1 includes a storage device 11 and executes a predetermined job. The job management apparatus 2 includes a schedule extraction unit 21 and a job management unit 22. The schedule extraction unit 21 analyzes the schedule of the job processed by the job execution apparatus 1 and extracts the performance of the storage area 12 used by the job in the schedule. The job management unit 22 sets a scheduled use time for allocating the storage area 12 to a job. The storage device management device 4 includes a storage device determination unit 41. The storage device determination unit 41 determines a plurality of storage devices 11 to be assigned to a job so as to satisfy the performance in the storage area 12 in the scheduled use time.

  As described above, the storage device management system 100 according to the first embodiment uses the storage devices 11 included in the plurality of job execution devices 1 to store the performance and the scheduled use time in the storage area 12 used by the job. Area 12 is secured. A job of the job execution device 1 uses a storage area 12 configured using the storage devices 11 of a plurality of job execution devices 1. Therefore, the job of the job execution device 1 accesses the storage device 11 provided in the plurality of job execution devices 1. Access to the plurality of storage devices 11 is distributed access to the plurality of job execution devices 1. Therefore, the storage device management system 100 can reduce contention for access to individual devices (in this case, the job execution device 1).

  Furthermore, the storage device management system 100 according to the first embodiment has an effect that the storage area 12 having performance (for example, throughput and capacity) used by the job can be set.

  The reason is as follows.

  First, the job management device 2 transmits the performance required for the job to the storage device management device 4. Then, the storage device management device 4 is provided in the job execution device 1 and uses the storage device 11 to construct a storage area 12 that satisfies the performance. Because of this construction, the storage area 12 constructed using the storage device 11 is an area that satisfies the performance requested by the job.

  Note that the job execution apparatuses 1 need not all have the same storage device 11. For example, some job execution apparatuses 1 may include a storage device 11 with low access performance. In this case, the storage device determination unit 41 of the storage device management device 4 may select the storage device 11 that satisfies the performance of the storage area 12 based on the performance of the storage device 11.

  Generally, the storage device 11 with high performance is more expensive than the storage device 11 with low performance. Therefore, providing the storage device 11 with high performance within a necessary range can reduce the overall cost of the storage device management system 100.

[Outline of Embodiment]
Next, an outline of the first embodiment will be described.

  FIG. 9 is a block diagram illustrating an example of a configuration of the storage device management system 101 that is an example of an overview of the storage device management system 100 according to the first embodiment.

  The storage device management system 101 includes a plurality of job execution devices 1, a job management device 2, and a storage device management device 5. Each of the job execution apparatuses 1 includes a storage device 11 and executes a predetermined job. The job management apparatus 2 includes a schedule extraction unit 21 and a job management unit 22. The schedule extraction unit 21 analyzes the schedule of the job processed by the job execution apparatus 1 and extracts the performance of the storage area 12 used by the job in the schedule. The job management unit 22 sets a scheduled use time for allocating the storage area 12 to a job. The storage device management device 5 includes a storage device determination unit 41. The storage device determination unit 41 determines a plurality of storage devices 11 to be assigned to a job so as to satisfy the performance in the storage area 12 in the scheduled use time.

  The storage device management system 101 configured as described above can obtain the same effects as the storage device management system 100.

  This is because the configurations of the job management device 2 and the storage device management device 5 included in the storage device management system 101 operate in the same manner as the corresponding configurations in the storage device management system 100.

  The storage device management system 101 is the minimum configuration in the embodiment of the present invention.

  The job management device 2 and the storage device management device 4 may be included in one device.

  FIG. 10 is a block diagram showing another configuration example outlined in the first embodiment.

  The storage device management device 6 is connected to a plurality of job execution devices 1. Each of the job execution apparatuses 1 includes a storage device 11 and executes a predetermined job. The storage device management device 6 includes a schedule extraction unit 21, a job management unit 22, and a storage device determination unit 41. The schedule extraction unit 21 analyzes the schedule of the job processed by the job execution apparatus 1 and extracts the performance of the storage area 12 used by the recording job in the schedule. The job management unit 22 sets a scheduled use time for allocating the storage area 12 to a job. The storage device determination unit 41 determines a plurality of storage devices 11 to be assigned to a job so as to satisfy the performance in the storage area 12 in the scheduled use time.

  The storage device management apparatus 6 configured in this way can obtain the same effects as the storage device management system 100. This is because each configuration included in the storage device management apparatus 6 operates in the same manner as the corresponding configuration in the storage device management system 100.

[Hardware configuration]
Next, the hardware configuration of the job execution device 1, job management device 2, storage device management device 4, storage device management device 5, and storage device management device 6 will be described using the storage device management device 4. For the job execution device 1 and the job management device 2, the storage device management device 4 in the following description is appropriately replaced with the job execution device 1, the job management device 2, the storage device management device 5, or the storage device management device 6. That's fine.

  For example, each component of the storage device management device 4 may be configured with a hardware circuit.

  Alternatively, in the storage device management device 4, each component may be configured using a plurality of devices connected via a network.

  Alternatively, in the storage device management device 4, the plurality of components may be configured with a single piece of hardware.

  Alternatively, the storage device management device 4 may be realized as a computer device including a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). In addition to the above configuration, the storage device management device 4 may be implemented as a computer device that further includes an input / output connection circuit (IOC). In addition to the above configuration, the storage device management device 4 may be realized as a computer device that further includes a network interface circuit (NIC: Network Interface Circuit).

  FIG. 11 is a block diagram illustrating a configuration of an information processing apparatus 600 that is an example of a hardware configuration including a CPU in the storage device management apparatus 4 according to the first embodiment.

  The information processing apparatus 600 includes a CPU 610, a ROM 620, a RAM 630, an internal storage device 640, an IOC 650, and a NIC 680, and constitutes a computer device.

  The CPU 610 reads a program from the ROM 620. The CPU 610 controls the RAM 630, the internal storage device 640, the IOC 650, and the NIC 680 based on the read program. The computer including the CPU 610 controls these configurations, and implements the functions of the storage device determination unit 41, the storage device allocation unit 43, and the storage device sharing unit 42 shown in FIG.

  When realizing each function, the CPU 610 may use the RAM 630 or the internal storage device 640 as a temporary storage medium for the program.

  In addition, the CPU 610 may read a program included in the recording medium 700 that stores the program so as to be readable by a computer by using a recording medium reading device (not shown). Alternatively, the CPU 610 may receive a program from an external device (not shown) via the NIC 680, store the program in the RAM 630 or the internal storage device 640, and operate based on the stored program.

  The ROM 620 stores programs executed by the CPU 610 and fixed data. The ROM 620 is, for example, a P-ROM (Programmable-ROM) or a flash ROM.

  The RAM 630 temporarily stores programs executed by the CPU 610 and data. The RAM 630 is, for example, a D-RAM (Dynamic-RAM).

  The internal storage device 640 stores data and programs that the information processing device 600 saves over a long period of time. Further, the internal storage device 640 may operate as a temporary storage device for the CPU 610. The internal storage device 640 is, for example, a hard disk device, a magneto-optical disk device, an SSD, or a disk array device.

  Here, the ROM 620 and the internal storage device 640 are non-transitory recording media. Meanwhile, the RAM 630 is a volatile recording medium. The CPU 610 can operate based on a program stored in the ROM 620, the internal storage device 640, or the RAM 630. That is, the CPU 610 can operate using a nonvolatile recording medium or a volatile recording medium.

  The IOC 650 mediates data between the CPU 610, the input device 660, and the display device 670. The IOC 650 is, for example, an IO interface card or a USB (Universal Serial Bus) card. Further, the IOC 650 is not limited to a wired connection such as a USB, but may be wireless.

  The input device 660 is a device that receives an input instruction from an operator of the information processing apparatus 600. The input device 660 is, for example, a keyboard, a mouse, or a touch panel.

  The display device 670 is a device that displays information to the operator of the information processing apparatus 600. The display device 670 is a liquid crystal display, for example.

  The NIC 680 relays data exchange with an external device (not shown) via the network. The NIC 680 is, for example, a fiber channel card. Furthermore, the NIC 680 is not limited to a wired line, and may use wireless.

  The information processing apparatus 600 configured as described above can obtain the same effects as the storage device management apparatus 4.

  This is because the CPU 610 of the information processing device 600 can realize the same function as the storage device management device 4 based on the program.

  While the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

  The present invention can be applied to an information processing system that requires high performance in a storage area, such as an HPC system and a supercomputer.

DESCRIPTION OF SYMBOLS 1 Job execution device 2 Job management device 3 Network 4 Storage device management device 5 Storage device management device 6 Storage device management device 11 Storage device 12 Storage area 21 Schedule extraction part 22 Job management part 41 Storage device determination part 42 Storage device sharing part 43 Storage device allocation unit 100 Storage device management system 101 Storage device management system 411 Storage device management table 412 Storage device allocation table 600 Information processing device 610 CPU
620 ROM
630 RAM
640 Internal storage device 650 IOC
660 Input device 670 Display device 680 NIC
700 recording medium

Claims (10)

A plurality of job execution devices each including a storage device and executing a predetermined job;
Analyzing the schedule of the job to be processed by the job execution device, and extracting the performance of the storage area used by the job in the schedule;
A job management device including job management means for setting a scheduled use time for allocating the storage area to the job;
A storage device management system including storage device determination means for determining a plurality of storage devices to be assigned to the job so as to satisfy the performance in the storage area at the scheduled use time. The job management means transmits a request for allocation of the storage device to the job to the storage device management device at the start of the job in the job execution device,
The storage device management system according to claim 1, further comprising storage device assignment means for assigning the storage device to the job when the storage device management device receives the allocation request. The job management means, at the end of the job, sends a request for release to the storage device assigned to the job to the storage device management device;
The storage device management system according to claim 2, wherein the storage device assignment unit releases the storage device assigned to the job when the release request is received. 4. The schedule extraction unit extracts the performance of the storage area used by the job based on a request for the storage area included in request information for requesting the job to the job execution apparatus. The storage device management system according to any one of the above. The storage device management system according to any one of claims 1 to 4, wherein the job execution devices are distributed and arranged via a network. The performance includes at least the capacity and throughput that the job uses as the storage area,
The storage device management system according to claim 1, wherein the scheduled use time includes a start time and an end time. Each of the storage device management systems includes a plurality of job execution devices that each include a storage device and executes a predetermined job.
Analyzing the schedule of the job processed by the job execution device, extracting the performance of the storage area used by the job in the schedule,
Set the scheduled use time to allocate the storage area to the job,
A storage device management method that determines a plurality of storage devices to be assigned to the job so that the performance in the storage area is satisfied in the scheduled use time. Each includes a storage device and is connected to a plurality of job execution devices that execute a predetermined job,
Analyzing the schedule of the job to be processed by the job execution device, and extracting the performance of the storage area used by the job in the schedule;
Job management means for setting a scheduled use time for allocating the storage area to the job;
And a storage device determination unit that determines a plurality of storage devices to be assigned to the job so that the performance in the storage area is satisfied in the scheduled use time. A storage device management device that includes a storage device and is connected to a plurality of job execution devices that execute a predetermined job,
Analyzing the schedule of the job processed by the job execution device, extracting the performance of the storage area used by the job in the schedule,
Set a scheduled use time to allocate the storage device to the job;
A storage device management method that determines a plurality of storage devices to be assigned to the job so that the performance in the storage area is satisfied in the scheduled use time. Each computer includes a storage device and is connected to a plurality of job execution devices that execute a predetermined job.
Analyzing the schedule of the job processed by the job execution device, and extracting the performance of the storage area used by the job in the schedule;
A process of setting a scheduled use time for allocating the storage area to the job;
A program for executing a process of determining a plurality of storage devices to be assigned to the job so as to satisfy the performance in the storage area at the scheduled use time.

Images