JP2024035507A - Information processing device, information processing method, and information processing program - Google Patents

Abstract

The present invention provides an information processing device, an information processing method, and an information processing program that can reduce the ratio of the total size of parity data to the total size of all data recorded on a magnetic tape. [Solution] An information processing device classifies a plurality of data to be recorded on a magnetic tape into a plurality of groups such that the difference in the total size of data between each group falls within a certain range; Executes processing to make the total size of multiple data classified into the same size, generates parity data using multiple data of each of multiple groups whose total size is the same, and generates parity data using multiple data and parity data. Controls recording on magnetic tape. [Selection diagram] Figure 8

Description

The present disclosure relates to an information processing device, an information processing method, and an information processing program.

Patent Document 1 discloses a technique for generating parity data from a plurality of data and recording the plurality of data and the parity data on different magnetic tapes.

Patent Document 2 discloses a technique for making multiple data of the same size by adding dummy data to data other than the maximum size among multiple data of different sizes when generating parity data from multiple data. Disclosed.

Japanese Patent Application Publication No. 2021-117772 Japanese Patent Application Publication No. 2015-143952

When a plurality of data and parity data generated from the plurality of data are made into one parity group and the plurality of data have the same size, the size often differs for each parity group. In this case, for example, if multiple pieces of data are made to the same size in the order in which they are received, and the same size pieces of data and parity data generated from the pieces of data are recorded on magnetic tape, the following problem will occur. There are cases where That is, in this case, the ratio of the total size of parity data to the total size of all data recorded on the magnetic tape may increase.

The present disclosure has been made in view of the above circumstances, and provides an information processing device, an information processing method, and an information processing method that can reduce the ratio of the total size of parity data to the total size of all data recorded on a magnetic tape. and information processing programs.

The information processing device of the first aspect is an information processing device including at least one processor, wherein the processor records a plurality of data to be recorded on a magnetic tape so that the difference in total size of data between each group is constant. Classify the data into multiple groups to fit within the range of , perform processing to make the total size of multiple data classified into multiple groups the same size, and then generates parity data using the data, and controls recording of the plurality of data and parity data on the magnetic tape.

In the information processing device of the second aspect, in the information processing device of the first aspect, the processor further comprises converting the plurality of data into plural data so that the difference in data compression rate between each group falls within a certain range. Categorize into groups.

The information processing device of the third aspect is the information processing device of the first aspect or the second aspect, wherein the data to be recorded on the magnetic tape includes one or more objects including data and metadata regarding the data. The processor further classifies the plurality of data into a plurality of groups such that the difference in the total number of objects included in the packed object between each group falls within a certain range.

In the information processing apparatus according to the fourth aspect, in the information processing apparatus according to the third aspect, the processor further stores the plurality of data into the plurality of groups such that the total number of objects included in the packed objects of each group is equal to or less than a threshold value. Classify into.

In the information processing device according to a fifth aspect, in the information processing device according to any one of the first to fourth aspects, the processor further determines that the index value representing the possibility of being read within a certain period is a threshold value. Classify the above multiple data into the same group.

The information processing device according to a sixth aspect is the information processing device according to any one of the first to fifth aspects, wherein the processor further includes a plurality of Classify data into multiple groups.

The information processing method of the seventh aspect classifies a plurality of data to be recorded on a magnetic tape into a plurality of groups such that the difference in the total size of data between each group falls within a certain range, and Executes processing to make the total size of multiple data classified into groups the same size, generates parity data using multiple data of each of multiple groups whose total size is the same, and generates parity data and parity data. A processor included in an information processing device executes processing for controlling recording of data on a magnetic tape.

An information processing program according to an eighth aspect classifies a plurality of data to be recorded on a magnetic tape into a plurality of groups such that the difference in the total size of data between each group falls within a certain range, and Executes processing to make the total size of multiple data classified into groups the same size, generates parity data using multiple data of each of multiple groups whose total size is the same, and generates parity data and parity data. This is for causing a processor included in an information processing device to execute processing for controlling recording of data on a magnetic tape.

According to the present disclosure, it is possible to reduce the ratio of the total size of parity data to the total size of all data recorded on a magnetic tape.

FIG. 1 is a block diagram showing an example of the configuration of an information processing system. FIG. 3 is a diagram for explaining objects. FIG. 3 is a diagram for explaining a packed object. FIG. 2 is a block diagram illustrating an example of the hardware configuration of an information processing device. FIG. 2 is a block diagram illustrating an example of a functional configuration of an information processing device. FIG. 3 is a diagram for explaining packed object classification processing. FIG. 3 is a diagram for explaining dummy data addition processing. FIG. 3 is a diagram for explaining parity data generation processing. FIG. 3 is a diagram for explaining a process of recording packed objects and parity data onto a magnetic tape. 3 is a flowchart illustrating an example of data recording processing.

Hereinafter, embodiments for implementing the technology of the present disclosure will be described in detail with reference to the drawings.

First, with reference to FIG. 1, the configuration of an information processing system 10 according to the present embodiment will be described. As shown in FIG. 1, the information processing system 10 includes an information processing device 12 and a tape library 14. An example of the information processing device 12 is a server computer or the like.

The tape library 14 includes a plurality of slots (not shown) and a plurality of tape drives 18, and each slot stores a magnetic tape T as an example of a recording medium. Each tape drive 18 is connected to the information processing device 12. The tape drive 18 writes data to or reads data from the magnetic tape T under the control of the information processing device 12 . An example of the magnetic tape T is an LTO (Linear Tape-Open) tape.

When data is written to or read from the magnetic tape T by the information processing device 12, the magnetic tape T to be written or read is loaded into a predetermined tape drive 18 from a slot. When writing or reading data on the magnetic tape T loaded into the tape drive 18 is completed, the magnetic tape T is unloaded from the tape drive 18 into the slot in which it was originally stored.

In this embodiment, as shown in FIG. 2 as an example, the unit for handling data recorded on the magnetic tape T includes data to be saved by the user such as document data and image data, and metadata related to the data. An example of a form to which objects are applied will be explained. In the example of FIG. 2, metadata is written as "meta". Note that a storage system that handles this object is called an object storage system. The metadata includes, for example, object identification information such as an object key, object name, data size, and attribute information such as a timestamp. Note that the recording order of data and metadata when recording objects on the magnetic tape T is not particularly limited, and may be the order of metadata and data, or the order of data and metadata.

Furthermore, in this embodiment, as shown in FIG. 3 as an example, an object (hereinafter referred to as a "packed object") in which one or more objects are grouped together according to a predetermined rule (hereinafter referred to as a "packing rule") Objects are recorded on the magnetic tape T in units. This is aimed at reducing the overhead caused by issuing a command for recording to the tape drive 18 when the information processing device 12 records an object on the magnetic tape T. A packed object is an example of data to be recorded on the magnetic tape T according to the disclosed technology. "Obj" in FIG. 3 represents an object. Note that although FIG. 3 shows an example in which one packed object includes four objects, the present invention is not limited to this. One packed object may include three or less objects, or may include five or more objects. Further, the number of objects included in a packed object may not be uniform.

Examples of packing rules include rules such as combining multiple objects that include data with the same extension into the same packed object, or combining multiple objects that are likely to be read at the same time into the same packed object. Further, an example of the packing rule is a rule such that a plurality of objects are combined into one packed object such that the size of one packed object is greater than or equal to a predetermined lower limit and less than an upper limit. Furthermore, an example of a packing rule is a rule that combines multiple objects into one packed object such that the number of objects included in one packed object is greater than or equal to a predetermined lower limit and less than an upper limit. . Furthermore, a plurality of packing rules may be combined.

Next, with reference to FIG. 4, the hardware configuration of the information processing device 12 according to this embodiment will be described. As shown in FIG. 4, the information processing device 12 includes a CPU (Central Processing Unit) 20, a memory 21 as a temporary storage area, and a nonvolatile storage section 22. The information processing device 12 also includes a display 23 such as a liquid crystal display, an input device 24 such as a keyboard and a mouse, a network I/F (InterFace) 25 connected to the network, and an external I/F (InterFace) 25 to which each tape drive 18 is connected. Contains F26. The CPU 20 , memory 21 , storage section 22 , display 23 , input device 24 , network I/F 25 , and external I/F 26 are connected to the bus 27 . CPU20 is an example of a processor.

The storage unit 22 is realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like. An information processing program 30 is stored in the storage unit 22 as a storage medium. The CPU 20 reads out the information processing program 30 from the storage unit 22, loads it into the memory 21, and executes the loaded information processing program 30.

By the way, the information processing device 12 according to this embodiment receives data and metadata transmitted from an external device such as a user terminal. For example, HTTP (Hypertext Transfer Protocol) and API (Application Programming Interface) provided by the information processing system 10 are used to transmit data from the external device. In this case, for example, data to be transmitted by the user is included in the HTTP body part, and metadata regarding the data is included in the HTTP header part. The information processing device 12 stores an object in which the received data and metadata are associated with each other in the storage unit 22 . The information processing device 12 generates a packed object based on one or more objects stored in the storage unit 22 and stores the generated packed object in the storage unit 22. The packed objects generated in this way are sequentially stored in the storage unit 22.

Next, with reference to FIG. 5, the functional configuration of the information processing device 12 according to this embodiment will be described. As shown in FIG. 5, the information processing device 12 includes a classification section 40, an execution section 42, a generation section 44, and a control section 46. By executing the information processing program 30, the CPU 20 functions as a classification section 40, an execution section 42, a generation section 44, and a control section 46.

The classification unit 40 classifies the plurality of packed objects stored in the storage unit 22 into a plurality of groups such that the difference in the total size of packed objects between each group falls within a certain range. In this embodiment, the number of groups is the same as the number of data used to generate parity data (that is, the number of data in one parity group), which will be described later. Specifically, for example, in the case of a configuration in which one parity data is generated from two data (so-called 2D+1P configuration), the classification unit 40 classifies the plurality of packed objects into two groups.

A specific example of classification processing by the classification section 40 will be described with reference to FIG. 6. Here, in order to make the explanation easier to understand, an example will be explained in which there are four packed objects to be classified by the classification unit 40, and the sizes of each packed object are 1 GB, 2 GB, 3 GB, and 4 GB. . Further, here, a case will be explained using an example in which the number of groups is two.

As shown in FIG. 6, in this example, the classification unit 40 classifies 1GB packed objects and 4GB packed objects into a first group, and classifies 2GB packed objects and 3GB packed objects into a second group. Classify into. As a result, the difference in the total size of packed objects between each group is kept within a certain range. In FIG. 6, the packed object is expressed as "PO", and the number in parentheses represents the size of the packed object. The classification process by the classification unit 40 is performed, for example, by formulating the combination of packed objects in each group as a combinatorial optimization problem and solving the formulated combinatorial optimization problem.

The execution unit 42 executes processing to make the total size of the plurality of packed objects classified into the plurality of groups by the classification unit 40 the same size. In this embodiment, as shown in FIG. 7 as an example, the execution unit 42 increases the total size of packed objects in each group by adding dummy data to groups other than the group with the largest total size of packed objects. A process for making the data the same size (hereinafter referred to as "dummy data addition process") is executed. Examples of dummy data include data padded with 0s, data padded with 1s, and the like. Further, the dummy data may be data represented by a bit string in which 0s and 1s are arranged according to a specific rule, such as data in which 0s and 1s are alternately repeated. In FIG. 7, packed objects are expressed as "PO" and dummy data are expressed as "dummy." Furthermore, in FIG. 7, the horizontal length of each rectangle represents the data size of that rectangle. These notation rules are the same in FIGS. 8 and 9, which will be described later. This dummy data addition process can also be said to be a process for handling a plurality of packed objects classified into each group as one logical packed object. In other words, this dummy data addition process increases the size of the logical packed objects in each group by adding dummy data to the logical packed objects in groups other than the group with the largest logical packed object size. It can also be said to be a process of making the sizes the same.

The generation unit 44 generates parity data using the plurality of packed objects of each of the plurality of groups whose total size is made the same by the execution unit 42. In this embodiment, as shown in FIG. 8 as an example, from a plurality of packed objects in two groups after the execution unit 42 executes the dummy data addition process, a size equal to the total size of packed objects in each of the two groups is selected. Generate one piece of parity data. The parity data according to this embodiment is such that even if one of the two groups of packed objects is lost, the one group of packed objects is combined with the remaining one group of packed objects and parity data. It is a redundant code that can be restored from

The control unit 46 controls recording onto the magnetic tape T a plurality of packed objects and parity data of each group whose total size is made the same by the execution unit 42. In this embodiment, as shown in FIG. 9 as an example, the control unit 46 distributes a plurality of packed objects and parity data of each group onto the same number of magnetic tapes T as the total value of the number of groups and the number of parity data. to control recording. As a result, even if one magnetic tape T becomes unusable, packed objects and parity data recorded on the remaining two magnetic tapes T are used to determine whether the unusable magnetic tape T Packed objects recorded in can be restored. Note that the control unit 46 can also perform the above control in parallel by setting the total value of the number of groups and the number of parity data to be less than or equal to the number of tape drives 18 that can be used at one time.

In this embodiment, when performing the above control, the control unit 46 performs control to record only the packed object on the magnetic tape T out of the packed object and the dummy data added to the packed object. Thereby, it is possible to suppress a decrease in the effective capacity of the magnetic tape T due to recording of dummy data on the magnetic tape T.

If this control is performed and one of the two groups of packed objects is lost, the control unit 46 replaces the remaining one group to match the size of the parity data. Add dummy data to the packed object. With this, it is possible to restore one group of packed objects that has been lost from one group of packed objects to which dummy data has been added and the parity data.

Note that when performing the above control, the control unit 46 may also perform control to record dummy data on the magnetic tape T. In this case, the process of adding dummy data when restoring the packed object described above becomes unnecessary.

Next, with reference to FIG. 10, the operation of the information processing device 12 according to this embodiment will be described. When the CPU 20 executes the information processing program 30, the data recording process shown in FIG. 10 is executed. The data recording process shown in FIG. 10 is executed, for example, when a user inputs an instruction to start execution via the input device 24, or when a predetermined number or more of packed objects are stored in the storage unit 22. .

In step S10 of FIG. 10, as described above, the classification unit 40 sorts the plurality of packed objects stored in the storage unit 22 so that the difference in the total size of packed objects between each group falls within a certain range. Categorize into multiple groups. In step S12, the execution unit 42 executes the dummy data addition process based on the plurality of packed objects classified into the plurality of groups in step S10, as described above.

In step S14, the generation unit 44 generates parity data using the plurality of packed objects of each of the plurality of groups whose total size has been made the same in step S12, as described above. In step S16, the control unit 46 performs control to record on the magnetic tape T the plurality of packed objects and parity data of each group whose total size has been made the same in step S14, as described above. When the process of step S16 is finished, the data recording process is finished.

As a comparative example, a case will be described in which four packed objects with sizes of 1 GB, 2 GB, 3 GB, and 4 GB are generated in this order, and parity data is generated from the two packed objects.

In this case, for example, if parity data is generated from two packed objects in the order in which the packed objects are generated, parity data is generated from a 1 GB packed object and a 2 GB packed object. Specifically, 1 GB of dummy data is added to a 1 GB packed object, and then 2 GB of parity data is generated.

Furthermore, in this case, parity data is generated from the 3GB packed object and the 4G packed object. Specifically, 1 GB of dummy data is added to a 3 GB packed object, and then 4 GB of parity data is generated. That is, in this example, the size of the parity data recorded on the magnetic tape T is 6 (=2+4) GB, and the total size of all data including packed objects and parity data is 16 (=1+2+3+4+6) GB. .

In contrast, in this embodiment, as described above, 1GB packed objects and 4GB packed objects are classified into the first group, and 2GB packed objects and 3GB packed objects are classified into the second group. be done. Then, parity data is generated from the first group of packed objects and the second group of packed objects. That is, in this embodiment, the size of parity data recorded on the magnetic tape T is 5 GB, and the total size of all data including packed objects and parity data is 15 (=1+2+3+4+5) GB.

Therefore, according to this embodiment, the ratio of the total size of parity data to the total size of all data recorded on the magnetic tape T can be reduced.

Note that in the above embodiment, the classification unit 40 may further classify the plurality of packed objects into a plurality of groups such that the difference in the compression rate of the packed objects between each group falls within a certain range. Specifically, if there are a plurality of combinations for classifying a plurality of packed objects into a plurality of groups such that the difference in the total size of packed objects between each group falls within a certain range, the classification unit 40 A combination in which the difference in compression rate of packed objects between each group falls within a certain range is selected from among the plurality of sets. Packed objects recorded on the magnetic tape T are compressed by the tape drive 18. Therefore, in this case, variations in size between groups of packed objects recorded on the magnetic tape T can be suppressed.

In the above embodiment, the classification unit 40 may further classify the plurality of packed objects into a plurality of groups such that the difference in the total number of objects included in the packed objects between each group falls within a certain range. good. Specifically, when a plurality of the above combinations exist, the classification unit 40 selects a combination in which the difference in the total number of objects included in packed objects between each group falls within a certain range from among the plurality of combinations. .

Furthermore, in the embodiment described above, the classification unit 40 may further classify the plurality of packed objects into a plurality of groups such that the total number of objects included in the packed objects of each group is equal to or less than a threshold value. Specifically, when a plurality of the above combinations exist, the classification unit 40 selects a combination in which the total number of objects included in the packed objects of each group is equal to or less than a threshold value from among the plurality of combinations.

Furthermore, in the embodiment described above, the classification unit 40 may further classify into the same group a plurality of packed objects whose index value representing the possibility of being read within a certain period is equal to or greater than a threshold value. Specifically, the classification unit 40 uses read history data for packed objects such as access logs to derive the frequency at which multiple packed objects are read within a certain period (for example, within one minute), The derived frequency is used as the index value for the plurality of packed objects.

Furthermore, in the above embodiment, the classification unit 40 may further classify the plurality of packed objects into a plurality of groups such that the total size of the packed objects in each group is equal to or less than a threshold value. Specifically, when a plurality of the above combinations exist, the classification unit 40 selects a combination in which the total size of packed objects in each group is equal to or less than a threshold value from among the plurality of combinations. Furthermore, in this embodiment, the classification unit 40 may dynamically change the threshold value. In this case, for example, the classification unit 40 determines that the smaller the free space of the storage area (for example, the storage unit 22) in which the data to be recorded on the magnetic tape T and the data read from the magnetic tape T is temporarily stored, , set the threshold to a small value.

Further, in the above embodiment, a case has been described in which the disclosed technology is applied to an object storage system, but the present invention is not limited thereto. The disclosed technology may be applied to a file storage system that handles data in file units.

Further, in the above embodiment, the hardware structure of a processing unit that executes various processes, such as the classification unit 40, execution unit 42, generation unit 44, and control unit 46, is as follows. Various types of processors can be used. As mentioned above, the various processors mentioned above include the CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as circuits such as FPGA (Field Programmable Gate Array) after manufacturing. A programmable logic device (PLD), which is a processor whose configuration can be changed, and a dedicated electrical device, which is a processor with a circuit configuration specifically designed to execute a specific process, such as an ASIC (Application Specific Integrated Circuit) Includes circuits, etc.

One processing unit may be composed of one of these various processors, or a combination of two or more processors of the same type or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). combination). Further, the plurality of processing units may be configured with one processor.

As an example of configuring multiple processing units with one processor, firstly, one processor is configured with a combination of one or more CPUs and software, as typified by computers such as a client and a server. There is a form in which a processor functions as multiple processing units. Second, there are processors that use a single IC (Integrated Circuit) chip, such as System on Chip (SoC), which implements the functions of an entire system including multiple processing units. be. In this way, various processing units are configured using one or more of the various processors described above as a hardware structure.

Furthermore, as the hardware structure of these various processors, more specifically, an electric circuit (circuitry) that is a combination of circuit elements such as semiconductor elements can be used.

Further, in the above embodiment, a mode has been described in which the information processing program 30 is stored (installed) in the storage unit 22 in advance, but the present invention is not limited to this. The information processing program 30 is provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. Good too. Further, the information processing program 30 may be downloaded from an external device via a network.

10 Information processing system 12 Information processing device 14 Tape library 18 Tape drive 20 CPU
21 Memory 22 Storage section 23 Display 24 Input device 25 Network I/F
26 External I/F
27 Bus 30 Information processing program 40 Classification section 42 Execution section 44 Generation section 46 Control section T Magnetic tape

Claims (8)

An information processing device comprising at least one processor,
The processor includes:
Sort multiple pieces of data to be recorded onto magnetic tape into multiple groups so that the difference in total data size between each group falls within a certain range,
performing processing to make the total size of the plurality of data classified into the plurality of groups the same size,
An information processing device that generates parity data using a plurality of data of each of a plurality of groups having the same total size, and controls recording of the plurality of data and the parity data on a magnetic tape. The processor includes:
The information processing apparatus according to claim 1, further comprising classifying the plurality of data into the plurality of groups such that a difference in data compression rate between each group falls within a certain range. The data to be recorded on the magnetic tape is a packed object that is a collection of one or more objects including data and metadata related to the data,
The processor includes:
The information processing apparatus according to claim 1 or 2, further comprising classifying the plurality of data into the plurality of groups such that a difference in the total number of objects included in packed objects between each group falls within a certain range. . The processor includes:
The information processing apparatus according to claim 3, further comprising classifying the plurality of data into the plurality of groups such that the total number of objects included in packed objects in each group is equal to or less than a threshold value. The processor includes:
The information processing apparatus according to claim 1 or claim 2, further comprising classifying a plurality of pieces of data whose index value representing the possibility of being read within a certain period of time is equal to or greater than a threshold value into the same group. The processor includes:
The information processing apparatus according to claim 1 or 2, further comprising classifying the plurality of data into the plurality of groups such that the total size of data in each group is equal to or less than a threshold value. Sort multiple pieces of data to be recorded onto magnetic tape into multiple groups so that the difference in total data size between each group falls within a certain range,
performing processing to make the total size of the plurality of data classified into the plurality of groups the same size,
A processor included in an information processing device generates parity data using a plurality of data of each of a plurality of groups having the same total size, and controls recording of the plurality of data and the parity data on a magnetic tape. Information processing method to be carried out. Sort multiple pieces of data to be recorded onto magnetic tape into multiple groups so that the difference in total data size between each group falls within a certain range,
performing processing to make the total size of the plurality of data classified into the plurality of groups the same size,
Generate parity data using a plurality of data of each of a plurality of groups having the same total size, and perform a process of controlling recording of the plurality of data and the parity data on a magnetic tape in a processor included in the information processing device. Information processing program for execution.