JP2019095925A - Information processing device and information processing program - Google Patents

Abstract

To surely reflect update contents of data onto a transfer destination.SOLUTION: A control unit 1b is configured to, when a fist operation to a data group 3a is conducted, register operation information 4a corresponding to the data group 3a in transfer control information 4; when the data group 3a is extracted as a transfer object, temporarily hold the operation information 4a as operation information 4a1, and transfer the data group 3a to a storage device 2; when a second operation to the data group 3a during transference of the data group 3a is conducted, update the operation information 4a registered in the transfer control information 4; when the transference of the data group 3a is completed, compare the operation information 4a registered in the transfer control information 4 with the held operation information 4a1; and, when these operation information do not coincide with each other, re-transfer the data group 3a to the storage device 2.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing apparatus and an information processing program.

  Storage services, such as online storage services and cloud storage services, that provide storage areas via networks are widespread. There is also a storage gateway that acts as a gateway between such storage services and the customer's computer.

  For example, the cloud storage gateway relays write data between the customer's computer and the cloud storage system. As a more specific example, the cloud storage gateway receives write data and its write request in file units from the customer's computer, and transfers the write data in object units to the cloud storage.

  Also, as an example of the related art, the following file server for storing data on an online storage has been proposed. In this file server, the configuration of blocks into which the file is divided, the file configuration, and the database storing the meta information are divided into a plurality of divided databases, and information in the databases is uploaded to the online storage in units of divided databases. Each divided database has a Dirty flag indicating whether its content has been updated, and based on the Dirty flag, it is determined whether the content of each divided database is to be uploaded to the online storage.

Japanese Patent Application Publication No. 2014-529111 JP 2012-141738 A

  By the way, the above storage gateway temporarily stores data to be transferred to the storage service in the local storage, and extracts data to be transferred from the data stored in the local storage. In this process, the process of storing data in the local storage, and the process of extracting data to be transferred from the local storage and the process of transferring the data may be performed asynchronously.

  In such a storage gateway, there is a problem in how to extract data to be transferred from the data stored in the local storage efficiently and accurately. For example, in order to extract data to be transferred efficiently, it is conceivable to record a flag indicating the presence or absence of updating, such as the above-mentioned Dirty flag, for each data group serving as a unit to be transferred to the external storage. However, in this method, when the original data group in the local storage is updated while transferring the data group to the storage service based on the flag, the processing module that executes the transfer processing generates the update I can not recognize. Therefore, there is a possibility that the updated contents of the data group can not be accurately reflected in the storage service.

Such a problem may occur not only in the storage gateway but also in an information processing apparatus which transfers stored data to an external storage apparatus.
In one aspect, the present invention aims to provide an information processing apparatus and an information processing program capable of reliably reflecting the update content of data on a transfer destination.

  In one scheme, an information processing apparatus having a storage unit and a control unit is provided. In the information processing apparatus, the storage unit performs transfer control in which one or more operation information corresponding to one of a plurality of data groups and one of the plurality of data groups respectively classified for each transfer unit to the external storage device is registered. Store information and When the control unit performs the first operation on the first data group among the plurality of data groups, the control unit registers the first operation information corresponding to the first data group in the transfer control information, and the first data group Is extracted as a transfer target, temporarily holds the first operation information corresponding to the first data group as holding information, and transfers the first data group to the external storage device to the external storage device. When the second operation is performed on the first data group while the first data group is being transferred, the first operation information registered in the transfer control information is updated, and the first data group to the external storage device is updated. When the transfer is completed, the first operation information registered in the transfer control information is compared with the held information, and if the first operation information and the held information do not match, the first corresponding to the first operation information is generated. External data storage Re-transferred to the location.

  Further, in one solution, an information processing program that causes a computer to execute the same processing as the above-described information processing apparatus is provided.

  In one aspect, data update content can be reliably reflected in the transfer destination.

FIG. 2 is a diagram illustrating an exemplary configuration and an exemplary process of an information processing system according to the first embodiment. It is a figure showing an example of composition of an information processing system concerning a 2nd embodiment. It is a block diagram showing the example of hardware constitutions of a cloud storage gateway. It is a block diagram which shows the structural example of the processing function with which a cloud storage gateway is provided. It is a figure for demonstrating a directory table. It is a figure which shows the data structural example of an entry table. It is a figure which shows the data structural example of a chunk map table. It is a figure which shows the data structural example of a chunk table. It is a figure which shows the data structural example of a chunk group table. It is a figure which shows the structural example of a chunk group. It is a figure which shows the example of the data structure of a file object and a chunk map object. It is a figure which shows the example of the data structure of a chunk group object. It is a figure which shows the 1st example of processing about registration of operation log, and transmission of an object. It is a figure which shows the 2nd process example about the registration of operation log, and transmission of an object. It is a figure which shows the data structural example of an operation log table. FIG. 7 is a diagram illustrating an example of a registration process of a file operation log. It is a figure which shows the example of a registration process of a data operation log. It is a flowchart which shows the process example of a file system process part when file write is requested | required. It is a flow chart which shows an example of file creation start processing. It is a flowchart which shows the example of a data write-in process. It is a flowchart which shows the example of a chunk write-in process. It is a flowchart which shows the example of a file creation completion process. It is a flow chart which shows an example of data writing processing when a file update is required. It is a flowchart which shows the process example of a cloud transfer process part. It is a flowchart (the 1) which shows the example of the cloud transfer process based on a file operation log. It is a flowchart (the 2) which shows the example of the cloud transfer process based on a file operation log. It is a flowchart which shows the example of the cloud transfer process based on a data operation log.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
First Embodiment
FIG. 1 is a diagram illustrating a configuration example and a processing example of the information processing system according to the first embodiment. The information processing system shown in FIG. 1 includes an information processing device 1 and a storage device 2 connected to the outside of the information processing device 1.

  The storage device 2 stores data transferred from the information processing device 1. The storage area of the storage device 2 may be realized by a single storage device or may be realized by a plurality of storage devices.

  The information processing apparatus 1 includes a storage unit 1a and a control unit 1b. The storage unit 1 a is realized by a storage area of a storage device provided in the information processing apparatus 1, such as a random access memory (RAM) or a hard disk drive (HDD). The control unit 1 b is realized, for example, as a processor included in the information processing apparatus 1.

  Data to be transferred to the storage device 2 is stored in the storage unit 1a in a state of being classified as data groups 3a, 3b, 3c,... For each transfer unit to the storage device 2. Further, transfer control information 4 is stored in the storage unit 1a. In the transfer control information 4, one or more pieces of operation information respectively corresponding to any of the data groups 3a, 3b, 3c,... Are registered.

  When the control unit 1 b performs an operation on a certain data group, the control unit 1 b registers operation information corresponding to the data group in the transfer control information 4. In addition, the control unit 1 b acquires operation information from the transfer control information 4 and transfers a data group corresponding to the acquired operation information to the storage device 2. Thus, the control unit 1b can asynchronously execute the operation on the data group and the transfer process of the data group to the storage device 2. Then, the control unit 1 b can determine the data group to be transferred to the storage device 2 by acquiring the operation information registered in the transfer control information 4. The control unit 1 b acquires the operation information from the transfer control information 4 in the order of registration.

Hereinafter, a specific example of the process of the control unit 1b will be described.
First, it is assumed that the control unit 1b performs the first operation on the data group 3a (step S1). Then, the control unit 1b registers the operation information 4a corresponding to the data group 3a which is the operation target in the transfer control information 4 (step S2). The first operation may be, for example, an operation of updating at least a part of data included in data group 3a, or an operation of newly storing data group 3a itself in storage unit 1a. May be

  Thereafter, when data group 3a is extracted as a transfer target, control unit 1b acquires operation information 4a corresponding to data group 3a from transfer control information 4, and temporarily stores acquired operation information 4a in storage unit 1a. Temporarily hold the area (step S3). Here, the held operation information 4a is distinguished as "operation information 4a1". At the same time, the control unit 1b transfers the data group 3a corresponding to the acquired operation information 4a to the storage device 2 (step S4).

  Here, it is assumed that the control unit 1b performs a second operation on the data group 3a while transferring the data group 3a to the storage device 2 (step S5). In this case, the control unit 1b updates the operation information 4a corresponding to the operation target data group 3a registered in the transfer control information 4 (step S6).

  Further, when the transfer of the data group 3a to the storage device 2 is completed, the control unit 1b compares the operation information 4a registered in the transfer control information 4 at this time with the operation information 4a1 stored ( Step S7). Here, when the operation information 4a and the operation information 4a1 coincide with each other, the control unit 1b determines that the data group 3a is not updated, and ends the data transfer process. At this time, for example, the control unit 1 b deletes the operation information 4 a from the transfer control information 4. On the other hand, when the operation information 4a and the operation information 4a1 do not match, the control unit 1b determines that the data group 3a has been updated, and retransfers the data group 3a to the storage device 2 (step S8). .

  As described above, when the control unit 1 b performs an operation on a data group while transferring a certain data group to the storage device 2, the control unit 1 b updates the operation information corresponding to the data group registered in the transfer control information 4. Do. Further, when the transfer of the storage device 2 of the data group is completed, the control unit 1b acquires the operation information again from the transfer control information 4, and the acquired operation information and the operation information held before the transfer Compare By this comparison, the control unit 1b can determine whether or not the data group is updated during transfer of the data group. Then, when the pieces of operation information do not match, the control unit 1b determines that the data group has been updated, and retransfers the data group to the storage device 2. Thus, even when the data group is updated during transfer of the data group, the control unit 1b can reliably reflect the updated content of the data group on the storage device 2.

Second Embodiment
FIG. 2 is a diagram showing an exemplary configuration of an information processing system according to the second embodiment. The information processing system illustrated in FIG. 2 includes a cloud storage gateway 100, a NAS (Network Attached Storage) client 210, and a storage system 220. The cloud storage gateway 100 connects to the NAS client 210 via the network 231 and also connects to the storage system 220 via the network 232. The network is, for example, a LAN (Local Area Network), and the network is, for example, a WAN (Wide Area Network).

  The storage system 220 provides cloud storage services via the network 232. In the following description, a storage area available to a service user (here, the cloud storage gateway 100) by the cloud storage service provided by the storage system 220 may be referred to as "cloud storage".

  Also, in the present embodiment, as an example, the storage system 220 is realized by an object storage in which data is managed in object units. For example, the storage system 220 is realized as a distributed storage system having a plurality of storage nodes 221 each including a control server 221a and a storage device 221b. In this case, in each storage node 221, the control server 221a controls access to the storage device 221b, and a part of the cloud storage is realized by the storage area of the storage device 221b. Further, the storage node 221 to which the object from the service user (cloud storage gateway 100) is to be stored is determined based on the information specific to the object.

  On the other hand, the NAS client 210 recognizes the cloud storage gateway 100 as a NAS server that provides a storage area managed by the file system. The storage area is a storage area by cloud storage provided by the storage system 220. Then, the NAS client 210 requests the cloud storage gateway 100 to read / write data in file units in accordance with, for example, the Network File System (NFS) protocol or the Common Internet File System (CIFS) protocol. That is, the NAS client 210 can use the cloud storage as a large-capacity virtual network file system by the NAS server function of the cloud storage gateway 100.

  The NAS client 210 executes, for example, backup software for data backup. Thereby, the NAS client 210 backs up the file stored in the NAS client 210 or the file stored in a server (for example, a business server) connected to the NAS client 210 to a storage area provided by the NAS server.

  The cloud storage gateway 100 is an example of the information processing apparatus 1 illustrated in FIG. The cloud storage gateway 100 relays data transferred between the NAS client 210 and the cloud storage. For example, the cloud storage gateway 100 receives a write request of a file from the NAS client 210 by the NAS server function, and internally caches the file for which the write is requested. The cloud storage gateway 100 divides the file requested to be written into chunks and stores the file in the cloud storage. At this time, a plurality of chunks whose total size exceeds a certain size are grouped as a "chunk group", and the chunk group is transferred to the cloud storage as an object.

  In addition, at the time of caching a file, the cloud storage gateway 100 divides the file into chunks and performs “duplicate elimination” so that chunks of the same content are not stored redundantly. Furthermore, data of divided chunks is stored in a compressed state. For example, in the cloud storage service, charging may be performed according to the amount of data stored. By performing deduplication and data compression, the amount of data stored in the cloud storage can be reduced, and the service usage cost can be suppressed.

  FIG. 3 is a block diagram showing an example of the hardware configuration of the cloud storage gateway. The cloud storage gateway 100 is realized, for example, as a computer as shown in FIG.

  The cloud storage gateway 100 is entirely controlled by the processor 101. The processor 101 is, for example, a central processing unit (CPU), a micro processing unit (MPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a programmable logic device (PLD). Also, the processor 101 may be a combination of two or more elements of a CPU, an MPU, a DSP, an ASIC, and a PLD.

A RAM 102 and a plurality of peripheral devices are connected to the processor 101 via a bus 108.
The RAM 102 is used as a main storage device of the cloud storage gateway 100. The RAM 102 temporarily stores at least a part of an OS (Operating System) program and application programs to be executed by the processor 101. The RAM 102 also stores various data necessary for processing by the processor 101.

  As peripheral devices connected to the bus 108, there are an HDD 103, a graphic processing unit 104, an input interface 105, a reader 106 and a communication interface 107.

  The HDD 103 is used as an auxiliary storage device of the cloud storage gateway 100. The HDD 103 stores an OS program, an application program, and various data. Note that as the auxiliary storage device, another type of non-volatile storage device such as a solid state drive (SSD) can also be used.

  A display device 104 a is connected to the graphic processing device 104. The graphic processing device 104 causes the display device 104 a to display an image in accordance with an instruction from the processor 101. Examples of the display device include a liquid crystal display and an organic EL (Electroluminescence) display.

  An input device 105 a is connected to the input interface 105. The input interface 105 transmits the signal output from the input device 105 a to the processor 101. The input device 105a includes a keyboard, a pointing device, and the like. Examples of pointing devices include a mouse, a touch panel, a tablet, a touch pad, and a trackball.

  The portable recording medium 106 a is attached to and detached from the reading device 106. The reading device 106 reads the data recorded on the portable recording medium 106 a and transmits the data to the processor 101. Examples of the portable recording medium 106 a include an optical disc, a magneto-optical disc, a semiconductor memory, and the like.

The communication interface 107 exchanges data with other devices via the network 107a.
By the hardware configuration as described above, the processing function of the cloud storage gateway 100 can be realized. The NAS client 210 and the control server 221a can also be realized as a computer having the same hardware configuration as that shown in FIG.

  FIG. 4 is a block diagram showing an exemplary configuration of processing functions provided in the cloud storage gateway. The cloud storage gateway 100 includes a data management unit 110, an operation log management unit 120, an NAS service processing unit 131, a file system processing unit 132, and a cloud transfer processing unit 133.

  The data management unit 110 and the operation log management unit 120 are realized, for example, as storage areas of storage devices included in the cloud storage gateway 100, such as the RAM 102 and the HDD 103. Further, the processing of the NAS service processing unit 131, the file system processing unit 132, and the cloud transfer processing unit 133 is realized, for example, by the processor 101 executing a predetermined program.

  The NAS service processing unit 131 executes interface processing as a NAS server. That is, the NAS service processing unit 131 receives a request for reading, writing, and deleting a file from the NAS client 210, and transmits the requested content to the file system processing unit 132. Then, the NAS service processing unit 131 responds to the NAS client 210 based on the processing result by the file system processing unit 132.

  While accessing the data management unit 110 and the operation log management unit 120, the file system processing unit 132 executes a process according to the request from the NAS client 210. For example, when writing of a new file is requested, the file system processing unit 132 divides the actual data of the file into chunks and stores the divided actual data in the data management unit 110 while excluding duplication. At this time, the file system processing unit 132 also registers information on files other than actual data in the data management unit 110. In addition, the file system processing unit 132 registers, in the operation log management unit 120, a log of operations relating to files and chunks, which were performed stepwise until the end of the file writing process.

  The data management unit 110 stores management information on the file, such as directory information of the file on the file system, information indicating the relationship between the file and the chunk and the chunk group, and actual data included in the file. On the other hand, the operation log management unit 120 stores the operation log referred to by the cloud transfer processing unit 133 in order to determine an object to be transferred to the cloud storage 240.

  Here, the information stored in the data management unit 110 will be described. The data management unit 110 stores a directory table 111, an entry table 112, a chunk map table 113, a chunk table 114, and a chunk group table 115. The directory table 111 and the entry table 112 hold information on the directory structure on the file system, and the chunk map table 113, the chunk table 114 and the chunk group table 115 hold information on chunks.

FIG. 5 is a diagram for explaining the directory table. FIG. 5A shows an example of the data configuration of the directory table 111, and FIG. 5B shows an example of the directory structure.
The directory table 111 is management information for expressing the hierarchical structure of the directory. As shown in FIG. 5A, in the directory table 111, records having respective items of "parent", "name" and "ino" are registered. Each record is associated with one directory (folder) in the directory structure or one file in the directory. “Parent” indicates the inode number of the parent directory, “name” indicates the name of the directory or file corresponding to the record, and “ino” indicates the inode number of the directory or file corresponding to the record.

  FIG. 5B shows a directory structure based on the records 111a to 111g illustrated in FIG. 5A. In this example, a subdirectory with directory name “A” and inode number “2” and a subdirectory with directory name “B” and inode number “4” are created under the root directory of inode number “1”. ing. Under the subdirectory of directory name "A", files of file name "X" and inode number "3" are stored. Subdirectories of directory name "C" and inode number "5" are generated under the subdirectory of directory name "B". Under the subdirectory, a file having a file name “Y” and an inode number “6”, and a file having a file name “Z” and an inode number “7” are stored.

  FIG. 6 is a view showing an example of the data configuration of the entry table. The entry table 112 is management information for holding directory or file metadata. In the entry table 112, records having items of “ino” “mode” “nlink” “uid” “gid” “size” “atime” “mtime” “ctime” are registered. Each record is associated with one directory or file.

  "Ino" indicates the inode number of the directory or file. "Mode" indicates authority information. "Nlink" indicates the number of hard links. “Uid” indicates the owner's user ID, and “gid” indicates the owner's group ID. "Size" indicates the size of the file. “Atime” indicates the last access time, “mtime” indicates the last update time, and “ctime” indicates the last state change time.

  When writing of a new file is requested from the NAS client 210, the file system processing unit 132 adds a record corresponding to the file to the directory table 111 and the entry table 112 described above.

  FIG. 7 is a diagram showing an example of the data configuration of the chunk map table. The chunk map table 113 is management information for associating a file with a chunk. Records having items of “ino”, “offset”, “size”, “gno” and “gindex” are registered in the chunk map table 113. Each record is associated with one chunk generated by dividing the actual data of the file.

  "Ino" indicates the inode number of the file containing the chunk. “Offset” indicates the offset amount from the beginning of the actual data of the file to the beginning of the chunk. The combination of "ino" and "offset" uniquely identifies a chunk in the file.

  "Size" indicates the size of the chunk. In the present embodiment, as an example, the size of the chunk is variable. For example, the file system processing unit 132 determines the division position of the actual data of the file so that a chunk including the same data is easily generated according to a predetermined operation rule. This creates a variable-length chunk.

  “Gno” indicates the group number of the chunk group to which actual data in the chunk (hereinafter referred to as “chunk data”) belongs, and “gindex” indicates the index number of the chunk data in the chunk group. Chunks and chunk data are associated by registering “ino” and “offset” and “gno” and “gindex” in the record.

  In the example of FIG. 7, the file of inode number "i1" is divided into two chunks, and the file of inode number "i2" is divided into four chunks. Also, the data management unit determines that the data of the two chunks contained in the former file and the data of the two chunks from the top of the chunks contained in the latter file belong to the chunk group of the group number "g1" It is stored in 110. Furthermore, the data of the third and fourth chunks from the top among the chunks included in the latter file are stored in the data management unit 110 as chunk data belonging to the chunk group of the group number “g2”.

  FIG. 8 is a diagram showing an example of the data configuration of the chunk table. The chunk table 114 is management information for holding information on individual chunk data. In the chunk table 114, records having items of "hash", "size", "refcnt", "gno" and "gindex" are registered. Each record is associated with one chunk data.

  “Hash” indicates a hash value calculated from chunk data. "Size" indicates the size of chunk data. “Gno” indicates the number of the chunk group to which the chunk data belongs, and “gindex” indicates the index number of the chunk data in the chunk group. One chunk data is identified by the combination of "gno" and "gindex".

  “Refcnt” indicates the value of “reference counter”, and this value indicates the number of chunks in the file that refer to chunk data. That is, “refcnt” indicates how many chunks the chunk data overlaps with.

  For example, when the data of a chunk generated from a certain file and the data of a chunk generated from another file are identical, those data are associated with common chunk data. In this case, one record corresponding to the chunk data is registered in the chunk table 114, and "2" is registered in "refcnt" as the value of the reference counter. In this case, a record corresponding to each chunk is registered in the chunk map table 113, and a common group number (gno) and an index number (gindex) indicating the above chunk data are registered in those records. .

  FIG. 9 is a view showing an example of the data configuration of the chunk group table. The chunk group table 115 is management information for holding information on each chunk group and holding each chunk data included in the chunk group. In the chunk group table 115, records having items of "gno", "gindex" and "data" are registered. Each record is associated with one chunk data.

  “Gno” indicates the group number of the chunk group to which the chunk data belongs, and “gindex” indicates the index number of the chunk data in the chunk group. "Data" indicates chunk data itself. In the example of FIG. 9, the chunk group of the group number "g1" includes four chunk data of the index numbers "x1" to "x4". Also, at least two chunk data of index numbers “x1” and “x2” are included in the chunk group of group number “g2”.

FIG. 10 is a diagram showing an example of the configuration of a chunk group. The chunk and chunk group generation method will be described with reference to FIG.
The table 115a illustrated in FIG. 10 is obtained by extracting, from the chunk group table 115, a record corresponding to chunk data belonging to the chunk group of the group number "1". Similarly, a table 115 b illustrated in FIG. 10 is obtained by extracting a record corresponding to chunk data belonging to the chunk group of the group number “2” from the chunk group table 115. A table 115c shown in FIG. 10 is obtained by extracting a record corresponding to chunk data belonging to a chunk group of group number "3" from the chunk group table 115.

  When writing of a new file or updating of an existing file is requested from the NAS client 210, the file system processing unit 132 divides actual data of the file into chunks. In the example of FIG. 10, it is assumed that the actual data of the file is divided into 13 chunks. Data of each chunk are represented as data D1 to D13 in order from the top. Here, in order to simplify the description, it is assumed that the contents of the data D1 to D13 are all different (that is, not overlapping). In this case, chunk data corresponding to each of the data D1 to D13 is stored individually.

  Each chunk data is assigned a group number (gno) and an index number (gindex) in the chunk group indicated by the number. Index numbers are assigned in the order in which non-overlapping chunk data is generated by file division. Also, when the total size of chunk data assigned to the same group number reaches a certain amount, the group number is counted up, and the next chunk data is assigned the group number after count up.

  The state of a chunk group in which the total size of chunk data has not reached a fixed amount is referred to as “active” that can accept the next chunk data. Also, the state of a chunk group in which the total size of chunk data has reached a certain amount is referred to as “inactive” in which the next chunk data can not be accepted.

  In the example of FIG. 10, first, the data D1 to D5 are assigned to the chunk group of the group number "1". Then, at this stage, it is assumed that the size of the chunk group of the group number “1” reaches a certain amount, and this chunk group becomes inactive. Then, a new group number "2" is assigned to the next data D6.

  Thereafter, it is assumed that data D6 to D11 are assigned to the chunk group of the group number "2", and at this stage, this chunk group becomes inactive. Then, a new group number "3" is assigned to the next data D6. In the example of FIG. 10, the data D12 and D13 are assigned to the chunk group of the group number "3", but at this stage this chunk group is in the active state. In this case, a group number “3” and an index number “3” are assigned to chunk data (not shown) generated next.

  The deactivated chunk group is a data unit when actual data in the file is transferred to the cloud storage 240. When a certain chunk group becomes inactive, one object (a data object described later) is generated from the chunk group and transferred to the cloud storage 240.

Hereinafter, the description will be continued returning to FIG.
The cloud transfer processing unit 133 transfers various data stored in the data management unit 110 to the cloud storage 240 asynchronously with the data operation on the data management unit 110 by the file system processing unit 132. As described above, data is transferred to the cloud storage 240 in object units.

  As illustrated in FIG. 4, the cloud transfer processing unit 133 generates three types of objects such as a file object 141, a chunk map object 142, and a chunk group object 143, and transfers the objects to the cloud storage 240. Here, the data structure of the file object 141, the chunk map object 142, and the chunk group object 143 will be described with reference to FIGS. 11 and 12. FIG.

  FIG. 11 is a diagram showing an example of the data structure of the file object and the chunk map object. FIG. 11A shows a file object 141, and FIG. 11B shows a chunk map object 142.

  The file object 141 is an object for storing information registered in the directory table 111 and the entry table 112 in the cloud storage 240 in file units. As shown in FIG. 11A, the file object 141 includes "parent" and "name" as items indicating object names. The registered information is copied to each item of "parent" and "name" from the item of the same name included in the record of the corresponding file among the records of the directory table 111. Therefore, in this example, the file corresponding to the file object 141 is identified by "parent" and "name".

  Further, the file object 141 includes “ino”, “mode”, “nlink”, “uid”, “gid”, “size”, “atime”, “mtime” and “ctime” as items indicating object values. In the item of "ino", registered information is copied from the item of "ino" included in the record of the file indicated by the "parent" and "name" of the object name among the records of the directory table 111. The items of “mode” “nlink” “uid” “gid” “size” “atime” “mtime” “ctime” have the same name included in the record of the file indicated by “ino” among the records of the entry table 112 The registered information is copied from the item of.

  On the other hand, the chunk map object 142 is an object for storing information registered in the chunk map table 113 and the chunk table 114 in the cloud storage 240 in file units. As shown in FIG. 11B, the chunk map object 142 includes “ino” as an item indicating an object name. In the item of “ino”, registered information is copied from the item of “ino” included in the record indicating the corresponding file among the records of the chunk map table 113. Thus, in this example, "ino" identifies the file corresponding to chunk map object 142.

  The chunk map object 142 also includes “offset”, “size”, “hash”, “gno”, and “gindex” as items indicating object values. In each item of "offset", "size", "gno" and "gindex", information registered from the item of the same name included in the record corresponding to the file indicated by "ino" among the records of the chunk map table 113 It is copied. In the item of “hash”, the registered hash value is copied from the item of “hash” included in the record of the chunk group indicated by “gno” and “gindex” among the records of the chunk table 114.

  The chunk map table 113 generally includes a record corresponding to each of a plurality of chunks associated with one file identified by "ino". Therefore, the object values of the chunk map object 142 include the registration information of each item for all chunks associated with the file.

  The file object 141 and the chunk map object 142 as described above are transferred to the cloud storage 240. As a result, not only the actual data of the file but also management information such as metadata of the file and configuration information of the chunk associated with the file can be stored in the cloud storage 240. Therefore, for example, even when the operation of the cloud storage gateway 100 is abnormally stopped, the directory structure of the file and the configuration of the chunk can be restored.

  FIG. 12 is a diagram showing an example of the data structure of the chunk group object. The chunk group object 143 is an object for storing information including actual data of files registered in the chunk group table 115 in the cloud storage 240 in units of chunk groups. As shown in FIG. 12, the chunk group object 143 includes “gno” as an item indicating an object name. In the item of "gno", registered information is copied from the item of "gno" included in the record indicating the corresponding chunk group among the records of the chunk group table 115. Therefore, the chunk group corresponding to the chunk group object 143 is identified by this "gno".

  The chunk group object 143 also includes “header” and “datalist” as object values. The chunk group table 115 generally includes a record corresponding to each of a plurality of chunk data included in one chunk group. In FIG. 12, the chunk group indicated by the object name "gno" is represented as including n pieces of chunk data.

  In the item of "datalist", information registered in the item of "data" from the records corresponding to each chunk data included in the chunk group indicated by "gno" of the object name among the records of the chunk group table 115 is It is copied. In the example of FIG. 12, chunk data # 0 to # n-1 respectively corresponding to the index numbers (gindex) "0" to "n-1" are set in the item "datalist". As described above, chunk data (actual data of a file) is set as an object value in the item of “data”, and the chunk data is transferred to the cloud storage 240 and stored.

  In the item "header", first, the number of chunk data set in "datalist", that is, the number of chunk data included in the corresponding chunk group is set. Furthermore, in the item "header", the offset amount from the top of the chunk group is set for each chunk data set in "datalist".

Here, returning to FIG. 4, a problem in the cloud transfer processing unit 133 generating an object as illustrated in FIGS. 11 and 12 will be described.
From the information registered in the data management unit 110, the cloud transfer processing unit 133 needs to determine information to be transferred to the cloud storage 240 as an object. As an example of this determination method, there is a method in which the cloud transfer processing unit 133 monitors the entire registration information of the data management unit 110. Since the cloud transfer processing unit 133 operates asynchronously with the file system processing unit 132 that updates the registration information of the data management unit 110, it is difficult to directly receive notification of the update information of the registration information from the file system processing unit 132 . Therefore, by monitoring the entire registration information of the data management unit 110, the cloud transfer processing unit 133 can recognize the update location of the registration information and determine the information to be transferred to the cloud storage 240. However, in this method, there is a problem that the processing load for monitoring the entire registration information is high in the data management unit 110, and the processing efficiency is low.

  Further, in the above-mentioned Patent Document 2, for each divided database which is a transfer unit of data to be updated to the online storage, a Dirty flag indicating whether or not the content has been updated is recorded. A conceivable method is to set a flag indicating whether the content has been updated for each information group included in the object, out of the registration information of the data management unit 110, using this technology. According to this method, the processing efficiency is improved because the information group to be transferred can be determined only by referring to the flag corresponding to each information group. However, the processing efficiency can not be said to be high in that it has to refer to the flags of all the information groups including those not to be transferred.

  Furthermore, in the above method using the flag, if the file system processing unit 132 updates the original information group corresponding to the object while the cloud transfer processing unit 133 transfers the object, the cloud transfer processing unit 133 I can not recognize the occurrence of the update. Therefore, there is also a problem that the updated contents of the information group may not be accurately reflected in the cloud storage 240.

  Therefore, in the present embodiment, the operation log management unit 120 is used so that the information to be transferred by the cloud transfer processing unit 133 can be determined efficiently and accurately. As shown in FIG. 4, the operation log management unit 120 stores an operation log table 121. In the operation log table 121, among the registration information of the data management unit 110, a record indicating each operation log is registered by FIFO (First In / First Out) method for each information group included in the object. Further, there are two types of operation logs: file operation log 122 and data operation log 123.

  There are two types of information groups corresponding to operation logs: information groups related to files and information groups related to chunk groups. The information group on files is an information group for generating the file object 141 and the chunk map object 142, and is generated and updated in file units. The information group includes information on one file registered in the directory table 111, the entry table 112, the chunk map table 113, and the chunk table 114. Then, the operation log corresponding to this information group is the file operation log 122 described above, and the file operation log 122 is referred to in order to determine whether transmission of the file object 141 and the chunk map object 142 which are file unit objects is necessary. Be done.

  On the other hand, the information group on the chunk group is an information group for generating the chunk group object 143, and is generated and updated in units of chunk groups. Each of the information groups includes information on one chunk group registered in the chunk group table 115, and also includes chunk data (actual data of file). The operation log corresponding to this information group is the data operation log 123 described above, and the data operation log 123 is referred to in order to determine whether it is necessary to transmit the chunk group object 143 which is an object in chunk group units.

  Further, the value of the update counter is recorded in each operation log. The value of the update counter is counted up each time the file system processing unit 132 performs an operation on the corresponding information group. Then, the value of the update counter is compared before and after the transmission process of the object to the cloud storage 240, thereby making it possible to recognize the update occurrence of the original information group during the transmission process.

  The cloud transfer processing unit 133 temporarily stores the value of the update counter recorded in the operation log in the count value storage unit 133a. The count value storage unit 133a is, for example, a temporary storage area realized by the RAM 102.

  FIG. 13 is a diagram illustrating a first processing example of registration of an operation log and transmission of an object. In the example of FIG. 13, first, it is assumed that the file system processing unit 132 performs an operation on a certain information group in the data management unit 110. This information group is one of the above-described information group related to the file and the information group related to the chunk group.

  At this point, when the operation log corresponding to the information group to be operated does not exist in the operation log table 121, the file system processing unit 132 newly generates the operation log 121a corresponding to the information group, and the operation log It registers in the table 121 (step S11). At this time, the operation content for the information group is recorded as log information in the operation log 121a, and an initial value "1" is recorded as the value of the update counter 121b.

  The operation log is registered in the operation log table 121 by the FIFO method. The cloud transfer processing unit 133 specifies the information group to be transmitted as an object by acquiring the operation log sequentially from the top of the operation log table 121. In the example of FIG. 13, it is assumed that the cloud transfer processing unit 133 has acquired the operation log 121a. At this time, the cloud transfer processing unit 133 temporarily stores the value “1” of the update counter 121 b recorded in the operation log 121 a in the count value storage unit 133 a (step S 12). Then, the cloud transfer processing unit 133 generates an object based on the operation content recorded in the operation log 121a, and transmits the object to the cloud storage 240 (step S13). Although described later in detail, the cloud transfer processing unit 133 can determine which information to extract from the corresponding information group and which type of object should be transmitted according to the operation content recorded in the operation log 121a. .

  When the transmission process of the object to the cloud storage 240 is completed, the cloud transfer processing unit 133 refers to the same operation log 121 a again to acquire the value of the update counter 121 b. The cloud transfer processing unit 133 compares the acquired value of the update counter 121b with the value of the update counter stored in the count value storage unit 133a in step S12 (step S14).

  In the example of FIG. 13, these values are identical. In this case, the cloud transfer processing unit 133 determines that the object transmission process corresponding to the operation content recorded in the operation log 121a is completed, and deletes the operation log 121a from the operation log table 121 (step S15). Then, the cloud transfer processing unit 133 acquires the next operation log from the operation log table 121, and executes object transmission processing according to the content.

  FIG. 14 is a diagram illustrating a second processing example of registration of an operation log and transmission of an object. In the example of FIG. 14, as in the example of FIG. 13, first, the file system processing unit 132 performs an operation on a certain information group in the data management unit 110, and at this time, an operation corresponding to the information group to be operated. It is assumed that the log does not exist in the operation log table 121. In this case, the file system processing unit 132 newly generates an operation log 121a corresponding to the information group and registers the operation log 121a in the operation log table 121 (step S21). In the operation log 121a, the operation content for the information group is recorded as log information, and an initial value "1" is recorded as the value of the update counter 121b.

  Further, the cloud transfer processing unit 133 acquires the operation log 121a, and temporarily stores the value “1” of the update counter 121b recorded in the operation log 121a in the count value storage unit 133a (step S22). Then, the cloud transfer processing unit 133 generates an object based on the operation content recorded in the operation log 121a, and transmits the object to the cloud storage 240 (step S23).

  Here, it is assumed that the file system processing unit 132 further performs an operation on the information group corresponding to the operation log 121 a before the transmission of the object is completed. At this time, the file system processing unit 132 refers to the operation log table 121 and determines whether the operation log corresponding to the information group of the operation target is registered in the operation log table 121. Then, when it is determined that the corresponding operation log 121a is registered, the file system processing unit 132 records the operation content in the operation log 121a and counts up the value of the update counter 121b to "2" (see FIG. Step S24).

  On the other hand, when the transmission process of the object to the cloud storage 240 is completed, the cloud transfer processing unit 133 refers to the same operation log 121 a again, and acquires the value of the update counter 121 b. The cloud transfer processing unit 133 compares the acquired value of the update counter 121b with the value of the update counter stored in the count value storage unit 133a in step S22 (step S25).

  In the example of FIG. 14, these values are different. In this case, the cloud transfer processing unit 133 determines that the object transmission process corresponding to the operation content recorded in the operation log 121a is not completed. Then, the cloud transfer processing unit 133 acquires the operation log 121a again from the operation log table 121, and temporarily stores the value “2” of the update counter 121b recorded in the acquired operation log 121a in the count value storage unit 133a. (Step S26). Then, the cloud transfer processing unit 133 generates an object based on the operation content recorded in the acquired operation log 121a, and transmits the object to the cloud storage 240 (step S27).

  The cloud transfer processing unit 133 repeats the processes of steps S24 to S27 until the value of the update counter stored in the count value storage unit 133a matches the value of the update counter 121b reacquired from the operation log table 121. . Then, when these values match, the cloud transfer processing unit 133 deletes the operation log 121a from the operation log table 121.

  According to the processes shown in FIGS. 13 and 14 described above, only one operation log corresponding to one information group is registered in the operation log table 121. When a file operation log corresponding to a certain information group is registered in the operation log table 121 and an operation is further performed on the same information group, the operation content is recorded in the registered operation log. Ru. Furthermore, when the transmission process of the objects corresponding to all the operation contents recorded in the operation log is completed, the operation log is deleted from the operation log table 121.

  The storage capacity of the operation log table 121 can be reduced by such a method. Further, by registering such an operation log in the operation log table 121 by the FIFO method, the cloud transfer processing unit 133 should transmit as an object only by referring to the operation log in order from the top of the operation log table 121. Identify information groups. Therefore, the processing load of the cloud transfer processing unit 133 can be reduced, and the processing efficiency can be improved.

  Furthermore, even if the original information group is updated while the information included in a certain information group is transmitted as an object by using the value of the update counter as shown in FIG. 13 and FIG. The updated content can be sent as an object to the cloud storage 240. Specifically, the cloud transfer processing unit 133 responds by the fact that the value of the update counter 121b reacquired in step S25 of FIG. 14 does not match the value of the update counter stored in the count value storage unit 133a. It can be recognized that the information group to be updated has been updated. Then, in step S27, the cloud transfer processing unit 133 can generate an object including the updated content of the information group based on the content of the latest operation log 121a, and can transmit the object to the cloud storage 240. Such processing can accurately reflect the updated content of the information group on the cloud storage 240.

  When the file system processing unit 132 records new operation content in the existing operation log, the file system processing unit 132 does not overwrite the new operation content in the operation content already recorded in the operation log. New operation content may be additionally recorded. For example, in step S24 of FIG. 14, it is assumed that the first information included in the information group corresponding to the operation log 121a is updated. At this time, the value of the update counter 121 b is counted up to “2”. Further, it is assumed that second information different from the first information included in the same information group is updated before the transmission process of the object in step S23 is completed. At this time, the value of the update counter 121 b is counted up to “3”.

  In such a case, when the file system processing unit 132 updates the first information, the file system processing unit 132 records the first operation content corresponding to the object including the first information in the operation log 121 a. Further, when the second information is updated, the file system processing unit 132 adds the second operation content corresponding to the object including the second information to the operation log 121a while leaving the first operation content. Record.

  Thereby, when the transmission of the object is completed and the operation log 121a is reacquired, the cloud transfer processing unit 133 generates both an object corresponding to the first operation content and an object corresponding to the second operation content. And send it to the cloud storage 240. That is, the first operation content is not deleted and remains in the operation log 121a, so that not only the update content of the second information but also the update content of the first information updated before the update is stored in the cloud storage 240 Can be accurately reflected.

Next, the operation log is specifically described with reference to FIGS.
FIG. 15 is a view showing an example of the data configuration of the operation log table. Among the records of the operation log table 121, the record indicating the file operation log has items of “counter” “type” “ino” “action” “update”. Further, among the records of the operation log table 121, the record indicating the data operation log has items of “counter” “type” “gno” “action” “update”.

  “Counter” indicates the identification number of the record (ie, operation log). The identification numbers are assigned in order of registration in the operation log table 121. “Type” indicates whether the operation log is a file operation log or a data operation log. In the example of FIG. 15, the operation logs 121-1 and 121-3 are file operation logs, and the operation logs 121-2 and 121-4 are data operation logs.

  “Ino” indicates an inode number corresponding to the file operation log, and the file operation log is associated with the file by the inode number. “Gno” indicates the group number of the chunk group corresponding to the data operation log, and the data operation log is associated with the chunk group by this group number.

  “Action” indicates the content of the operation performed by the file system processing unit 132. For each operation content recorded in “action”, the type of object to be transmitted and information to be copied to the object from the data management unit 110 are determined in advance. Therefore, the cloud transfer processing unit 133 can generate a necessary object from the content of the operation recorded in “action” and transmit it to the cloud storage 240.

  "Update" indicates the value of the update counter described above. The initial value of the update counter is “1”, and this value is counted up by the file system processing unit 132 each time an operation is performed on the corresponding information group.

  Here, the details of the content of the operation recorded in “action” will be described. First, the contents of the operation recorded in “action” of the file operation log will be described. The contents of this operation include "file creation", "file update", "file creation complete" and "file deletion".

  “File creation in progress” is log information recorded when registration information in the directory table 111 and the entry table 112 is updated in response to a file operation request from the NAS client 210. The “file operation request” referred to here is a request for writing a new file from the NAS client 210, updating the file, or deleting the file. When “file creation in progress” is recorded, the information of the record corresponding to the file registered in the directory table 111 and the entry table 112 is to be transferred by the file object 141.

  “File update” is log information recorded when management information on actual data of a file is updated following an operation corresponding to “file creation in progress”. When "file update" is recorded, part of the information of the record corresponding to the file registered in the entry table 112 is updated from the time when "file creation in progress" is recorded. Therefore, at this time, the information of the record corresponding to the file registered in the entry table 112 is to be transferred by the file object 141.

  “File creation complete” is log information recorded when all processing in response to the file operation request from the NAS client 210 is completed. At this time, the information of the record corresponding to the file registered in the chunk map table 113 and the chunk table 114 is to be transferred by the chunk map object 142.

  The above “file creation”, “file update”, and “file creation complete” are until the processing according to the request is completed when writing of a new file or updating of a file is requested from the NAS client 210 The contents of the operation performed stepwise by the file system processing unit 132 are shown.

  On the other hand, when deletion of a file is requested from the NAS client 210, “file deletion” is recorded in “action”. In this case, the record associated with the file to be deleted is deleted from the directory table 111, the entry table 112, and the chunk map table 113. Therefore, when “file deletion” is recorded, a deletion request for the file object 141 and the chunk map object 142 specifying the object name indicating the file to be deleted is sent to the cloud storage 240.

  FIG. 16 is a diagram showing an example of registration processing of a file operation log. When writing of a new file is requested from the NAS client 210, the file system processing unit 132 registers the file operation log 122a corresponding to the file in the operation log table 121 (step S31). At this time, “file creation in progress” is recorded in “action”, and the value “1” of the update counter is recorded in “update”.

  Further, following the file write request, the file system processing unit 132 receives the actual data of the file from the NAS client 210 and stores it in the data management unit 110. On the other hand, at this point in time, it is assumed that the cloud transfer processing unit 133 has acquired the file operation log 122a. In this case, the cloud transfer processing unit 133 recognizes the target file from "ino" of the file operation log 122a. In addition, the cloud transfer processing unit 133 reads necessary information determined from “file creation in progress” from the data management unit 110, generates a file object 141 including the read information, and transmits the file object 141 to the cloud storage 240.

  Here, it is assumed that the process of storing the actual data of the file in the data management unit 110 is completed before the transmission of the file object 141 is completed. In this case, the file system processing unit 132 additionally records "file update" in "action" of the file operation log 122a, and counts up the value of "update" to "2" (step S32).

  Furthermore, it is assumed that the entire response processing according to the write request from the NAS client 210 is completed before the transmission of the file object 141 is completed. Alternatively, it is assumed that the entire response processing is completed before the transmission of the file object 141 based on the “file update” is completed. In these cases, the file system processing unit 132 additionally records "file creation complete" in "action" of the file operation log 122a, and counts up the value of "update" to "3" (step S33).

  In "action", for example, "0x01" is registered as a value indicating "file creation in progress", "0x02" is registered as a value indicating "file update", and as a value indicating "file creation complete" "0x04" is registered. Then, when recording a value for "action", the logical sum of the value to be newly recorded and the value recorded in "action" is calculated, and the calculated value is overwritten on "action". Ru. For example, in the state of step S32, the logical sum "0x03" of "0x01" indicating "file creation in progress" and "0x02" indicating "file update" is recorded in "action". That is, in this example, a bit string having bits respectively corresponding to “file creation in progress”, “file update”, and “file creation complete” is recorded in “action”. Then, when one of these operation contents is recorded, the bit corresponding to the operation contents is updated from “0” to “1”.

  Next, it is assumed that deletion of a file is requested from the NAS client 210 before transmission of the file object 141 based on “file creation in progress” or “file update” is completed. Alternatively, it is assumed that deletion of a file is requested from the NAS client 210 before transmission of the chunk map object 142 based on “file creation complete” is completed. In these cases, the file system processing unit 132 records "file deletion" in "action" of the file operation log 122a, and counts up the value of "update" to "4" (step S34).

  When “file deletion” is recorded, the file object 141 and the chunk map object 142 in which object names indicating files to be deleted are specified are deleted from the cloud storage 240. Therefore, “file deletion” may be recorded by overwriting information already recorded in “action”.

  Next, the contents of the operation recorded in “action” of the data operation log will be described. The contents of this operation include "during data writing", "data writing completed" and "data deletion".

  “During data writing” indicates that the total size of chunk data included in the chunk group has not reached a predetermined size, that is, the chunk group is active. In the state in which "data writing" is recorded, an object to be transmitted to the cloud storage 240 is not generated. When “data writing in progress” is registered in “action” of the data operation log, the cloud transfer processing unit 133 skips processing based on the data operation log, and the next data recorded in the operation log table 121 Get operation log.

  “Data write complete” indicates that the total size of chunk data included in the chunk group has reached a predetermined size, that is, the chunk group is inactive. In this case, the information of the record corresponding to the chunk group registered in the chunk group table 115 is to be transferred by the chunk group object 143.

  "Delete data" indicates that a chunk group is to be deleted. In this state, all chunk data included in the chunk group is invalidated (that is, the value of the reference counter corresponding to all chunk data is “0” as the file is deleted or updated). ) State. In this case, from the chunk map table 113, the record associated with the chunk group to be deleted is deleted. Therefore, when “data deletion” is recorded, a deletion request for the chunk group object 143 specifying the object name indicating the chunk group to be deleted is sent to the cloud storage 240.

  FIG. 17 is a diagram showing an example of registration processing of data operation log. When a new chunk group is generated in response to a file write or update request from the NAS client 210, the file system processing unit 132 registers the data operation log 123a corresponding to this chunk group in the operation log table 121 (step S41). At this time, “during data writing” is recorded in “action”, and an initial value “1” of the update counter is recorded in “update”.

  Thereafter, it is assumed that the total size of chunk data included in the above chunk group has reached a predetermined size. At this time, the file system processing unit 132 overwrites and records “data write complete” in “action” of the data operation log 123a, and counts up the value of “update” to “2” (step S42).

  Here, it is assumed that the cloud transfer processing unit 133 acquires the data operation log 123 a and recognizes that “data write completion” is recorded in “action”. In this case, the cloud transfer processing unit 133 recognizes the target chunk group from “gno” of the data operation log 123a. In addition, the cloud transfer processing unit 133 reads information from the record of the corresponding chunk group registered in the chunk group table 115, generates a chunk group object 143 including the read information, and transmits the chunk group object 143 to the cloud storage 240.

  Here, it is assumed that all chunk data included in the chunk group corresponding to the data operation log 123a becomes invalid before the transmission of the chunk group object 143 is completed. In this case, the file system processing unit 132 overwrites "data deletion" in "action" of the data operation log 123a and records the result, and counts up the value of "update" to "3" (step S43). The cloud transfer processing unit 133 can recognize that the corresponding chunk group has become unnecessary by reacquiring the data operation log 123a after the transmission of the chunk group object 143 is completed. Then, the cloud transfer processing unit 133 can delete the chunk group object 143 corresponding to this chunk group from the cloud storage 240. Thereby, the update content in the chunk group table 115 accompanying deletion of a chunk group can be reflected on the cloud storage 240.

  Next, processing of the cloud storage gateway 100 will be described using a flowchart. First, processing when writing of a new file is requested from the NAS client 210 will be described using FIG. 18 to FIG.

FIG. 18 is a flowchart illustrating an example of processing of the file system processing unit when file writing is requested.
[Step S101] The file system processing unit 132 receives a file write request from the NAS client 210 via the NAS service processing unit 131. At this time, directory information indicating a file write destination, a file name, and a file size are designated. When receiving the write request, the file system processing unit 132 executes file creation start processing.

  [Step S102] When the file system processing unit 132 receives, through the NAS service processing unit 131, actual data of the file for which writing has been requested, the file system processing unit 132 executes data writing processing.

  [Step S103] When the data writing process is completed, the file system processing unit 132 transmits response information indicating the completion of the writing to the NAS client 210 via the NAS service processing unit 131. At the same time, the file system processing unit 132 executes file creation completion processing.

FIG. 19 is a flowchart illustrating an example of the file creation start process. The process of FIG. 19 corresponds to the process of step S101 of FIG.
[Step S111] The file system processing unit 132 adds, to the directory table 111, a record corresponding to the file requested to be written.

  [Step S112] The file system processing unit 132 adds, to the entry table 112, a record corresponding to the file requested to be written. In the added record, for example, information is registered in each item of "ino" and "size".

  [Step S113] The file system processing unit 132 adds, to the operation log table 121, the file operation log corresponding to the file requested to be written. In the added file operation log, “file creation in progress” is recorded in the item of “action”, and “1” is recorded in the item of “update” as the value of the update counter.

FIG. 20 is a flowchart showing an example of the data writing process. The process of FIG. 20 corresponds to the process of step S102 of FIG.
[Step S121] The file system processing unit 132 divides the actual data of the file received from the NAS client 210 into chunks.

[Step S122] The file system processing unit 132 executes the processing up to the loop end of step S124 for each chunk generated by division.
[Step S123] The file system processing unit 132 executes chunk write processing for the chunk to be processed.

[Step S124] When the file system processing unit 132 executes the process of step S123 for all chunks, the file system processing unit 132 executes the process of step S125.
[Step S125] The file system processing unit 132 updates the record corresponding to the file requested to be written, which is registered in the entry table 112. In this process, for example, information is registered in each item of “mode” “nlink” “uid” “gid” “atime” “mtime” “ctime”.

  [Step S126] The file system processing unit 132 refers to the operation log table 121 and determines whether there is a file operation log corresponding to the file requested to be written. When the corresponding file operation log is found, the file system processing unit 132 executes the processing of step S128. When the corresponding file operation log does not exist, the file system processing unit 132 executes the processing of step S127.

  [Step S127] The file system processing unit 132 adds the file operation log corresponding to the file requested to be written to the operation log table 121. At this time, in the added file operation log, “file update” is recorded in the item “action”, and “1” is recorded in the item “update” as the value of the update counter.

  [Step S128] The file system processing unit 132 updates the file operation log found from the operation log table 121. Specifically, “file update” is additionally recorded in the “action” item, and the value of the update counter recorded in the “update” item is counted up.

FIG. 21 is a flowchart illustrating an example of the chunk write process. The process of FIG. 21 corresponds to the process of step S123 of FIG.
[Step S131] The file system processing unit 132 calculates a hash value based on data of the chunk to be processed.

  [Step S132] The file system processing unit 132 adds a record corresponding to the chunk to be processed to the chunk map table 113. At this time, in the record to be added, information is registered in each item of “ino”, “offset”, and “size”.

  [Step S133] The file system processing unit 132 refers to the chunk table 114, and determines whether there is a record in which the hash value calculated in step S131 is registered in the item of “hash”. When the corresponding record is found, the file system processing unit 132 executes the process of step S134, and when the corresponding record does not exist, the file system processing unit 132 executes the process of step S135.

  [Step S134] The file system processing unit 132 refers to the record found from the chunk table 114 in step S133, and counts up the value of the reference counter recorded in the item “refcnt”. Further, the file system processing unit 132 adds the items “gno” and “gindex” of the record added to the chunk map table 113 in step S 132 to the items “gno” and “gindex” of the record found from the chunk table 114. Register the registered information.

  [Step S135] The file system processing unit 132 refers to the chunk group table 115, and acquires the group number registered in the last record (that is, the largest group number at this time).

  [Step S136] The file system processing unit 132 determines whether the total size of chunk data included in the chunk group of the group number acquired in step S135 is equal to or greater than a predetermined value. The file system processing unit 132 executes the process of step S137 when the total size is equal to or greater than the predetermined value, and executes the process of step S139 when the total size is less than the predetermined value.

[Step S137] The file system processing unit 132 generates a new group number by counting up the group number acquired in Step S135.
[Step S138] The file system processing unit 132 adds a data operation log to the operation log table 121. At this time, in the added data operation log, the group number generated in step S 137 is recorded in the item “gno”, “during data writing” is recorded in the item “action”, and “update” In the item, “1” is recorded as the value of the update counter.

  [Step S139] The file system processing unit 132 adds a record to the chunk table 114. At this time, in the record to be added, the hash value calculated in step S131 is recorded in the item "hash", the size of the corresponding chunk is recorded in the item "size", and the item "refcnt" And “1” is recorded as the value of the reference counter. In addition, if it is determined Yes in step S136, the group number generated in step S137 is recorded in the item of “gno”, and an index number indicating the first chunk is recorded in the item of “gindex”. Ru. On the other hand, when it is determined No in step S136, the group number acquired in step S135 is recorded in the item of "gno". In the item "gindex", an index number indicating the next order of the last chunk data included in the chunk group corresponding to the group number is recorded.

  [Step S140] The file system processing unit 132 adds a record to the chunk group table 115. At this time, in the record to be added, the same information as the information recorded in the item with the same name in step S139 is recorded in each item of “gno” and “gindex”, and the chunk data is compressed in the item of “data” Are recorded as they are.

  [Step S141] The file system processing unit 132 determines whether the total size of chunk data included in the chunk group of the group number recorded in the record at Step S139 is equal to or greater than a predetermined value. The file system processing unit 132 executes the processing of step S142 when the total size is equal to or larger than the predetermined value, and ends the chunk writing process when the total size is smaller than the predetermined value.

  [Step S142] In this case, the operation log table 121 records the group number recorded in the record in steps S139 and S140, and the data operation log in which "data writing" is recorded in the item of "action". Exists. The file system processing unit 132 overwrites “data write complete” in the item “action” and records the data operation log, and counts up the value of the update counter recorded in the item “update”.

FIG. 22 is a flowchart showing an example of the file creation completion process. The process of FIG. 22 corresponds to the process of step S103 of FIG.
[Step S151] The file system processing unit 132 refers to the operation log table 121 and determines whether a file operation log corresponding to the file requested to be written exists. When the corresponding file operation log is found, the file system processing unit 132 executes the processing of step S153. When the corresponding file operation log does not exist, the file system processing unit 132 executes the processing of step S152.

  [Step S152] The file system processing unit 132 adds, to the operation log table 121, the file operation log corresponding to the file requested to be written. At this time, in the added file operation log, “file creation complete” is recorded in the item “action”, and “1” is recorded in the item “update” as the value of the update counter.

  [Step S153] The file system processing unit 132 updates the file operation log found from the operation log table 121. Specifically, “file creation complete” is additionally recorded in the “action” item, and the value of the update counter recorded in the “update” item is counted up.

  Next, processing when an update of an existing file is requested from the NAS client 210 will be described. When a file update is requested, the file system processing unit 132 basically executes three steps of processing such as steps S101, S102, and S103 shown in FIG. Hereinafter, the difference in processing from when writing of a new file is requested in each stage will be described.

  First, in the processing of the first stage corresponding to step S101, the following processing is executed. The file system processing unit 132 updates the record corresponding to the file requested to be updated, registered in the entry table 112. In this update, the value recorded in the "size" item is updated with the size of the updated file.

  Next, the file system processing unit 132 refers to the operation log table 121 to determine whether a file operation log corresponding to the file requested to be updated exists. If the corresponding file operation log exists, the file system processing unit 132 records “file creation in progress” in the “action” item of the file operation log, and counts up the value recorded in the “update” item. Do. On the other hand, when the corresponding file operation log does not exist, the file system processing unit 132 adds, to the operation log table 121, the file operation log corresponding to the file requested to be written. In the added file operation log, “file creation in progress” is recorded in the item of “action”, and “1” is recorded in the item of “update” as the value of the update counter.

Next, in the second stage processing corresponding to step S102, data writing processing as shown in the next FIG. 23 is executed.
FIG. 23 is a flowchart showing an example of the data writing process when a file update is requested.

  [Step S161] The file system processing unit 132 stores, in the RAM 102, the position of the record corresponding to the file before update (old file) registered in the chunk map table 113.

  [Step S162] The file system processing unit 132 executes the processing of FIG. 20 with the actual data of the file after update received from the NAS client 210 as the processing target. As a result, the actual data of the updated file is divided into chunks, and information on each chunk is registered in the chunk map table 113, the chunk table 114, and the chunk group table 115.

  [Step S163] The file system processing unit 132 executes, for each chunk of the old file, the processing up to the loop end in step S169 based on the contents of the record indicated by the position stored in step S161.

  [Step S164] The file system processing unit 132 refers to the record stored in the chunk table 114 and storing the chunk data corresponding to the chunk to be processed, and the value of the reference counter recorded in the item “refcnt” Count down.

  [Step S165] The file system processing unit 132 acquires a group number (gno) from the record referred to in Step S164. The file system processing unit 132 acquires, from the chunk table 114, the values of reference counters corresponding to all chunk data belonging to the chunk group indicated by the group number.

  [Step S166] When all the values of the reference counter acquired in Step S165 are “0”, the file system processing unit 132 executes the process of Step S167, and at least one of those values is “1” or more. In the case, the process of step S168 is performed.

  [Step S167] In this case, all chunk data included in the chunk group indicated by the group number acquired in step S165 is in an invalid state in which any chunk does not refer to any chunk data. Therefore, the file system processing unit 132 deletes the record corresponding to this chunk group from the chunk group table 115. This will delete the chunk group.

  Further, the file system processing unit 132 registers, in the operation log table 121, a data operation log for deleting the information of the chunk group from the cloud storage 240. That is, in this data operation log, “data deletion” is recorded in the item “action”, and “1” is recorded in the item “update” as the value of the update counter. In addition, when the data operation log corresponding to the applicable chunk group is already registered, the file system processing unit 132 overwrites “data deletion” in the item “action” of the data operation log and records it, “update” Count up the value recorded in the item of.

[Step S168] The file system processing unit 132 deletes the record indicated by the position stored in step S161 from the chunk map table 113.
If the result of the determination in step S166 is "No" and the value of the reference counter after the countdown becomes "0" in step S164, the file system processing unit 132 selects the record including the reference counter in the chunk table 114. You may delete it from In addition, if “Yes” is determined in step S166, the file system processing unit 132 may delete records of all chunk data belonging to the chunk group deleted in step S167 from the chunk table 114.

  [Step S169] When the file system processing unit 132 executes the processing of steps S164 to S168 for all chunks of the old file, the file writing processing is ended.

  Finally, in the third step processing corresponding to step S103, the file system processing unit 132 executes processing similar to that of FIG. 22 with the file operation log corresponding to the file requested for updating as the processing target.

By the above-described procedure, the processing of the file system processing unit 132 when the update of the existing file is requested from the NAS client 210 is completed.
When deletion of a file is requested from the NAS client 210, the following processing is executed. The file system processing unit 132 executes the processing of steps S164 to S168 in FIG. 23 with each chunk included in the file to be deleted as the processing target. As a result, the value of the reference counter of the chunk data associated with the file to be deleted is updated. Also, when a chunk group for which the reference counter for all chunk data has become “0” appears, a data operation log for deleting the information group related to the chunk group from the cloud storage 240 is registered in the operation log table 121 Ru.

  Thereafter, the file system processing unit 132 deletes the record corresponding to the file to be deleted from the directory table 111 and the entry table 112. Furthermore, the file system processing unit 132 executes the following file operation log registration process.

  In this registration process, when the file operation log corresponding to the file to be deleted does not exist in the operation log table 121, the file system processing unit 132 manipulates the file operation log specifying the inode number of the file to be deleted. Register on In the file operation log, “file deletion” is recorded in the “action” item, and “1” is recorded in the “update” item. On the other hand, when the file operation log corresponding to the file to be deleted exists in the operation log table 121, the file system processing unit 132 overwrites “file deletion” in the item “action” in the file operation log and is recorded. , The value of the item "update" is counted up. Through such registration processing, a file operation log for deleting the information group related to the file to be deleted from the cloud storage 240 is registered in the operation log table 121.

Next, processing of the cloud transfer processing unit 133 will be described. FIG. 24 is a flowchart illustrating an example of processing of the cloud transfer processing unit.
[Step S201] The cloud transfer processing unit 133 acquires, from the operation log table 121, the first operation log (that is, the record with the smallest value recorded in the item “counter”).

  [Step S202] The cloud transfer processing unit 133 determines the type of the operation log based on the item “type” of the acquired operation log. When the acquired operation log is a file operation log, the cloud transfer processing unit 133 executes the process of step S203. When the acquired operation log is a data operation log, the cloud transfer processing unit 133 executes the process of step S204.

[Step S203] The cloud transfer processing unit 133 executes cloud transfer processing based on the file operation log.
[Step S204] The cloud transfer processing unit 133 executes cloud transfer processing based on the data operation log.

  25 and 26 are flowcharts showing an example of the cloud transfer process based on the file operation log. The processes of FIGS. 25 and 26 correspond to the process of step S203 of FIG.

  [Step S211] The cloud transfer processing unit 133 extracts the value of the update counter recorded in the item “update” from the file operation log, and stores the value in the count value storage unit 133a. The file operation log from which this value is extracted is the file operation log acquired in step S202 in FIG. 24 when step S211 is executed after step S202 in FIG. If step S211 is executed after step S227, the file operation log is the file operation log reacquired in step S227.

  [Step S 212] If the item “action” in the acquired operation log includes “file creation in progress”, the cloud transfer processing unit 133 executes the process in step S 213 and “file creation in progress” is included. If not, the process of step S214 is performed.

  [Step S213] The cloud transfer processing unit 133 generates a file object whose object name is “parent” or “name” of the file corresponding to the acquired file operation log. At this time, information is copied from the record corresponding to the file registered in the directory table 111 and the entry table 112 in each item of the object value of the file object. The cloud transfer processing unit 133 transmits the generated file object to the cloud storage 240.

  [Step S214] If the item “action” in the acquired operation log includes “file update”, the cloud transfer processing unit 133 executes the process of step S215, and “file update” is included. If not, the process of step S221 of FIG. 26 is executed.

  [Step S215] The cloud transfer processing unit 133 generates a file object whose object name is “parent” or “name” of the file corresponding to the acquired file operation log. At this time, information is copied from the record corresponding to the file registered in the directory table 111 and the entry table 112 in each item of the object value of the file object.

  The cloud transfer processing unit 133 transmits the generated file object to the cloud storage 240. In this case, the cloud storage 240 is required to update the file object corresponding to the same file.

Hereinafter, the description will be continued with reference to FIG.
[Step S221] If the item of “action” in the acquired operation log includes “file creation complete”, the cloud transfer processing unit 133 executes the process of step S222, and “file creation complete” is included. If not, the process of step S223 is performed.

  [Step S222] The cloud transfer processing unit 133 generates a chunk map object whose object name is the inode number of the file corresponding to the acquired file operation log. At this time, information is copied from the record corresponding to the file registered in the chunk map table 113 and the chunk table 114 in each item of the object value of the chunk map object. The cloud transfer processing unit 133 transmits the generated chunk map object to the cloud storage 240.

  In addition, for example, when “file creation completed” and “file creation” or “file update” are recorded in the “Action” item at the time of proceeding to step S222, it is determined in the immediately preceding step S212 or step S215. File objects are sent to the cloud storage 240 based on “file creation” and “file update”. As described above, by recording "file creation complete" without overwriting the operation content such as "file creation" or "file update", it is necessary to meet the operation content such as "file creation" or "file update". Information can be reliably transferred from the data management unit 110 to the cloud storage 240.

  [Step S223] When the item of “action” in the acquired operation log includes “file deletion”, the cloud transfer processing unit 133 executes the process of step S224 and includes “file deletion”. If not, the process of step S225 is performed.

  [Step S224] The cloud transfer processing unit 133 generates a file object and a chunk map object whose object name is the identification information of the file corresponding to the acquired file operation log. These file objects and chunk map objects are objects for requesting deletion of the stored file objects and chunk map objects corresponding to the corresponding file from the cloud storage 240. The cloud transfer processing unit 133 transmits the generated file object and chunk map object to the cloud storage 240.

  [Step S225] The cloud transfer processing unit 133 refers to the operation log table 121, and reacquires the value of the update counter recorded in the item “update” from the same file operation log as step S202 in FIG.

  [Step S226] The cloud transfer processing unit 133 compares the value of the update counter stored in the count value storage unit 133a in step S211 with the value of the update counter reacquired in step S225. When these values are the same, the cloud transfer processing unit 133 executes the process of step S228, and when the values are different, the cloud transfer processing unit 133 executes the process of step S227.

  [Step S227] The cloud transfer processing unit 133 reacquires the file operation log of the reference destination in step S225 from the operation log table 121. Thereafter, the process proceeds to step S211 in FIG. 25, and the process is continued based on the re-acquired file operation log. That is, the processes of steps S227, S211 to S215, and S221 to S226 are repeated until the values of the update counter match in step S226.

[Step S228] The cloud transfer processing unit 133 deletes the file operation log referenced in step S225 from the operation log table 121.
FIG. 27 is a flowchart illustrating an example of cloud transfer processing based on a data operation log. The process of FIG. 27 corresponds to the process of step S204 of FIG.

  [Step S231] The cloud transfer processing unit 133 extracts the value of the update counter recorded in the item of “update” from the data operation log, and stores the value in the count value storage unit 133a. The data operation log as the extraction source of this value is the data operation log acquired in step S202 of FIG. 24 when step S231 is executed after step S202 of FIG. Next, when step S231 is executed, it is the data operation log reacquired in step S239.

  [Step S232] When “during data writing” is recorded in the item of “action” of the acquired data operation log, the cloud transfer processing unit 133 executes the process of step S233, and records the other information If yes, the process of step S234 is performed.

  [Step S233] The cloud transfer processing unit 133 determines whether a predetermined time has elapsed since the creation of the acquired data operation log. When the fixed time has elapsed, the cloud transfer processing unit 133 executes the process of step S235, and when the fixed time has not elapsed, the process ends.

  [Step S234] If “data write completion” is recorded in the item of “action” of the acquired data operation log, the cloud transfer processing unit 133 executes the process of step S235. On the other hand, when “data deletion” is recorded in the item, the cloud transfer processing unit 133 executes the process of step S236.

  [Step S235] The cloud transfer processing unit 133 generates a chunk group object whose object name is the group number recorded in the acquired data operation log. At this time, the cloud transfer processing unit 133 acquires chunk data from the record of the chunk group table 115 corresponding to the group number, and stores the chunk data in the item “datalist” of the chunk group object. In addition, the cloud transfer processing unit 133 stores information necessary for the item of “header” of the chunk group object based on the chunk data. The cloud transfer processing unit 133 transmits the generated chunk group object to the cloud storage 240.

  [Step S236] The cloud transfer processing unit 133 generates a chunk group object whose object name is the group number recorded in the acquired data operation log. The chunk group object is an object for requesting deletion of the stored chunk group object corresponding to the corresponding group number from the cloud storage 240. The cloud transfer processing unit 133 transmits the generated chunk group object to the cloud storage 240.

  [Step S237] The cloud transfer processing unit 133 refers to the operation log table 121, and reacquires the value of the update counter recorded in the item “update” from the same data operation log as step S202 in FIG.

  [Step S238] The cloud transfer processing unit 133 compares the value of the update counter stored in the count value storage unit 133a in step S231 with the value of the update counter reacquired in step S237. The cloud transfer processing unit 133 executes the process of step S240 when these values are the same, and executes the process of step S239 when the values are different.

  [Step S239] The cloud transfer processing unit 133 reacquires, from the operation log table 121, the data operation log referred to in step S237. Thereafter, the process proceeds to step S231, and the process is continued based on the re-acquired data operation log. That is, the process of steps S239 and S231 to S238 is repeated until the value of the update counter matches in step S238.

[Step S240] The cloud transfer processing unit 133 deletes the data operation log referenced in step S237 from the operation log table 121.
The processing functions of the devices (the information processing device 1 and the cloud storage gateway 100) described in the above embodiments can be realized by a computer. In that case, a program describing the processing content of the function that each device should have is provided, and the above processing function is realized on the computer by executing the program on the computer. The program in which the processing content is described can be recorded on a computer readable recording medium. Examples of the computer-readable recording medium include a magnetic storage device, an optical disc, a magneto-optical recording medium, and a semiconductor memory. The magnetic storage device includes a hard disk drive (HDD), a flexible disk (FD), a magnetic tape, and the like. The optical disc includes a CD (Compact Disc), a DVD (Digital Versatile Disc), a Blu-ray Disc (BD), and the like. Magneto-optical recording media include MO (Magneto-Optical disk) and the like.

  In the case of distributing the program, for example, portable recording media, such as DVDs and CDs, on which the program is recorded are sold. Alternatively, the program may be stored in the storage device of the server computer, and the program may be transferred from the server computer to another computer via a network.

  The computer executing the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. Also, each time a program is transferred from a server computer connected via a network, the computer can sequentially execute processing in accordance with the received program.

The following appendices will be further disclosed regarding the above-described embodiments.
(Supplementary Note 1) A plurality of data groups respectively classified for each transfer unit to the external storage apparatus, and transfer control information in which one or more operation information corresponding to any of the plurality of data groups are registered are stored. A storage unit and a control unit;
The control unit
When a first operation is performed on a first data group among the plurality of data groups, the first operation information corresponding to the first data group is registered in the transfer control information;
When the first data group is extracted as a transfer target, the first operation information corresponding to the first data group is temporarily held as holding information, and the first data group is externally transmitted. Transfer to the storage device,
If a second operation is performed on the first data group while the first data group is being transferred to the external storage device, the first operation information registered in the transfer control information is updated;
When transfer of the first data group to the external storage device is completed, the first operation information registered in the transfer control information is compared with the held information;
When the first operation information does not match the held information, the first data group corresponding to the first operation information is retransferred to the external storage device.
Information processing device.

(Supplementary Note 2) In the registration of the first operation information, a predetermined count value is set in the first operation information,
In the update of the first operation information, the count value is updated,
In the comparison between the first operation information and the held information, the count value included in the first operation information is compared with the count value included in the held information.
The information processing apparatus according to appendix 1.

(Supplementary Note 3) In the registration of the first operation information, a first operation content indicating the first operation is set in the first operation information,
In the transfer of the first data group, data extracted from the first data group is transferred to the external storage device based on the first operation content set in the first operation information. ,
In the update of the first operation information, a second operation content indicating the second operation is set in the first operation information,
In the retransfer of the first data group, data extracted from the first data group is transferred to the external storage device based on the second operation content set in the first operation information. Do,
The information processing apparatus according to appendix 1 or 2.

(Supplementary Note 4) In the update of the first operation information, the second operation content is additionally set to the first operation information,
In the retransfer of the first data group, when the first operation content and the second operation content are set in the first operation information, the first operation content is set based on the first operation content. Transferring data extracted from one data group and data extracted from the first data group based on the second operation content to the external storage device;
The information processing apparatus according to appendix 3.

(Supplementary Note 5) In retransfer of the first data group, the first operation information registered in the transfer control information is temporarily held as the holding information,
Further, when the retransfer of the first data group is completed, the control unit compares the first operation information registered in the transfer control information with the stored information, and the first operation information Deleting the first operation information from the transfer control information when the stored information matches
The information processing apparatus according to any one of appendices 1 to 4.

(Supplementary Note 6) The operation information is acquired in the order of registration from the transfer control information.
The information processing apparatus according to any one of appendices 1 to 5.
(Supplementary Note 7) Each of the plurality of data groups is a management data group related to a file for which writing is requested from the external information processing apparatus to the external storage apparatus,
In the first operation and the second operation, the external information processing apparatus is requested to write the first file corresponding to the first data group from the external information processing apparatus, and then the external information processing apparatus completes the write processing. It is an operation that is performed stepwise on the management data group corresponding to the first file until it responds.
The information processing apparatus according to any one of appendices 1 to 6.

(Supplementary Note 8) The control unit further divides data of each file requested to be written from the external information processing apparatus and allocates the divided data to divided areas, and eliminates overlapping of divided data included in each of the divided areas. Stored in the storage unit in the state,
The management data group corresponding to the first data group includes the division area generated by dividing the first file and the storage unit of the division data included in each of the generated division areas. Data including a correspondence relationship with a storage location, and metadata of the first file,
The first operation is a write operation of the metadata, and the second operation is a write operation of data indicating the correspondence.
The information processing apparatus according to appendix 7.

(Supplementary Note 9) The control unit further divides data of a file for which writing is requested from the external information processing device to the external storage device, and assigns the divided data to divided regions, and the divided data included in the divided regions , Storing the data in the storage unit in a state where the duplication is eliminated,
In storing the divided data, divided data groups generated by grouping one or more pieces of divided data are stored in the storage unit as one of data groups included in the plurality of data groups, respectively.
The information processing apparatus according to any one of appendices 1 to 6.

(Supplementary note 10)
The plurality of data groups stored in the storage unit when the first operation is performed on the first data group among the plurality of data groups stored in the storage unit and classified according to transfer units to the external storage device The first operation information corresponding to the first data group is registered in transfer control information in which one or more operation information corresponding to any of the data groups are registered,
When the first data group is extracted as a transfer target, the first operation information corresponding to the first data group is temporarily held as holding information, and the first data group is externally transmitted. Transfer to the storage device,
If a second operation is performed on the first data group while the first data group is being transferred to the external storage device, the first operation information registered in the transfer control information is updated;
When transfer of the first data group to the external storage device is completed, the first operation information registered in the transfer control information is compared with the held information;
When the first operation information does not match the held information, the first data group corresponding to the first operation information is retransferred to the external storage device.
An information processing program that executes processing.

(Supplementary Note 11) In the registration of the first operation information, a predetermined count value is set in the first operation information,
In the update of the first operation information, the count value is updated,
In the comparison between the first operation information and the held information, the count value included in the first operation information is compared with the count value included in the held information.
The information processing program according to appendix 10.

(Supplementary Note 12) In the registration of the first operation information, a first operation content indicating the first operation is set in the first operation information,
In the transfer of the first data group, data extracted from the first data group is transferred to the external storage device based on the first operation content set in the first operation information. ,
In the update of the first operation information, a second operation content indicating the second operation is set in the first operation information,
In the retransfer of the first data group, data extracted from the first data group is transferred to the external storage device based on the second operation content set in the first operation information. Do,
The information processing program according to appendix 10 or 11.

(Supplementary Note 13) In retransfer of the first data group, the first operation information registered in the transfer control information is temporarily held as the holding information,
On the computer
When retransfer of the first data group is completed, the first operation information registered in the transfer control information is compared with the held information, and the first operation information matches the held information. When it occurs, the first operation information is deleted from the transfer control information,
Make the process run further,
The information processing program according to any one of appendices 10 to 12.

(Supplementary Note 14) Each of the plurality of data groups is a management data group related to a file for which writing is requested from the external information processing apparatus to the external storage apparatus,
In the first operation and the second operation, the external information processing apparatus is requested to write the first file corresponding to the first data group from the external information processing apparatus, and then the external information processing apparatus completes the write processing. It is an operation that is performed stepwise on the management data group corresponding to the first file until it responds.
The information processing program according to any one of appendices 10 to 13.

(Supplementary Note 15) Data of each file requested to be written from the external information processing apparatus is divided and allocated to divided areas, and divided data included in each of the divided areas is stored in the storage unit in a state where duplication is excluded Do,
Further causing the computer to execute the process;
The management data group corresponding to the first data group includes the division area generated by dividing the first file and the storage unit of the division data included in each of the generated division areas. Data including a correspondence relationship with a storage location, and metadata of the first file,
The first operation is a write operation of the metadata, and the second operation is a write operation of data indicating the correspondence.
The information processing program according to appendix 14.

1 information processing apparatus 1a storage unit 1b control unit 2 storage device 3a, 3b, 3c data group 4 transfer control information 4a, 4a1 operation information

Claims (10)

A storage unit for storing a plurality of data groups respectively classified for each transfer unit to the external storage device, and transfer control information in which one or more operation information corresponding to any of the plurality of data groups are registered; And a control unit,
The control unit
When a first operation is performed on a first data group among the plurality of data groups, the first operation information corresponding to the first data group is registered in the transfer control information;
When the first data group is extracted as a transfer target, the first operation information corresponding to the first data group is temporarily held as holding information, and the first data group is externally transmitted. Transfer to the storage device,
If a second operation is performed on the first data group while the first data group is being transferred to the external storage device, the first operation information registered in the transfer control information is updated;
When transfer of the first data group to the external storage device is completed, the first operation information registered in the transfer control information is compared with the held information;
When the first operation information does not match the held information, the first data group corresponding to the first operation information is retransferred to the external storage device.
Information processing device. In the registration of the first operation information, a predetermined count value is set in the first operation information,
In the update of the first operation information, the count value is updated,
In the comparison between the first operation information and the held information, the count value included in the first operation information is compared with the count value included in the held information.
An information processing apparatus according to claim 1. In the registration of the first operation information, a first operation content indicating the first operation is set in the first operation information,
In the transfer of the first data group, data extracted from the first data group is transferred to the external storage device based on the first operation content set in the first operation information. ,
In the update of the first operation information, a second operation content indicating the second operation is set in the first operation information,
In the retransfer of the first data group, data extracted from the first data group is transferred to the external storage device based on the second operation content set in the first operation information. Do,
The information processing apparatus according to claim 1. In the update of the first operation information, the second operation content is additionally set to the first operation information,
In the retransfer of the first data group, when the first operation content and the second operation content are set in the first operation information, the first operation content is set based on the first operation content. Transferring data extracted from one data group and data extracted from the first data group based on the second operation content to the external storage device;
The information processing apparatus according to claim 3. In retransfer of the first data group, the first operation information registered in the transfer control information is temporarily held as the holding information,
Further, when the retransfer of the first data group is completed, the control unit compares the first operation information registered in the transfer control information with the stored information, and the first operation information Deleting the first operation information from the transfer control information when the stored information matches
The information processing apparatus according to any one of claims 1 to 4. The operation information is acquired from the transfer control information in the order of registration.
The information processing apparatus according to any one of claims 1 to 5. Each of the plurality of data groups is a management data group related to a file for which writing is requested from the external information processing apparatus to the external storage apparatus,
In the first operation and the second operation, the external information processing apparatus is requested to write the first file corresponding to the first data group from the external information processing apparatus, and then the external information processing apparatus completes the write processing. It is an operation that is performed stepwise on the management data group corresponding to the first file until it responds.
The information processing apparatus according to any one of claims 1 to 6. The control unit further divides data of each file requested to be written from the external information processing apparatus and assigns the divided data to divided areas, and stores the divided data respectively included in the divided areas in a state where duplication is excluded. Store in the department,
The management data group corresponding to the first data group includes the division area generated by dividing the first file and the storage unit of the division data included in each of the generated division areas. Data including a correspondence relationship with a storage location, and metadata of the first file,
The first operation is a write operation of the metadata, and the second operation is a write operation of data indicating the correspondence.
The information processing apparatus according to claim 7. The control unit further divides data of a file for which writing is requested from the external information processing apparatus to the external storage device, assigns the divided data to divided areas, and eliminates duplication of divided data respectively included in the divided areas. Stored in the storage unit in the
In storing the divided data, divided data groups generated by grouping one or more pieces of divided data are stored in the storage unit as one of data groups included in the plurality of data groups, respectively.
The information processing apparatus according to any one of claims 1 to 6. On the computer
The plurality of data groups stored in the storage unit when the first operation is performed on the first data group among the plurality of data groups stored in the storage unit and classified according to transfer units to the external storage device The first operation information corresponding to the first data group is registered in transfer control information in which one or more operation information corresponding to any of the data groups are registered,
When the first data group is extracted as a transfer target, the first operation information corresponding to the first data group is temporarily held as holding information, and the first data group is externally transmitted. Transfer to the storage device,
If a second operation is performed on the first data group while the first data group is being transferred to the external storage device, the first operation information registered in the transfer control information is updated;
When transfer of the first data group to the external storage device is completed, the first operation information registered in the transfer control information is compared with the held information;
When the first operation information does not match the held information, the first data group corresponding to the first operation information is retransferred to the external storage device.
An information processing program that executes processing.

Images