JP2013003883A - Storage device, storage method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To add a new data format without greatly modifying a duplication exclusion type storage device, and to prevent a single fault point from being generated.SOLUTION: The storage device includes a duplication exclusion control part which extracts at least partial data from file data as real data according to a data format of file data, generates a stub file by replacing the part of real data of the file data with a storage destination address of the real data of the storage device, and adding an identifier indicative of the data format to the file data, and stores the file in the storage device; and a duplication data management part which holds a digest value of the data stored in the storage device and a storage destination address of the data in the storage device, and determines whether a digest value matching the digest value of the real data is held.

Description

  The present invention relates to a storage apparatus, a storage method, and a program, and more particularly, to a deduplication type storage apparatus, a storage method, and a program that eliminate storing files redundantly.

  In a storage device that centrally stores data generated by a plurality of computing terminals, at the stage of storing data in a physical storage medium such as a hard disk drive, it is determined whether or not data is already stored, A method called deduplication that reduces physical recording capacity by recording only pointer information to already stored duplicate data without storing data in the storage medium for duplicate data It has been known.

  Deduplication determines duplication with already stored data in units of files or in units of physical data blocks that are fixedly allocated when data is stored in a storage medium on the file system. When duplication is judged, digest data with a small size of several tens to several hundreds of bits generated by hash functions such as SHA1 (Secure Hash Algorithm 1) and MD5 (Message Digest 5) used in digital authentication are compared. By doing so, it is determined whether or not the files or data blocks are configured by the same byte string.

  By adopting such a duplication determination method using digest data, it is possible to reduce the processing cost required for the duplication determination executed on the storage apparatus. In particular, in storage processing where high-speed I / O processing is expected to be performed, it is possible to suppress degradation in performance of I / O processing by performing duplication determination at the same time as I / O processing.

  As described above, the deduplication storage apparatus having digest data as a duplication determination means is widely used for the purpose of reducing the data storage cost, particularly in a computing environment in which many files and data blocks composed of the same byte sequence are expected. It has been adopted. For example, the deduplication storage apparatus is used in, for example, a storage apparatus for storing backup data and a storage apparatus for storing image data of system parts of a plurality of virtual OSs.

  As an example of a deduplication storage device, Patent Document 1 describes a storage system that eliminates duplicate storage and reduces the data capacity.

JP 2010-204970 A

  The following analysis was made by the present inventors.

  In the above-described deduplication storage apparatus, there is a single data duplication determination unit when determining the duplication of stored data. That is, duplication determination is performed for each fixed single data unit such as a file unit or a fixed block unit.

  On the other hand, instead of performing duplication determination in fixed units such as file units and fixed block units, by changing the data division method used for duplication determination according to the type of specific file or data format, A method for extracting more duplicate data is known.

  As described above, in the deduplication storage, by using various duplication determination units, it is possible to detect data that is highly likely to be duplicated without omission.

  By the way, in the case of a system that changes the data division method used for duplication determination according to the type of data format, the data division method, data configuration information, and divided data used for data restoration after deduplication The storage location information management mechanism must be prepared for each data format and centrally managed. Therefore, when a data format to be deduplicated is newly added, it cannot be incorporated into an existing data management mechanism, and there is a problem that it is necessary to provide a plurality of management mechanisms. In addition, when data is centrally managed in this way, all data after deduplication cannot be restored due to loss of management data, and the data management part has a single point of failure (SPOF: Single Point of Failure). There is also a problem of becoming. For example, according to the storage system described in Patent Document 1, the divided data is centrally managed by the metadata storage unit, and the metadata storage unit becomes a single failure point.

  In view of this, it is an object of the deduplication storage apparatus to be able to add a new data format without significantly modifying the apparatus and to prevent a single failure point from occurring. An object of the present invention is to provide a storage apparatus, a storage method, and a program for solving such a problem.

The storage apparatus according to the first aspect of the present invention is:
File data to be stored in the storage device is input, and at least a part of the data is extracted as real data from the file data according to the data format of the file data, and the real data portion of the file data is extracted from the file data. A deduplication controller that replaces the storage address of the actual data in the storage device and generates a stub file in which an identifier indicating the data format is added to the file data, and stores the generated stub file in the storage device;
Duplicate data that holds the digest value of the data stored in the storage device and the storage destination address of the data in the storage device and determines whether or not the digest value that matches the digest value of the actual data is held And a management unit.

The storage method according to the second aspect of the present invention is:
A step of inputting file data to be stored in the storage device, and extracting at least a part of the data as actual data from the file data according to the data format of the file data;
Determining whether the digest value of the actual data matches the digest value of the data stored in the storage device;
Replacing the real data portion of the file data with a storage address of the real data in the storage device and generating a stub file in which an identifier indicating the data format is added to the file data;
Storing the stub file in the storage device.

The program according to the third aspect of the present invention is:
A process for inputting file data to be stored in a storage device and extracting at least a part of the data as actual data from the file data according to the data format of the file data;
A process of determining whether or not the digest value of the actual data matches the digest value of the data stored in the storage device;
Processing to replace the portion of the actual data in the file data with a storage address of the actual data in the storage device and to generate a stub file in which an identifier indicating the data format is added to the file data;
And causing the computer to execute processing for storing the stub file in the storage device.
The program can be recorded on a computer-readable storage medium and can be a program product. Further, a non-transient medium can be used as the storage medium.

  According to the storage apparatus, the storage method, and the program according to the present invention, in the deduplication storage apparatus, a new data format can be added without significantly modifying the apparatus, and a single point of failure does not occur. can do.

It is a block diagram which shows the structure of the storage system provided with the storage apparatus which concerns on one Embodiment as an example. It is a block diagram which shows as an example the structure of the deduplication management part in the storage apparatus which concerns on one Embodiment. It is a figure which shows the example of a data structure inside the file data used as the deduplication process object. It is a figure which shows the example of a data structure inside a stub file. It is a figure which shows the example of a division | segmentation of the file data used as a duplication exclusion process target. It is a figure which shows the example of a data structure inside a stub file.

  First, the outline of the present invention will be described. Note that the reference numerals of the drawings attached to this summary are merely examples for facilitating understanding, and are not intended to limit the present invention to the illustrated embodiment.

  The storage apparatus according to the present invention does not centrally manage information used for data restoration after deduplication, that is, data division method, data configuration information, and storage destination information of divided data, instead of centralized management by the system. This is a deduplication storage-compatible storage apparatus provided with means for managing the distribution in units.

  Further, the storage apparatus of the present invention can manage the above information within a single framework even when the data division method is changed according to the data format.

Furthermore, the storage apparatus of the present invention, when performing duplication determination of file data to be deduplicated storage target, data dividing means for changing the data dividing method for each file format,
For each file, file configuration management information required when restoring the divided data as file data, storage address information of the data after division, and information including an identifier indicating how the file was divided, Data management means for storing as a stub file described below instead of the file data body;
When reading the deduplicated stored file data, the file structure management information, the storage address information of the divided data, and the identifier indicating how the file was divided are read from the stub file data, Data restoration means for reading out the data constituting the file data from the storage and restoring the data as file data.

  In addition, when storing data, the storage device of the present invention stores duplication determination means for confirming whether data duplication has already occurred with data already stored, and stores data only when no duplication of data has occurred. Deduplication storage means. A stub file refers to a file that stores a group of metadata for restoring the file data body, not the file data body.

  1 to 4, the storage apparatus (20) of the present invention receives file data to be stored in the storage device (25) as an input, and at least a part of the file data according to the data format of the file data. Is extracted as real data (see FIG. 3), the real data portion of the file data is replaced with the storage address of the real data in the storage device, and an identifier (deduplication is shown) indicating the data format. A stub file (see FIG. 4) in which the method identifier is added to the file data is generated, and the generated stub file is stored in the storage device (25), and the deduplication control unit (103) is stored in the storage device (25). Data digest value and storage device for the data Holds the storage address in 25), comprising the duplicate data management unit determines whether to hold the digest value matches the digest value of the real data (104), the.

  If the duplicate data management unit (104) holds the digest value that matches the digest value of the actual data, the duplicate data management unit (104) sets the storage destination address of the data corresponding to the matched digest value in the storage device (25). The deduplication control unit (103) is notified, and the deduplication control unit (103) generates the stub file by replacing the real data portion of the file data with the notified storage destination address. May be.

  Further, when the duplicate data management unit (104) does not hold a digest value that matches the digest value of the actual data, the duplicate data management unit (104) notifies the duplicate elimination control unit (103) to that effect, and receives the notified duplicate. The exclusion control unit (103) stores the actual data in the storage device (25), replaces the actual data portion of the file data with the storage destination address, generates the stub file, and stores the stored data The destination address may be notified to the duplicate data management unit (104), and the duplicate data management unit (104) may hold the storage destination address as the storage destination address in the storage device (25) of the actual data.

  Further, upon receiving a stub file read request, the deduplication control unit (103) reads the actual data from the storage device (25) by referring to the storage destination address included in the stub file, and The storage address part is replaced with the actual data, the identifier included in the stub file is deleted, the stub file is restored to the original data format, and the restored file data is sent to the request source. You may make it do.

  According to the present invention, even when a data division method corresponding to a plurality of data formats is adopted, division by deduplication is achieved by using a single data management method using a stub file to be saved instead of the file data body. It is not necessary to change the basic management method of the collected data. Therefore, even when the corresponding data format is newly expanded, it is possible to expand the data format within a single data management framework using a stub file, and it is not necessary to modify the system on a large scale.

  In addition, according to the present invention, various types of information necessary for restoration can be distributed and managed as stub files for each file. Therefore, unlike the case where various types of information are centrally managed, it is possible to avoid a situation where the centralized management portion becomes a single failure point and all data cannot be restored. In addition, the load at the time of data reading can be reduced by distributed management.

In the present invention, the following modes are possible.
[Form 1]
The storage device according to the first aspect is as described above.
[Form 2]
When the duplicate data management unit holds a digest value that matches the digest value of the actual data, the duplicate data management unit stores the storage destination address of the data corresponding to the matched digest value in the storage device to the deduplication control unit. The deduplication control unit may generate the stub file by replacing the real data portion of the file data with the notified storage destination address.
[Form 3]
When the duplicate data management unit does not hold a digest value that matches the digest value of the actual data, the duplicate data management unit notifies the deduplication control unit to that effect, and the deduplication control unit Storing in the storage device, replacing the real data portion of the file data with the storage destination address to generate the stub file, and notifying the duplicate data management unit of the storage destination address; The data management unit may hold the storage destination address as a storage destination address of the actual data in the storage device.
[Form 4]
When the deduplication control unit receives a stub file read request, the deduplication control unit reads the actual data from the storage device with reference to the storage destination address included in the stub file, and reads the storage destination address portion in the stub file. In addition to the replacement with the actual data, the identifier included in the stub file may be deleted, the stub file may be restored to the file data of the original data format, and the restored file data may be sent to the request source. .
[Form 5]
The storage device may further include the storage device.
[Form 6]
The storage method according to the second aspect is as described above.
[Form 7]
In the above storage method, when the digest value of the actual data matches the digest value of the data stored in the storage device, the actual data portion of the file data in the stub file generation step May be replaced with a storage destination address in the storage device of data corresponding to the matched digest value.
[Form 8]
In the above storage method, if the digest value of the actual data and the digest value of the data stored in the storage device do not match, storing the actual data in the storage device;
Replacing the actual data portion of the file data with a storage destination address of the actual data in the storage device.
[Form 9]
In the above storage method, upon receiving a stub file read request, referring to a storage destination address included in the stub file, reading the actual data from the storage device;
Replacing the portion of the storage address in the stub file with the actual data, deleting the identifier included in the stub file, and restoring the stub file to file data of the original data format;
And sending the restored file data to the request source.
[Mode 10]
The program is related to the third viewpoint.
[Form 11]
In the above program, when the digest value of the actual data matches the digest value of the data stored in the storage device, in the stub file generation process, the portion of the actual data in the file data is You may make it replace with the storage destination address in the said storage device of the data corresponding to the matched digest value.
[Form 12]
In the above program, when the digest value of the actual data and the digest value of the data stored in the storage device do not match, a process of storing the actual data in the storage device;
You may make it make a computer perform the process which replaces the part of the said actual data of the said file data with the storage destination address in the said storage device of the said actual data.
[Form 13]
In the above program, when a read request for a stub file is received, a process of reading actual data from the storage device with reference to a storage destination address included in the stub file;
A process of replacing the storage address part in the stub file with the actual data, deleting an identifier included in the stub file, and restoring the stub file to file data of an original data format;
A process for sending the restored file data to the request source may be executed by a computer.

(Embodiment)
A storage apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating an example of a configuration of a storage system including a storage apparatus according to the present embodiment. Referring to FIG. 1, the storage system includes a client 1, a storage device 20, and a management client 30. These devices are connected to each other via the network 10.

  The client 1 includes a network file system client 2. Data access from the client 1 to the storage apparatus 20 via the network 10 is executed via the network file system client 2. The data access is executed using an industry standard protocol such as NFS (Network File System) or CIFS (Common Internet File System).

  Referring to FIG. 1, the storage apparatus 20 includes a network file system server 21, a filter unit 22, a deduplication management unit 23, a file system 24, a storage device 25, and a management console 26.

  The network file system server 21 mediates data processing such as data writing and reading in the storage apparatus 20 based on a data access request from the network file system client 2 of the client 1.

  The filter unit 22 transfers, to the deduplication management unit 23, a request for file data stored in the deduplication among data read requests included in the data access request. The filter unit 22 holds a list of file data stored in a deduplication manner. The filter unit 22 transfers the request to the deduplication management unit 23 when an entry of the file data to be read designated by the request exists in the list. On the other hand, if the request is a request other than a read process or a request for a read process for file data whose entry does not exist in the list, the filter unit 22 transfers the request to the file system 24.

  The deduplication management unit 23 manages and controls deduplication storage.

  The file system 24 performs storage management of data in the storage device 25, and reading and writing of data.

  The storage device 25 includes a plurality of physical storage media such as HDDs (hard disk drives) and stores data permanently.

  The management console 26 receives a command command from the outside of the storage apparatus 20 and transfers it to the deduplication management unit 23.

  The management client 30 accesses the management console 26 of the storage apparatus 20 via the network 10 and transmits various command execution instructions to the deduplication management unit 23.

  FIG. 2 is a block diagram illustrating an example of the configuration of the deduplication management unit 23 in the storage apparatus 20 according to the present embodiment. Referring to FIG. 2, the deduplication management unit 23 includes a file access unit 101, a command execution unit 102, a deduplication control unit 103, and a duplicate data management unit 104. The deduplication control unit 103 includes a plurality of (for example, N) deduplication units.

  The file access unit 101 reads file data from the file system 24 and writes file data to the file system 24 in response to a request from the deduplication control unit 103.

  The deduplication control unit 103 includes a plurality of deduplication control units 1 to N, and each deduplication control unit executes data deduplication processing and data restoration processing for the file format in charge.

  The duplicate data management unit 104 tabulates and stores the digest value of data stored in the storage device 25 via the file system 24, the address information of the data storage destination, and the number of times of data duplication. Further, the duplicate data management unit 104 determines whether or not the data subject to deduplication storage processing is duplicated with the data stored in the storage device 25 via the file system 24 based on the tabulated data. judge.

<Deduplication storage>
First, the procedure in which the storage apparatus 20 stores data while eliminating duplicate data will be described with reference to FIGS. Here, as an example, the data that is normally stored in the storage device 25 without deduplication is specified as a file data group to be deduplicated and stored by an instruction from the management client 30 via the management console 26. A case in which the deduplicated file is stored will be described.

  The management client 30 registers a list in which the data storage destination path of the file data to be deduplicated stored is registered in the command execution unit 102 of the deduplication management unit 23 via the management console 26, and stores the deduplication storage. Issue an instruction. In response to the deduplication storage command, the command execution unit 102 scans the list of registered file data, and sequentially stores the entries of the file data registered in the list.

  The command execution unit 102 extracts the extension given to the file name from the storage path registered in the file data entry, and from among the N deduplication units provided in the deduplication control unit 103, A deduplication unit corresponding to the extension is selected, and path information and a deduplication storage command are issued.

  The deduplication control unit 103 reads the corresponding file data stored in the storage device 25 managed by the file system 24 from the path information based on the path information and the deduplication command, via the file access unit 101.

  FIG. 3 is a diagram illustrating an internal data configuration example of file data to be subjected to deduplication processing. As shown in FIG. 3, file data is data that can be copied to other file data by itself, such as text data, image data, and video data, or data that can be extracted as single file data (hereinafter referred to as “file data”). "Real data") and metadata that is given by the application when the file is generated and controls display control of the file. The deduplication control unit 103 includes N deduplication units each corresponding to an extension in order to analyze the configuration information inside the file for each extension.

  The deduplication control unit 103 analyzes the structure of the file data and extracts all the actual data from the file data. The determination as to whether or not there is an overlap is performed on the actual data.

  The deduplication control unit 103 calculates the digest value of the actual data using a common hash function (SHA-1, HD5, etc.) that calculates the digest value of the data, and transmits the digest value to the duplicate data management unit 104. .

  The duplicate data management unit 104 scans the entries in the table managed by the duplicate data management unit 104 and confirms whether there is an entry that matches the digest value.

"If there is no entry that matches the digest value"
If there is no matching entry, the duplicate data management unit 104 registers the digest value as a new entry in the table, assuming that there is no data duplication, and responds to the deduplication control unit 103 that there is no duplication.

  After receiving the response that there is no duplication, the deduplication control unit 103 newly creates actual data corresponding to the digest value as file data in the file system 24 via the file access unit 101, and creates the actual data as a storage device. 25.

  After storing the actual data, the deduplication control unit 103 transmits the storage destination address information of the file system 24 to the duplicate data management unit 104 together with the digest value.

  Based on the data, the duplicate data management unit 104 registers the storage destination address information in the corresponding digest value entry, sets the number of duplicates to 1, and completes registration of the entry in the deduplication control unit 103. Notify that.

"When there is an entry that matches the digest value"
On the other hand, if there is an entry that matches the digest value transmitted from the deduplication control unit 103, the duplicate data management unit 104 determines that there is data duplication, and the data storage destination address registered in the entry Information is transmitted to the deduplication control unit 103, and the number of duplicate entries is increased by one.

  FIG. 4 is a diagram showing an example of the data structure inside the stub file. Referring to FIG. 4, after the digest value match confirmation processing in the duplicate data management unit 104 is completed for all the actual data extracted from the file data, the duplicate elimination control unit 103 has the configuration shown in FIG. In the file data, the actual data part is replaced with the storage destination address information of the file system 24, and the identifier (eg, number) of the corresponding deduplication unit of the deduplication control unit 103 is added to the head of the file data as a deduplication method identifier. And the processed file data (that is, the stub file) is overwritten on the file data to be deduplicated stored in the file system 24.

  Further, the deduplication control unit 103 registers the file identifier of the stub file in the filter unit 22. Here, the file identifier is an identifier that can uniquely identify a file in the file system 24 of the storage apparatus 20.

  By executing the above process for all file data registered as a list in the command execution unit 102, the deduplication storage process is completed.

"When deduplicating and storing file data whose structure cannot be analyzed"
When the N deduplication units included in the deduplication control unit 103 are file data whose structure cannot be analyzed, for example, when they do not match the corresponding extension, or when the file data has no extension As shown in FIG. 5, data is divided into fixed-length or variable-length blocks from the beginning, and deduplicated and stored. In this case, as shown in FIG. 6, the stub data includes a deduplication scheme identifier and a set of data storage destination address information.

<Read data>
Next, a procedure for reading the deduplicated stored file data by a read request from the client 1 will be described.

  First, the client 1 transmits an identifier of a file to be read and a read request to the storage apparatus 20 via the network file system client 2.

  The network file system server 21 of the storage apparatus 20 transmits a file identifier and a data read request to the file system 24 via the filter unit 22 based on the file identifier and the read request.

  The filter unit 22 scans whether the file identifier is registered in the list of file data stored in a deduplication manner.

  If the file identifier is not registered in the list, the filter unit 22 transmits a data read request and the file identifier to the file system 24. Thereafter, the file data is returned to the client 1 through a data reading procedure from a general file system.

  On the other hand, when the file identifier is registered in the list, the filter unit 22 transmits a data read request and the file identifier to the deduplication management unit 23.

  Based on the data read request and the file identifier, the deduplication control unit 103 of the deduplication management unit 23 receives the corresponding file data (in this case, the deduplication storage target) from the file system 24 via the file access unit 101. Therefore, it corresponds to the data of the stub file).

  The deduplication control unit 103 selects a corresponding deduplication unit from the N deduplication units according to the deduplication method identifier in the stub file data.

  The deduplication control unit 103 analyzes the data structure in the stub file corresponding to FIG. 4 or FIG. 6, and based on the data storage destination address, all the file data recorded in the stub file are converted into the file access unit. Read out from the file system 24 via 101.

  The deduplication control unit 103 replaces the read file data as actual data with the data at the data storage destination address part in the stub file, deletes the deduplication method identifier, and restores the file data before the deduplication storage process. . Further, the deduplication control unit 103 transmits the restored file data to the filter unit 22.

  The filter unit 22 transmits file data to the network file system server 21. The file data is transmitted to the client 1 via the network 10. Thus, the file data reading process is completed.

  The embodiment of the present invention has been described in detail above with reference to the drawings. However, the specific configuration for carrying out the present invention is not limited to the above-described configuration, and various design changes can be made without departing from the scope of the present invention.

  For example, when the storage apparatus 20 has a computer system therein and the operations of the above-described processing units are stored in a computer-readable recording medium in the form of a program, the computer reads and executes the program. As a result, the above processing may be performed. Here, examples of the computer-readable recording medium include a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, and a semiconductor memory. Further, the computer program may be distributed to a computer via a communication line, and the computer that has received the distribution may execute the program.

  The program may be for realizing a part of the functions described above. Furthermore, the program may be a so-called difference file (difference program) that realizes the above-described function in combination with a program already recorded in the computer system.

  It should be noted that the disclosures of prior art documents such as the above patent documents are incorporated herein by reference. Within the scope of the entire disclosure (including claims) of the present invention, the embodiment can be changed and adjusted based on the basic technical concept. Various combinations and selections of various disclosed elements are possible within the scope of the claims of the present invention. That is, the present invention of course includes various variations and modifications that could be made by those skilled in the art according to the entire disclosure including the claims and the technical idea.

DESCRIPTION OF SYMBOLS 1 Client 2 Network file system client 10 Network 20 Storage apparatus 21 Network file system server 22 Filter part 23 Deduplication management part 24 File system 25 Storage device 26 Management console 30 Management client 101 File access part 102 Command execution part 103 Deduplication control part 104 Duplicate data management department

Claims (10)

File data to be stored in the storage device is input, and at least a part of the data is extracted as real data from the file data according to the data format of the file data, and the real data portion of the file data is extracted from the file data. A deduplication controller that replaces the storage address of the actual data in the storage device and generates a stub file in which an identifier indicating the data format is added to the file data, and stores the generated stub file in the storage device;
Duplicate data that holds the digest value of the data stored in the storage device and the storage destination address of the data in the storage device and determines whether or not the digest value that matches the digest value of the actual data is held And a management unit.   When the duplicate data management unit holds a digest value that matches the digest value of the actual data, the duplicate data management unit stores the storage destination address of the data corresponding to the matched digest value in the storage device to the deduplication control unit. 2. The notification according to claim 1, wherein the deduplication control unit generates the stub file by replacing the real data portion of the file data with the notified storage address. Storage device.   When the duplicate data management unit does not hold a digest value that matches the digest value of the actual data, the duplicate data management unit notifies the deduplication control unit to that effect, and the deduplication control unit Storing in the storage device, replacing the real data portion of the file data with the storage destination address to generate the stub file, and notifying the duplicate data management unit of the storage destination address; The storage apparatus according to claim 1 or 2, wherein the data management unit holds the storage destination address as a storage destination address of the actual data in the storage device.   When the deduplication control unit receives a stub file read request, the deduplication control unit reads the actual data from the storage device with reference to the storage destination address included in the stub file, and reads the storage destination address portion in the stub file. The stub file is replaced with the actual data, the identifier included in the stub file is deleted, the stub file is restored to the original data format, and the restored file data is sent to the request source. The storage apparatus according to any one of claims 1 to 3.   The storage apparatus according to any one of claims 1 to 4, further comprising the storage device. A step of inputting file data to be stored in the storage device, and extracting at least a part of the data as actual data from the file data according to the data format of the file data;
Determining whether the digest value of the actual data matches the digest value of the data stored in the storage device;
Replacing the real data portion of the file data with a storage address of the real data in the storage device and generating a stub file in which an identifier indicating the data format is added to the file data;
Storing the stub file in the storage device.   When the digest value of the actual data and the digest value of the data stored in the storage device match, in the stub file generation step, the portion of the actual data of the file data is matched with the digest value The storage method according to claim 6, wherein the data is replaced with a storage destination address in the storage device. If the digest value of the actual data does not match the digest value of the data stored in the storage device, storing the actual data in the storage device;
The storage method according to claim 6, further comprising a step of replacing the real data portion of the file data with a storage destination address of the real data in the storage device. When receiving a read request for a stub file, referring to a storage destination address included in the stub file, reading actual data from the storage device;
Replacing the portion of the storage address in the stub file with the actual data, deleting the identifier included in the stub file, and restoring the stub file to file data of the original data format;
The storage method according to claim 6, further comprising a step of sending the restored file data to a request source. A process for inputting file data to be stored in a storage device and extracting at least a part of the data as actual data from the file data according to the data format of the file data;
A process of determining whether or not the digest value of the actual data matches the digest value of the data stored in the storage device;
Processing to replace the portion of the actual data in the file data with a storage address of the actual data in the storage device and to generate a stub file in which an identifier indicating the data format is added to the file data;
A program for causing a computer to execute processing for storing the stub file in the storage device.

Images