JPH11272575A - Compression file server system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce server and transfer loads to the utmost by compressing a file for read and write by a unit of division, retaining it in the server and performing all of division, compression and development at a client. SOLUTION: Each file is compressed and retained in a file retaining part 12 of a file server 1 after it is divided into plural pieces and a divided file index is provided in the file retaining part 12 of the file server 1 so that access is made to the file by the unit of division. Writing and reading by the unit of division are performed corresponding to an access request from the client 2 by a file retrieving and access means 11. Each client 2 is provided with a file compressing, developing and dividing means 21 to transmissibly read and write the file of the file server 1, to divide, to compress and to develop the file for an application 22. A server load is suppressed to divide, to compress and to develop the file at the client 2, data to be transferred on a network 3 is required compressed data by the unit of division, with a small amount of excess data and the transfer load on the network 3 is also suppressed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a compressed file server system, and in particular, a client divides, compresses, and decompresses a file stored in a server, and adds an index to the server so that the compressed data can be searched in divided units. The present invention relates to a compressed file server system in which a load on a server and a network is reduced by storing the compressed file server.

[0002]

2. Description of the Related Art In recent years, in a computer environment where data needs to be unified and centralized, large-scale disk devices connected to a network by a server-client system are commonly used. In such a system configuration, since the operation of each client only reads and writes a file from / to the server in response to a request, it is considered that this processing does not significantly increase the load on each client. . On the other hand, since the server and the network receive access from a plurality of clients at random, it is assumed that the load is concentrated, and reducing the load required for each access is an important issue.

[0003] In order to reduce the load on the network due to the file transfer, a method of transmitting data after compressing it on the transmitting side and decompressing it on the receiving side and expanding it to the original data has conventionally been performed. In addition to this, JP-A-5-3
Japanese Patent Application Publication No. 13984 proposes a file transfer processing method capable of reducing the load of compression and decompression on a server and also reducing the storage capacity required for storing files. In this method, the file compressed and transferred from the client is stored in the server as compressed data.
There is an advantage that the load of decompression (at the time of storage) and compression (at the time of read transfer) of the compressed data on the server can be reduced together with the reduction of the storage capacity.

[0004]

However, in the conventional file server including the method disclosed in Japanese Patent Application Laid-Open No. Hei 5-313984, reading and writing of files are performed on a file-by-file basis. Increases the capacity of data transferred on the network for one file access, and also increases the load on the server. The purpose of file access that occurs in a client is various, and not only does the file need not always require the entire file but often only a part of the file is sufficient. Therefore, the compressed data is stored in the server as it is by employing the compressed transfer of the file.
Even in the method of 13984, an unnecessary portion is often transferred at the time of file access for reading and writing on a file basis, and the effect of reducing the network load is not sufficient, and a problem to be improved remains. .

An object of the present invention is to divide a file and store it in a compressed data format in a server so that the file can be accessed in units of division, and to divide, compress and decompress the file on the client, thereby reducing the load on the server and the network. Another object of the present invention is to provide a compressed file server system in which the reduction of the number of files is more effective.

[0006]

According to a first aspect of the present invention, there is provided a compressed file server system, comprising: a file storage unit for storing each file in a divided compression format having an index for each division unit in a file server provided on a network; File search access means for performing read / write processing of a compressed file in a required division unit in response to an access request from each client and transmitting / receiving the compressed client to / from the corresponding client via the network; File compression / expansion / division means for dividing and compressing a file, adding index information for each division unit, transferring the file to the file server, and expanding the compressed file transferred from the file server.

According to a second aspect of the present invention, there is provided the compressed file server system according to the first aspect, wherein the index information given by the file compression / expansion / division means for each division unit of each file is the division unit before compression. Is characterized by the relative position information in the file before division compression where the head of the file is located and the data length of the division unit before compression.

According to a third aspect of the present invention, there is provided the compressed file server system according to the first or second aspect, wherein the file compression / expansion / division unit is configured to update one file unit to be updated when updating the file. When the data volume of exceeds the predetermined upper limit of division,
The post-update data is re-divided, compression processing is performed for each of the divided units after the re-division, new index information is added to each of the divided units, transferred to the file server, and replacement with one of the divided units before the update is requested. It is characterized by having a subdivision function.

According to a fourth aspect of the present invention, there is provided the compressed file server system according to the third aspect, wherein the file compression / expansion / division means is configured to reduce an updated data amount of one division unit to be updated when updating the file. When the predetermined division lower limit value is decreased and divided, the compressed file of any one of the division units before and after the division unit is obtained from the file server, decompressed, and combined with the updated data to merge the division units. After that, it has a merging function of performing compression processing, adding new index information, transferring it to the file server, and requesting replacement with two division units before updating.

According to a fifth aspect of the present invention, there is provided the compressed file server system according to the third aspect, wherein the file server monitors a load state and stores the load state in the file storage unit when the load is smaller than a specified value. A first division for retrieving a divided unit having a size of a compressed file smaller than a predetermined lower limit, performing expansion and compression, merging the divided unit with one of the preceding and following divided units, and reorganizing the divided unit. It is characterized by having a file adjusting means.

According to a sixth aspect of the present invention, there is provided the compressed file server system according to the first or second aspect, wherein the file server monitors a load state and stores the file when the load is smaller than a prescribed value. A search is made for a division unit in which the size of the compressed file stored in the copy unit is smaller than a predetermined lower limit value and a division unit larger than a predetermined upper limit value, and the unit is expanded and compressed, and a division unit smaller or smaller than the lower limit value is searched. A second divided file adjusting means for merging with the divided unit and redividing the divided unit larger than the upper limit to reorganize the divided unit is provided.

[0012]

Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of the first embodiment of the present invention. As shown in FIG. 1, the compressed file server method according to this embodiment
And a plurality of clients 2 and a network 3 connecting them. The file server 1 stores a file in a divided compression format in which an index is added for each division unit.
File search access means 11 for writing and reading in division units in response to an access request from
Each client 2 is provided with a file compression / expansion / division unit 21 for reading / writing a file on the file server 1 transparently to the application 22 and dividing, compressing and decompressing the file.

As shown in FIG. 1, the file storage unit 12 divides one file F into a plurality of division units, compresses each of the divided units, and compresses the divided files into divided compressed files F1, F2, F2.
F3 and F4, and in addition to a file index f for specifying the file F, a divided file index f1 for searching the storage position and data length of each of the divided compressed files F1, F2, F3 and F4. f2, f3, f4
Is provided. That is, the file F is provided with an index enabling access for each division unit, and is stored in the divided compression format of the divided compressed files F1 to F4.

The file search access means 11 writes a file to the file storage unit 12 and reads a file from the file storage unit 12, but does not divide, compress, or decompress the file. That is, client 2
The divided and compressed files transferred from the network 3 via the network 3 are divided into divided file indexes f1, f2, f3, f
4 is stored in the compressed format and stored in the compressed format. In addition, the access request from the client 2 refers to the divided file index and reads only the required divided compressed file in the required division unit. Forward.

The file compression / expansion / division means 21 of the client 2 provides a function of enabling the application 22 to transparently process a file in the divided compression format on the file server 1 in the same manner as a normal file. Performs file transfer, division, compression and decompression. That is, in response to a file storage request from the application 22, the file requested to be stored is divided and subjected to compression processing for each division unit, and index information for identifying each division unit is added in addition to the file name. To the file server 1. In response to a file reference and update request from the application 22, the compressed data in the required division unit is acquired from the file server 1, decompressed and decompressed to the original file state, and provided to the application 22. In the case of updating, the updated file is compressed again and transferred.

According to the above configuration, only the compressed file is handled on the file server 1, and only the actually necessary compressed data of the division unit flows through the network 3, and the disk device of the file server 1 The capacity can be used effectively, and the load on the file server 1 and the network 3 can be significantly reduced. Although the load on the client increases due to the division, compression, and decompression of the file, the client often operates alone, and there is no problem with a slight increase in the load.

The operation of the embodiment shown in FIG.
This will be specifically described with reference to FIGS. FIG. 2 is an explanatory diagram of the operation of the file compression / expansion / division means 21 and the file search / access means 11 when the application 22 requests the storage of a newly created file. The newly created 750 KByte file is divided into four parts. 2 until it is stored in the file storage unit 12.

First, the file compression / expansion / division means 21
File F (750 KBy) from application 22
te), when a write request is received, 200 KB
File F into divided files F-1 and F-
2, F-3, F-4, and compress them to generate divided compressed files F1, F2, F3, F4, and add index information S1, S2, S3, S4 to each of them, and The information is transmitted to the file server 1 via the network 3 together with information (file name) and information such as the data length after compression. Here, as the index information S1 to S4, the leading file F of the divided files F-1 to F-4 before compression is used.
The upper position (relative position information) and the data length are used.

The file search access means 11 of the file server 1 stores the divided compressed files F1 to F4 transmitted from the client 2 in a compressed state in the file storage unit 1.
2 in order to respond quickly to the next partial access, in addition to the file index f, the divided file index f1
To f4. The divided file indexes f1 to f4
Uses the index information S1 to S4 transferred together with the divided compressed files F1 to F4 as a search key part, and is associated with the storage positions X1 to X4 of the divided compressed files F1 to F4.

FIG. 3 shows the file compression / expansion / division means 21 when the application 22 requests a file reference.
FIG. 4 is an explanatory diagram of the operation of FIG. From the application 22 of the client 2 to the first 250 KB of the file F
Upon receiving a request to read 00 KB data,
The file compression / expansion / division means 21 transfers the request to the file server 1. In response to this request, the file search access unit 11 of the file server 1 refers to the divided file indexes f1 to f4 of the file F, and stores the corresponding divided compressed file F2 and its index information S2 in the file storage unit 12
And transfers it to the client 2. And
When the file compression / expansion / division means 21 receives the transferred divided compressed file F2 and the index information S2, the compressed data is expanded and expanded into the uncompressed divided file F-2.
50 from the top corresponding to the requested 250 KB byte
The data of 100 KB is extracted from the position of the KB byte, and the extracted data is transferred to the application 22.

The above description is based on the application 22
In the case where the data range requested to be read from is within one division unit, but extends over two division units, the file search access unit 11 of the file server 1
Makes a decision with reference to the index, transfers the corresponding two divided compressed files (for example, F2 and F3) and the index information (S2 and S3) to the client 2, and the file compression / expansion / dividing means 21 Is extracted, and data in the requested range is extracted with reference to the index information and passed to the application 22.

FIG. 4 is a diagram for explaining the operation of the file compression / expansion / division means 21 when the application 22 requests data addition to an existing file, and shows a case where the amount of additional data is small. When the application 22 requests to add 100 Kbytes of data to the existing file F at 250 Kbytes, the file compression / expansion / division means 2
1 is 250 Kbytes of file F in file server 1
Request a split compressed file containing the eyes. In response to this request, the corresponding divided compressed file F2 and index information S2
Is received from the file server 1, expanded and restored to the divided file F-2, and additional data 100 KByte from the application 22 is added to the designated position,
Split file F as combined data of 300 KB
2a is created. Finally, the divided file F-2a is compressed to create a divided compressed file F2a including 100 Kbytes of additional data, and transferred to the file server 1 together with the updated index information S2a.
Request replacement with 2.

FIG. 5 shows the divided compressed file F2a updated and transferred in the processing of FIG.
It is explanatory drawing of the process which replaces with 2. First, the divided compressed file F2 before the update is replaced with the updated divided compressed file F2.
a, and the divided file index f2 for retrieving them is updated to f2a. At the same time that the search key portion of the divided file index f2a is changed to the index information S2a, the information indicating the data length of the divided compressed file is also changed because the size of the divided compressed file F2a has been increased by replacement. Subsequent divided compressed file F3
F4 has no change, but divided file indexes f3a, f4
The information (the position on the pre-compression file and the data length) of the search key part of a is changed so as not to be inconsistent. As a result, FIG.
The hatched part inside is updated.

The case where data is newly added to a predetermined position of an existing file has been described with reference to FIGS. 4 and 5, but the application 22 requests partial reading of the existing file for updating, and The processing when the data amount of the partial file returned later increases or decreases is basically the same as this. However, the process (-) up to the creation of the combined data in FIG. 4 is a process of replacing the extracted data portion with the returned update data after the reference process shown in FIG. The same applies to the case of requesting partial deletion by specifying a range from an existing file.The specified part is deleted from the divided file expanded in the same procedure as in the case of reference, a new divided file is created, and recompression is performed. Transfer to the file server 1 to make a replacement request.

The above description of FIGS. 4 and 5 is based on the case where the size of the divided compressed file is changed by the update but the number of divisions of the file is not changed. Next, FIG. 6 and FIG. This will be described with reference to FIG.

As shown in FIG.
There is a request to add 400 KB data to the KB, and the expanded file F of 200 KB expanded.
It is assumed that 400 KByte is added to 2 and a 600 KByte divided file F-2b is created. Here, 400K is set as the division upper limit value for limiting the size of the division file.
It is assumed that Byte is set, and if the number of bytes exceeds the value by updating, the data is divided. Since the updated divided file F-2b is 600 KByte, it becomes a division target, and is divided into two divided files F-2c and F-2d each of which is 300 KByte. Subsequently, these files are compressed into two divided compressed files F2.
c and F2d, and index information S2c and S2 respectively.
d is added and transferred to the file server 1. These processes are performed by the file compression / expansion / division unit 21 of the client 2.
It is performed by

As shown in FIG. 7, the two divided compressed files F2c and F2d sent to the file server 1 are replaced by the divided compressed file F2 before the update as shown in FIG. Split file index f2 using index information S2c, S2d
It is changed to c, f2d. At the same time, the divided file indexes f3 and f corresponding to the subsequent divided compressed files F3 and F4
4 so that there is no inconsistency in the divided file indexes f3b and f4b.
Is changed to

FIG. 8 is a block diagram showing the configuration of the second embodiment of the present invention. As shown in FIG. 8, the compressed file server system according to the second embodiment includes a file server 1a, a plurality of clients 2, and a network 3 for connecting these clients. The difference from the embodiment is that the file server 1a includes a divided file adjusting unit 13. When the size of the divided compressed file stored in the file storage unit 12 becomes small enough to affect the disk access performance by updating from the client 2, the divided file adjusting unit 13 Perform processing to change to the size of the range. Since this processing involves the expansion and compression of the divided compressed file, the state of the load on the file server 1a is monitored, and the processing is automatically performed when the load is small.

By updating the file from the client 2, the size of the division unit of the file G stored in the file storage unit 12 of the file server 1a is changed to the divided compressed file G2 (30 Kbytes before compression, 10 Kbytes after compression) shown in FIG.
KBByte), the file server 1
It is desirable that the size of the divided compressed file be kept within a certain range, since the disk access performance of a will be degraded. However, the reorganization of the divided compressed file is expanded,
Since the process involves a large load accompanied by compression, the process is not performed when a small-sized divided compressed file is transferred from the client 2, but is stored as it is at that time. After that, the divided file adjusting means 13 monitors the load status of the file server 1, and when the load of the file server 1 a becomes smaller than a certain value, the processing performance is affected through the file search access means 11. Retrieve the smaller divided compressed file and combine it with the immediately preceding or succeeding divided compressed file to perform reorganization.

As a specific operation, when the size before compression becomes smaller than a predetermined lower limit value (for example, 50 Kbytes) as in the divided compressed file G2 in FIG. 9, the divided compressed file G2 is divided immediately after the divided file. Combine with the compressed file G3. For this purpose, the divided compressed files G2, G3
Are decompressed and combined, compression is performed again after combination to create a divided compressed file G3a, the divided compressed file G2 is deleted, and the divided compressed file G3 is divided into divided compressed files G3
Replace with a. Also, with the change in the configuration of the divided compressed files, the divided file indexes g1, g2, g3, and g4 are divided so that the divided file indexes g1, g2a, and g3a are consistent.
Change to Note that the lower limit value of the compressed divided file at the time of reorganization may be set by the size after compression instead of the size before compression.

In the above description of the second embodiment, the file compression / expansion / division means of the client is the same as that of the first embodiment. When the number of files exceeds the limit, the system is provided with a re-segmentation function to re-segment the segmented file. For this reason, the divided file adjusting means only needs to have a function of reorganizing and changing the size of the divided compressed file to a certain range when the size of the divided compressed file in the file storage unit becomes smaller than the lower limit. On the other hand, if the client's file compression / expansion / division means does not have a re-division function,
If the divided file adjusting means of the file server is provided with a function of searching for a divided compressed file exceeding a predetermined upper limit and dividing and reorganizing it after decompression, almost the same effect can be obtained.

Although the divided file adjusting means in the second embodiment monitors the load on the file server and operates automatically when the load is light, the file compression of the client is performed by the maintenance terminal of the file server. Even if the same function as the decompressing / dividing means is provided and the recompression of the divided compressed file is performed periodically, the number of maintenance steps increases, but almost the same effect can be expected as the system performance.

Further, in the first embodiment described above, the file compression / expansion / division means of the client, when the size of the updated divided file exceeds a predetermined division upper limit at the time of updating, the divided file is divided. Although it is assumed that it has a re-segmentation function to re-segment, when the updated data amount of the segment file to be updated decreases when the file is updated and it falls below a predetermined division lower limit, Requests and obtains one of the pre- and post-divided compressed files from the file server, expands and combines them with the updated data, merges the divided units, compresses the merged new divided file, and obtains new index information. It may be provided with a merging function for requesting the file to be transferred to a file server and replaced with two divided compressed files before updating.
With such a configuration, the size of the divided compressed file can always be kept within a certain range without providing the file server with the divided file adjusting means or periodically reorganizing the divided compressed file from the maintenance terminal. Can be kept within.

In the description of the first embodiment, the index information to be added to the divided compressed file and transferred by the file compression / expansion dividing means of the client includes relative position information in the first file of the divided file before compression. And the data length. The index information is not limited to this, and may be relative position information within the file at the beginning and end of the divided file, or may be updated by using both the data length of the entire file and the relative position information at the beginning of the divided file. At times, the data length of the updated divided file may be added and transferred.

[0036]

As described above in detail, according to the compressed file server method of the present invention, a file is divided and compressed, an index is added so that the file can be searched in units of division, the file is stored on a disk of the file server, and the processing load is increased. As a general rule, the split compression and decompression are performed not by the file server but by the client, so that the load on the file server where processing from many clients is concentrated can be reduced, and the disk resources can be effectively used. Furthermore, in response to a request from a client, only data of a required division unit, which has been divided and compressed, is transmitted / received via a network, so that the load on the network can be reduced.

If the file server is provided with a divided file adjusting means having a load monitoring and compression / decompression function, and the reorganization processing of the divided unit is performed when the load on the file server is low, the divided unit stored on the disk is reduced. It is possible to prevent the size of the file server from becoming too small, affecting the access performance, and reducing the performance of the file server.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a first exemplary embodiment of the present invention.

FIG. 2 is an explanatory diagram of a processing procedure when a new file is created.

FIG. 3 is an explanatory diagram of a processing operation on the client side when referring to a file.

FIG. 4 is an explanatory diagram of a basic processing operation on the client side when updating a file.

FIG. 5 is an explanatory diagram of a basic processing operation on the file server side when a file is updated.

FIG. 6 is an explanatory diagram of an operation on the client side when a data increase amount due to updating is large.

FIG. 7 is an explanatory diagram of the operation on the file server side when the data increase amount due to the update is large.

FIG. 8 is a block diagram showing a configuration of a second exemplary embodiment of the present invention.

FIG. 9 is an explanatory diagram of processing of a divided file adjustment unit according to the second embodiment.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1, 1a File server 2 Client 3 Network 11 File search access means 12 File storage unit 13 Divided file adjustment means 21 File compression / expansion division means 22 Application

Claims (6)

[Claims] 1. A file storage unit for storing each file in a divided compression format having an index for each division unit in a file server provided on a network, and a division unit required in response to an access request from each client. A file search access means for performing read / write processing of the compressed file and transmitting / receiving the compressed file to / from the corresponding client via the network, and each client divides and compresses the file and stores index information for each division unit. A compressed file server system comprising: a file compression / expansion / division unit that assigns and transfers the compressed file to the file server and expands a compressed file transferred from the file server. 2. Index information given by the file compression / expansion / division means for each division unit of each file includes relative position information in the file before division compression in which the head of the division unit before compression is located, and relative information in the file before compression. 2. The compressed file server system according to claim 1, wherein the data length of the divided file unit is composed of: 3. The file compression / expansion / dividing means, when the updated data amount of one division unit to be updated at the time of file update increases and exceeds a predetermined division upper limit value, renews the updated data. It has a re-segmentation function that performs compression processing for each of the divided units after division and re-segmentation, assigns new index information to each of them, transfers them to the file server, and requests replacement with one before the update. The compressed file server system according to claim 1 or 2, wherein 4. The file compression / expansion / division means, when a data amount after update of one division unit to be updated is reduced and a predetermined division lower limit value is divided when the file is updated, before and after the division unit. A compressed file of any of the division units is obtained from the file server, expanded, combined with the updated data, merged with the division unit, and then subjected to compression processing, and added with new index information to the file server. 4. The compressed file server system according to claim 3, further comprising a merging function for requesting replacement with two divided units before transferring and updating. 5. The file server monitors a load state and searches for a division unit in which the size of a compressed file stored in the file storage unit is smaller than a predetermined lower limit when the load is smaller than a prescribed value. 4. The compressed file server according to claim 3, further comprising a first divided file adjusting unit that performs expansion, compression, and merges the divided unit with one of the preceding and succeeding divided units to reorganize the divided unit. method. 6. The file server monitors a state of a load, and when the load is smaller than a predetermined value, a size of a compressed file stored in the file storage unit is smaller than a predetermined lower limit and a predetermined division unit. Searches for a division unit larger than the upper limit, expands and compresses it, merges the division unit smaller than the lower limit with one of the previous or next division unit, and re-divides the division unit larger than the upper limit to re-divide the division unit. 3. The compressed file server system according to claim 1, further comprising a second divided file adjusting unit for performing knitting.