KR20170014276A - Variable block processing file system and file variable block processing method based on fixed block - Google Patents

Variable block processing file system and file variable block processing method based on fixed block Download PDF

Info

Publication number
KR20170014276A
KR20170014276A KR1020150107330A KR20150107330A KR20170014276A KR 20170014276 A KR20170014276 A KR 20170014276A KR 1020150107330 A KR1020150107330 A KR 1020150107330A KR 20150107330 A KR20150107330 A KR 20150107330A KR 20170014276 A KR20170014276 A KR 20170014276A
Authority
KR
South Korea
Prior art keywords
block
file
data
change
variable
Prior art date
Application number
KR1020150107330A
Other languages
Korean (ko)
Other versions
KR101729624B1 (en
Inventor
유영준
김병관
고영웅
Original Assignee
한림대학교 산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 한림대학교 산학협력단 filed Critical 한림대학교 산학협력단
Priority to KR1020150107330A priority Critical patent/KR101729624B1/en
Publication of KR20170014276A publication Critical patent/KR20170014276A/en
Application granted granted Critical
Publication of KR101729624B1 publication Critical patent/KR101729624B1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/064Management of blocks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/0643Management of files
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0683Plurality of storage devices

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present invention relates to a variable block processing file system and a file variable block processing method based on a fixed block, and a variable block processing file system based on a fixed block according to the present invention includes a storage medium for storing file data, A file manager for storing the file data in a plurality of fixed blocks if the file is first stored in the storage medium by the application program; A file change detecting unit for storing changed block data in the memory when the file change data is detected by the application program; and, when the file change data is requested to be stored by the application program, Only the index information is changed for the missing block, And a file change processing unit for processing the block having the data change by processing it as a variable block, thereby minimizing the writing process to the storage medium, thereby increasing the usable period of the storage medium.

Description

[0001] The present invention relates to a variable block processing file system and a file variable block processing method based on a fixed block,

The present invention relates to a variable block processing file system and a file variable block processing method based on a fixed block, and more particularly, to a variable block processing method and a variable block processing method, which are based on a fixed block for minimizing a writing process to a storage medium, A file system, and a file variable block processing method.

The file system is responsible for managing various types of files. That is, the file system is responsible for storing the operating system, various executable files, and user data in the computer system. In the user area, the file is regarded as a byte stream, and the file data is read or written through the file handle.

The read / write request for the file in the user area is transferred to the file system of the kernel area, and finally converted to a read / write request for the block of data stored in the block device. Therefore, the byte stream processed in the user area is converted into a block unit at the device level, which causes various problems in the operation on the file.

For example, a case in which a portion of a file is read into a memory buffer of a user area, and then a byte of data is added or deleted in the middle of the buffer will be described. In this case, it is possible to easily change the data while moving the file offset to the area to be changed in the user area and inserting or deleting the new byte. However, if such a change is stored in the file system, . The byte stream present on the memory buffer must ultimately be stored in a block on the disk, and the data change of one byte will require the modification of all subsequent data blocks in succession.

This problem occurs in the same way when insertion or deletion of data occurs in units of blocks as well as insertion or deletion in units of bytes. For example, an application such as Winzip or WinRAR, which is widely used in desktop computers, is a structure in which a plurality of sub-files constitute a single file, and each sub-file has a block-aligned structure I have. Therefore, the process of deleting or inserting a subfile corresponds to a process of inserting or deleting consecutive blocks. However, most file systems handle file modification operations like ordinary files, without regard to operations related to insertion or deletion of such block-level data.

In order to solve the above-described problems, the present inventors have filed a patent application No. 2014-0107590 as a variable block processing file system and method based on a fixed block that enables storage of variable-size file data on a fixed-size block. The application of the variable block in the block unit file system solves the overload problem due to the write operation in the existing file system by changing the index to the data block instead of performing the write operation in the whole block, The elimination of operations has also resulted in a significant improvement in overall system performance. However, a write operation to a permanent storage medium such as a hard disk or a flash memory may not be stored last after completion of a document operation, for example, when a document operation is performed, but may involve a lot of write operations during a document operation.

Patent Document 1: Patent Application No. 2014-0107590

In order to solve the above problems, the present invention provides a variable block processing file system and a file variable block processing method based on a fixed block for minimizing the writing process to a storage medium in a fixed block unit file system capable of variable block processing And to provide the above objects.

According to an aspect of the present invention, there is provided a variable block processing file system based on a fixed block, comprising: a storage medium for storing file data; a memory for storing temporarily changed block data; A file management unit for storing file data in a plurality of fixed blocks if the file is first stored in the storage medium by the application program; and a change management unit for storing the change block data in the memory when the change of the file data by the application program is detected A file change detecting unit for determining whether or not the file change data is requested to be stored by the application program, and for storing the file change data in the memory by using the change block data stored in the memory, To the block with the data change. And a file change processor for processing and storing the variable block.

Wherein the file management unit includes a file management area and a data storage area, and the file management unit stores the fixed block information in the file management area when the file is first stored in the storage medium, Wherein the block change processing unit changes only the index information of the file management area for a block that is not changed among a plurality of fixed blocks stored in the storage medium, The variable block information may be further stored in the management area and the corresponding variable block data may be stored in the data storage area.

The fixed block information includes position information of the data storage area for each block, and the variable block information may include the number of change blocks and the length of file data stored in the change block.

Wherein the file change detection unit classifies one file data to be stored in the storage medium into a predetermined unit size smaller than the block size and extracts a plurality of hash values when the file storage in the storage medium is first performed by the application program, It is possible to detect whether there is a modification of the file data by the application program through the similarity comparison using a plurality of hash values.

Wherein the file change detector stores change block and variable length block information in the memory in response to the change block data, and the file change processor, when the file change data is requested to be stored by the application program, Comparing the changed block stored in the memory and storing at least one block of the file change data as a variable block using information stored in the variable length block if the block matches the change block stored in the memory.

According to another embodiment of the present invention, a variable block processing based on a fixed block including a storage medium for storing file data and a memory for storing temporarily changed block data, The method includes: a file management step of storing file data in a plurality of fixed blocks if the file storage in the storage medium by the application program is the first; and, when a change of file data by the application program is detected, And a file change detection step of, when the file change data is requested to be stored by the application program, using the change block data stored in the memory, Only the index information is changed, and the data And a file change processing step of processing the block having the change by processing it as a variable block and storing it.

According to the above-described configuration, the present invention minimizes the writing process to the storage medium in the file system of the fixed block unit capable of variable block processing, thereby increasing the usable period of the storage medium.

In addition, the present invention makes it possible to store file data of a variable size in a block of a fixed size, so that file storage according to a change of file data can be performed quickly.

In addition, the present invention can extend the lifetime of a flash memory or the like, which is a semiconductor type nonvolatile memory, by minimizing the recording of data in the data area of the storage medium.

1 is a block diagram illustrating an entire block diagram of a fixed block processing file system based on a fixed block according to an embodiment of the present invention.
Figs. 2A to 2C are diagrams for explaining variable processing based on a fixed block according to the present invention. Fig.
3 is a diagram showing an embodiment of metadata according to the present invention.
4A and 4B are diagrams for explaining file change data processing in a variable block processing file system based on a fixed block according to the present invention.
FIG. 5 is a diagram showing the file change detection unit shown in FIG. 1 in more detail.
6 is a diagram illustrating the operation of the overall system according to another embodiment of the present invention.
7 is a flowchart illustrating a file variable block processing method based on a fixed block according to another embodiment of the present invention.

Hereinafter, preferred embodiments of a variable block processing file system and a file variable block processing method based on a fixed block according to the present invention will be described with reference to the accompanying drawings. In the following description of the present invention, it is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the technical scope of the present invention. Will be.

1 is a block diagram illustrating an entire block diagram of a fixed block processing file system based on a fixed block according to an embodiment of the present invention.

1, a variable block processing file system based on a fixed block includes a storage medium 110, a file management unit 120, a file change detection unit 130, a memory 140, and a file change processing unit 150, .

The storage medium 110 is divided into a file management area 112 and a data storage area 114 for storing file data. Here, the storage medium 110 is a permanent storage medium, and includes not only a hard disk but also a flash memory, which is a semiconductor memory.

In the file management area 112, fixed block information related to fixed-length block information and block variable information for storing variable-length information are stored in a fixed-length block. In the data storage area 114, a file management area 112 And the file information is stored according to the block variable information. On the other hand, the fixed block information includes position information of the data storage area 114 for each block. The block variable information may include the number of changed blocks and the number of bytes of file data stored in the changed block.

The file management unit 120 may include a file creation module 122, for example, to create and manage files on the storage medium 110. [

The file generation module 122 stores the file data in a plurality of fixed blocks when the file is initially stored in the storage medium 110. [ The file generation module 122 stores the fixed block information in the file management area 112 when the file is first stored in the storage medium 110 and stores the file data in the data storage area 114 allocated to the fixed block information Can be stored.

The file change detection unit 130 detects whether a file created and stored in the storage medium 110 has been changed by an application program or the like and includes a hash value extraction module 132 and a similarity comparison module 132 do.

The hash value extraction module 132 divides one file data to be stored into a predetermined unit size smaller than the block size when the file management unit 120 stores the file in the storage medium 110 for the first time, Extract the hash value.

The similarity comparison module 132 compares the hash values extracted from the hash value extraction module 132 with the hash values associated with the file data stored in the storage medium 110 to check whether the same hash value exists.

The memory 140 is for storing information and data of a change block and includes a change block storage area 142. [ The change block storage area 142 stores the information of the change block and the information of the variable length block.

The file change processing unit 150 is for processing the change block processing and the index modification of the fixed block due to the file change 150. The change block detection module 152, the block variable processing module 154, and the index modification module 156 .

The change block detection module 152 detects whether there is a data change block matching the change block stored in the memory 110 by the operation of the file change detection unit 130 when the application requests storage of the file change data. For this, the change block detection module 152 may use the similarity comparison module 132.

If the block change processing module 154 detects that the change block detection module 152 has the change block stored in the memory 110, it determines whether the size of the data of the change block is equal to the size of the fixed block, The modified block is processed as a variable block and stored. In addition, the block variable processing module 154 may further store the block variable information for the changed block in the file management area 112 upon storage, and may store the corresponding file data only for the changed block. When the index modification module 156 desires to change the index information with respect to the unaltered blocks, the block variable processing module 154 may process some of the file data existing in the beginning of the unaltered blocks as a block variable have.

The index correction module 156 uses the file similarity measured by the change block detection module 152 to store the unchanged block in the data storage area 114 for the same unchanged blocks with the same file data, And stores the corrected index in the data storage area 114 of the unchanged block stored in the storage unit 112. [

Hereinafter, embodiments of the present invention will be described in more detail.

Figs. 2A to 2C are diagrams for explaining variable processing based on a fixed block according to the present invention. Fig.

FIG. 2A is a conceptual diagram of a conventional file system. When the file is stored in the actual storage medium 110 as a byte stream, the file is stored through the chunking step. At this time, the INODE has information to record the position of each block together with the information of the file. Here, the fixed size of each block may be, for example, 512 bytes.

2B is a conceptual diagram of a conventional file system for modifying stored file data. As shown in FIG. 2B, when the "K" is deleted, the file is changed from the third block to the internal file data in the chunking step. In this case, the existing block may be invalidated according to the file system, the new block may be allocated and the file data may be stored, or the existing block may be overwritten.

FIG. 2C is a conceptual diagram for modifying file data using a variable based on a fixed block according to the present invention. As shown in FIG. 2C, when the "K" is deleted, the file data size smaller than 512 bytes is stored in the third block in the chunking step. And INODE modifies the block index on the changed contents. In the case of storing file data according to the present invention, only the third block in the data block is changed, and no change occurs in the remaining blocks.

3 is a diagram showing an embodiment of metadata according to the present invention.

INODE stores basic information for file I / O. In the present invention, an existing INODE is extended to be based on a fixed block to support variable processing of the block. As shown in FIG. 3, in the metadata information according to the present invention, two pieces of information are added in addition to the information (B0, B1, B2, and B3) about the existing fixed block as well as the variable processing of the fixed block. A variable block (VB) records how many block variables it has. If VB is 1, it indicates that one block has been changed by the variable processing of the block in the file data. A variable block index (VBI) indicates an index of a block in which variable processing for a block has occurred. In FIG. 3, V1, V2, V3 and V4 are shown.

As shown in FIG. 3, it is assumed that 16 bytes have been deleted due to the change of the file data in the third block. In this case, only 496 bytes are stored in the block located at 53, and the remaining 16 bytes are meaningless data. In order to refer to this in the file system, the corresponding information must be described in the INODE. 1 is stored in VB because one variable process is generated for the fixed block. In V1, it is informed that block 53 is a block having a variable process for a fixed block, and information about that 496 bytes is recorded in block 53 is stored.

In FIG. 3, variable processing for the fixed block is limited to a maximum of four, but the maximum number can be increased as needed. However, when variable processing for a maximum number of fixed blocks is required, garbage collection is forcibly performed to remove all the variable processing for the fixed block.

4A and 4B are diagrams for explaining file change data processing in a variable block processing file system based on a fixed block according to the present invention.

4A shows a file change data processing method in a conventional file system. As shown in FIG. 4A, when a new file is created, the conventional file system can not utilize the block even though there are four blocks without data change. Because there is a block without data change, it is difficult to share block by block due to byte-shift phenomenon.

4B shows a method of processing file change data using variable processing based on a fixed block according to the present invention. As shown in FIG. 4B, when the variable block processing file system based on the fixed block implemented by the present invention is used, chunking is performed based on the unchanged block to find no change for the four blocks And four blocks with no change can be stored in the nonvolatile memory instead of being newly stored in the nonvolatile memory by modifying the index into blocks of the existing file.

FIG. 5 is a diagram showing the file change detection unit shown in FIG. 1 in more detail.

5, the file change detection unit 130 includes a hash value extraction module 132 and a similarity comparison module 132. The file change detection unit 130 includes a HDD 510 as a storage medium 110, A file creation memory 520 associated with the volatile memory where new file data is created and stored, and a hash value storage memory 530 associated with the volatile memory in which the hash value for the file is stored. Here, the file creation memory 520 and the hash value storage memory 530 may be separate memories, but may be one memory.

When the file creation module 122 of the file management unit 120 stores the file stored in the file creation memory 520 in the HDD 510, the hash value extraction module 132 stores the file in the file creation memory 520 And extracts the hash value. In this case, the hash value extraction module 132 may divide the file data into a predetermined unit size smaller than the block size, and extract the hash value for each predetermined unit size. The hash values extracted from the hash value extraction module 132 may be stored in the hash value storage memory 530.

The similarity comparison module 132 compares the hash values extracted by the hash value extraction module 132 with the hash values associated with the respective file data stored in the hash value storage memory 530 to check whether the same hash value exists . The hash values associated with each file data stored in the hash value storage memory 530 can be read to the HDD 510. [

The similarity digest hashing (SDHASH) scheme or the SSDEEP scheme can be used to process the file similarity.

Instead of choosing arbitrary chunk sizes using a rabin fingerprint, SDHASH uses stochastic features using the newness entropy, so that the feature point selection does not shift to a specific part of the data, Reliability is improved. The selected feature points measure the statistical based file similarity using Bloom filter. The 64-byte minutiae are used to generate the minutiae by selecting the minutiae that contain the popularity score above a given threshold value. In addition, among the selected minutiae, minutiae which have a value of more than 100 and less than 990 are excluded, thereby lowering the positive error rate of the Bloom filter.

In SDHASH, the Bloom filter is generated with a fixed size of 256 bytes. 128 feature point values are assigned to each generated bloom filter. If the number of minutiae exceeds 128, a new Bloom filter is added and minutiae values are assigned. Each feature point is computed as a 160-bit hash value using the SHA-1 hash function. The SDHASH comparison process compares all generated Bloom filters one by one and measures the file similarity by averaging all Bloom filter comparison values.

SSDEEP performs file similarity measurement using two hash methods. The rolling hash function moves the contents of the file one byte at a time and generates a hash value. Calculate the trigger position, which is the boundary point inside the file, using the generated rolling hash value. If a trigger location is found, it is treated as a single chunk from the beginning of the file to the trigger location, and the junk generates a junk hash using the MD5 hash function. The generated chunk hash value is converted into a bit string using Base64 encoding, extracts the last 6 bits of the converted bit string, and concatenates the bit values of each bit string to generate a file digest.

The file change detection unit 130 can maintain the similarity value of each file processed by the above-described technique in a string form in the INODE. Therefore, when a new file is imported or generated, the file change detection unit 130 can find the most similar file through comparison of hash strings.

6 is a diagram illustrating an operation of a fixed block processing file system based on a fixed block according to another embodiment of the present invention.

The user application program may be a user application program, such as a general text editor, for example a Hangul word processor (HWP) program.

The VLFS library is an example of a virtual file system that provides basic functions for opening, closing, and locating files in an existing file system, such as open (), close (), and lseek () Block change check function vlfs_check (), a read function vlfs_read () for a variable length block, and a write function vlfs_write () for a variable length block.

The above open (), close () and lseek () functions are related to the file management unit 120 shown in FIG. 1, the above vlfs_check () function is related to the file change detection unit 130, and vlfs_read , and vlfs_write () functions are associated with the file change processing unit 150.

The block change check function vlfs_check () is used to check whether a modification operation occurs by using a user application program. It checks whether there is a change block before calling the vlfs_write () function. If the change block is empty, It terminates without calling vlfs_write (), but if the change block is not empty, it determines that a modification has occurred and calls vlfs_write () to save it to disk. In addition, the block change check function vlfs_check () is used for comparing and judging whether or not the corresponding block data is modified data when it is stored in the disk.

In the memory 140 related to the file system, information of a change block and a variable-length block are stored. In the change block, data is temporarily stored when a modification operation is performed using a user application program, the modified data is divided into a size of a given block, and block position information is allocated. In the variable-length block, variable-length block information including a variable area is stored.

When a user application program needs to make modifications such as adding new data to a file using the VLFS library, it is necessary to know the location of the data added through the application and the size of the added data. The additional data information for this is basically stored in the change block. In addition, a "change check function" is used to check whether or not data is added.

For example, the file " A " stored in the disk which is the storage medium 110 shown in Fig. 1 is opened by using the open () function of the VLFS library. The user confirms the contents through various user application programs such as a text editor and inserts additional contents as needed. When insertion of new data occurs, the offset indicating the position to be added and the size of the data to be added are stored in the change block. Thereafter, the change check function compares the data contents of the file " A " with the change block to determine whether or not the data is added. Each block data of the file " A " is compared with the block data stored in the change block, and if it is not the same block data, the data is judged as already stored in the disk and the next blocks are compared. In the case of block data modified in file "A", one block data is stored among the block data stored in the change block, and the vlfs_write () function is called to store the corresponding data on the disk. The data of the position and the data size of the block data stored in the change block is referred to, and the corresponding data is stored in the disk block and the process is terminated.

Hereinafter, the process of detecting the file change of the modified data will be described in more detail.

The file change detection unit 130 stores the position information and the data size of the data in the change block when the position to be corrected and the size of the data are determined in the file. If the size of the added data is equal to the size of the given data block according to the size of the data to be modified, simply adding the block and changing the position of the existing blocks accordingly is performed. If the size of the added data is smaller than the size of the given block, the remaining area of the block is treated as a variable area after the data is added. A block in which a variable area exists is called a variable length block, and a variable length block is used to store information of the corresponding block. The block information stored in the variable length block is also stored in the change block.

If the size of the modified data is larger than the size of the given data block, the added data is divided into several parts using the size of the given block. If the size of the given block is 4KB and the size of the added data is 10KB, the data is divided into 3 blocks having sizes of 4KB, 4KB and 2KB, and all three blocks are stored in the change block. In this case, since only 2 KB smaller than the size of a given block is used in the last third block, the remaining area is treated as a variable area and the corresponding information is additionally stored in the variable-length block.

Hereinafter, the process of storing the added new data in the storage medium will be described in more detail.

When the vlfs_wirte function is called (S902), the user application program passes the entire block contents of data from the beginning to the vlfs_check function.

The change block detection module 152 compares the contents of each transferred data block with the change block. In this case, it is preferable to compare the hash values extracted by the hash value extraction module 132 using the similarity comparison module 134.

If the same block is not found, the change block detection module 152 determines that the data block is already stored in the disk block. In this case, since the data block is stored in the disk, a write operation for storing the data in the disk is unnecessary. However, if the data block is the same data block as the data block in the change block, the data block corresponds to the newly added data block and is stored on the disk through the vlfs_write function.

When a change block is detected by the change block detection module 152, the block variable processing module 154 compares the position comparison of the new data block to be added. If the location of the added data block is the boundary of each block, the block variable processing module moves the existing block address values backward by the number of blocks used in the added data, allocates data to the new disk block, The connection address value is set in the middle part. However, if the location of the added data is not the boundary of the block but the middle of the block, the block at the position where the new data is added is divided and stored as a variable length block according to the size of the added data.

7 is a flowchart illustrating a file variable block processing method based on a fixed block according to another embodiment of the present invention.

The file creation module 122 creates a new file in the user application program (S702). If the user application program requests the storage medium 110 to store a new file generated by the file creation module 122, the file change detection unit 130 determines whether the file is first stored in the storage medium 110 (S704).

If the file is first stored in the storage medium 110, the hash value extraction module 132 divides one file data to be stored into a predetermined unit size smaller than the block size, extracts a plurality of hash values (S706 ), And the file management unit 120 stores the fixed blocks from which the hash values are extracted in the storage medium 110 (S708).

The file change detection unit 130 detects whether there is a change in the file data by the user application program. To this end, the file change detection unit 130 determines whether the file data stored in the storage medium 110 is modified by the file creation module 120 through the similarity comparison using the hash values obtained from the hash value extraction module 132 (S710).

When the modification of the file data is detected, the file change detection unit 130 stores the information of the change block and the variable length block in the change block storage area 142 of the memory 140 (S712).

Thereafter, when the user application program requests storage of changed file data, the change block detection module 152 determines whether there is a change block in the file data to be stored using the similarity comparison module 132 (S714).

If there is no change block, the index modification module 156 changes only the index information in the file management area (S716). The block variable processing module 154 further stores the block variable information for the changed block in the file management area 112 (S614) and stores the corresponding file data only for the changed block (S616).

The scope of protection of the present invention should be interpreted according to the claims. It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit and scope of the invention as defined by the appended claims. It should be interpreted that it is included in the scope of right.

110: Storage medium 112: File management area
114: Data storage area 120: File management part
122: file generation module 130: file change detection unit
132: hash value extraction module 134: similarity comparison module
140: memory 142: change block storage area
150: File change processing unit 152: Change block detection module
164: block variable processing module 156: index modification module
510: HDD 520: file creation memory
530: Hash value storage memory

Claims (8)

A storage medium for storing file data;
A memory for storing temporarily changed block data;
A file management unit for storing file data in a plurality of fixed blocks if the file storage in the storage medium by the application program is the first,
A file change detecting unit for storing change block data in the memory when a change of file data by the application program is detected;
When the storage of the file change data is requested by the application program, only the index information is changed for the block among the plurality of fixed blocks stored in the storage medium using the change block data stored in the memory, And a file change processing unit for processing the block having the variable block as a variable block and storing the processed variable block as a variable block.
The method according to claim 1,
Wherein the storage medium includes a file management area and a data storage area,
Wherein the file management unit stores the fixed block information in the file management area and stores the file data in the data storage area allocated to the fixed block information when the file storage in the storage medium is the first,
Wherein the block change processing unit changes only the index information of the file management area for a block that has not been changed among the plurality of fixed blocks stored in the storage medium and outputs the variable block information to the file management area And stores the variable block data in the data storage area. The variable block processing file system according to claim 1,
3. The method of claim 2,
Wherein the fixed block information includes position information of the data storage area for each block,
Wherein the variable block information includes a number of change blocks and a length of file data stored in the change block.
4. The method according to any one of claims 1 to 3,
Wherein the file change detection unit classifies one file data to be stored in the storage medium into a predetermined unit size smaller than the block size and extracts a plurality of hash values when the file storage in the storage medium is first performed by the application program, And detects whether there is a modification of file data by the application program through a similarity comparison using a plurality of hash values.
5. The method of claim 4,
Wherein the file change detection unit stores change block and variable length block information for the change block data in the memory,
Wherein the file change processing unit compares the entire block of the file change data with the change block stored in the memory when the application program requests the storage of the file change data and at least one block of the file change data is stored in the memory Block, and stores the information in the variable block using the information stored in the variable-length block.
A file variable block processing method in a variable block processing file system based on a fixed block including a storage medium for storing file data and a memory for storing temporarily changed block data,
A file management step of storing file data in a plurality of fixed blocks if the file storage in the storage medium by the application program is the first,
A file change detection step of storing change block data in the memory when a change of file data by the application program is detected;
When the storage of the file change data is requested by the application program, only the index information is changed for the block among the plurality of fixed blocks stored in the storage medium using the change block data stored in the memory, And a file change processing step of storing the processed block in a variable block and storing the processed variable block in a variable block.
The method according to claim 6,
Wherein the storage medium includes a file management area and a data storage area,
Wherein the file management step stores the fixed block information in the file management area and stores the file data in the data storage area allocated to the fixed block information,
Wherein the block change processing step changes only the index information of the file management area for a block having no change among the plurality of fixed blocks stored in the storage medium, And stores the variable block data in the data storage area. The method of claim 1,
8. The method according to claim 6 or 7,
The file change detection step may include a step of dividing one file data to be stored in the storage medium into a predetermined unit size smaller than a block size and extracting a plurality of hash values, And detecting whether there is a modification of the file data by the application program through comparison of similarity using the plurality of hash values.
KR1020150107330A 2015-07-29 2015-07-29 Variable block processing file system and file variable block processing method based on fixed block KR101729624B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020150107330A KR101729624B1 (en) 2015-07-29 2015-07-29 Variable block processing file system and file variable block processing method based on fixed block

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020150107330A KR101729624B1 (en) 2015-07-29 2015-07-29 Variable block processing file system and file variable block processing method based on fixed block

Publications (2)

Publication Number Publication Date
KR20170014276A true KR20170014276A (en) 2017-02-08
KR101729624B1 KR101729624B1 (en) 2017-05-02

Family

ID=58155657

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020150107330A KR101729624B1 (en) 2015-07-29 2015-07-29 Variable block processing file system and file variable block processing method based on fixed block

Country Status (1)

Country Link
KR (1) KR101729624B1 (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140107590A (en) 2011-12-22 2014-09-04 스카니아 씨브이 악티에볼라그 Method and module for determining of at least one reference value

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140107590A (en) 2011-12-22 2014-09-04 스카니아 씨브이 악티에볼라그 Method and module for determining of at least one reference value

Also Published As

Publication number Publication date
KR101729624B1 (en) 2017-05-02

Similar Documents

Publication Publication Date Title
US8959089B2 (en) Data processing apparatus and method of processing data
CN107787489B (en) File storage system including a hierarchy
US20190324954A1 (en) Two-stage front end for extent map database
US9141633B1 (en) Special markers to optimize access control list (ACL) data for deduplication
US10860232B2 (en) Dynamic adjustment of fingerprints added to a fingerprint index
US8914338B1 (en) Out-of-core similarity matching
US9158465B2 (en) Performing authorization control in a cloud storage system
US9424185B1 (en) Method and system for garbage collection of data storage systems
EP2363815B1 (en) System for permanent file deletion
US9639275B2 (en) Managing data within a storage device based on file system metadata
EP3864519A1 (en) Key-value store tree data block spill with compaction
US9183218B1 (en) Method and system to improve deduplication of structured datasets using hybrid chunking and block header removal
US11609849B2 (en) Deduplication system threshold based on a type of storage device
CN106980680B (en) Data storage method and storage device
US11169968B2 (en) Region-integrated data deduplication implementing a multi-lifetime duplicate finder
KR101729624B1 (en) Variable block processing file system and file variable block processing method based on fixed block
US11507273B2 (en) Data reduction in block-based storage systems using content-based block alignment
US20190108238A1 (en) Block storage device with optional deduplication
KR101553028B1 (en) Variable processing file system and file variable processing method based on fixed block
CN107506156B (en) Io optimization method of block device
US20240037034A1 (en) Data intake buffers for deduplication storage system
US11119681B2 (en) Opportunistic compression
WO2023241771A1 (en) Deduplication mechanism on sequential storage media
CN118159936A (en) Parallel deduplication mechanism on sequential storage media
WO2024032898A1 (en) Choosing a set of sequential storage media in deduplication storage systems

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E701 Decision to grant or registration of patent right
GRNT Written decision to grant