KR20210056636A - Method for Fast Block Deduplication and transmission by multi-level PreChecker based on policy - Google Patents

Abstract

Provided are a data backup method and a recording medium. According to the present data backup method, a changed file among a plurality of files at regular time intervals is extracted; a compare block composed of only a changed block is created by dividing the changed file into a plurality of blocks and removing an unaltered block among the plurality of blocks; 100% data recovery function can be provided in an event of a failure of a first system by being capable of providing the data backup method that compresses the compare block and providing data synchronization function through real-time block data duplexing transmission; and data compression efficiency and transmission efficiency can be increased by removing a duplication block in advance when transmitting the block data.

Description

Method for Fast Block Deduplication and transmission by multi-level PreChecker based on policy}

It relates to a data backup method, and more particularly, to a data backup method for backing up files at high speed.

The content described in this section merely provides background information on the present embodiment and does not constitute the prior art.

Conventionally, when applying the backup data duplication, it was necessary to stop the system for the backup system to install and apply the system. However, in the case of a system in which the service is in progress, there are many negative views on the application of an additional backup system due to disadvantages due to system interruption and interworking problems occurring during the additional installation process.

Accordingly, there is a need for a high-speed block data transmission technology for the construction of a non-stop (zero downtime) redundant backup system and real-time operation of the redundant backup system.

The present invention was conceived to solve the above problems, and an object of the present invention is to extract a changed file from among a plurality of files at regular time intervals, divide the changed file into a plurality of blocks, and It is to provide a data backup method and a recording medium for generating a compare block composed of only changed blocks by removing unchanged blocks, and compressing the compare block.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present invention belongs from the following description. I will be able to.

According to an embodiment of the present invention for achieving the above object, a data backup method performed by a computer includes: extracting a changed file from among a plurality of files; Dividing the changed file into a plurality of blocks; Removing an unchanged block from among the plurality of blocks to generate a compare block composed of only changed blocks; And compressing the compare block.

In the generating of the compare block, one block may be divided into a plurality of partitions, it may be checked whether to be changed in a partition unit, and it may be determined that a block including at least one changed partition is a changed block.

In addition, in the step of generating the compare block, it is possible to simultaneously check whether or not to change the predetermined number of partitions in parallel.

In addition, the plurality of blocks may have different sizes of at least two of the blocks, and the plurality of partitions may have the same size of each partition.

Further, in the step of compressing, a compression rate for the compare block may be calculated, and compression may be performed only when the compression rate is greater than or equal to a specific value.

The method may further include a step of synchronizing by applying the compressed compare block to a corresponding existing backup file.

It may further include a step of transmitting the compressed compare block to an external backup device.

Meanwhile, in a computer-readable recording medium containing a computer program for performing a data backup method performed by a computer according to an embodiment of the present invention, a changed file is extracted from a plurality of files at a predetermined time interval. step; Dividing the changed file into a plurality of blocks; Removing an unchanged block from among the plurality of blocks to generate a compare block composed of only changed blocks; And compressing the compare block may include a computer program for performing a data backup method including.

According to various embodiments of the present invention, a changed file is extracted from a plurality of files at regular time intervals, the changed file is divided into a plurality of blocks, and an unchanged block is removed from among the plurality of blocks, thereby comparing only changed blocks. It is possible to provide a data backup method and a recording medium that creates a compare block and compresses a compare block, providing a data synchronization function through real-time block data redundancy transmission, thereby recovering 100% data in case of a primary system failure. It is possible to provide and increase data compression efficiency and transmission efficiency by removing redundant blocks in advance when transmitting block data.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those of ordinary skill in the art from the following description. will be.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as part of the detailed description to aid in understanding of the present invention, provide embodiments of the present invention, and describe the technical features of the present invention together with the detailed description.
1 is a diagram showing the configuration of a computer, according to an embodiment of the present invention;
2 is a flow chart for explaining a data backup method according to an embodiment of the present invention;
3 is a schematic diagram of a data backup method according to an embodiment of the present invention;
4 is a diagram illustrating an example of a block and a partition according to an embodiment of the present invention;
5 is a diagram showing a memory processing method in block units according to an existing method;
6 is a diagram illustrating a memory processing method in a partition unit according to an embodiment of the present invention;
7 is a diagram showing the configuration of a backup system according to an embodiment of the present invention.

In order to clarify the features and advantages of the problem solving means of the present invention, the present invention will be described in more detail with reference to specific embodiments of the present invention shown in the accompanying drawings.

However, in the following description and the accompanying drawings, detailed descriptions of known functions or configurations that may obscure the subject matter of the present invention will be omitted. In addition, it should be noted that the same components are denoted by the same reference numerals as much as possible throughout the drawings.

The terms or words used in the following description and drawings should not be interpreted as being limited to their usual or dictionary meanings, and the inventor may appropriately define the concept of terms for describing his or her invention in the best way. It should be interpreted as a meaning and concept consistent with the technical idea of the present invention based on the principle that there is.

Therefore, the embodiments described in the present specification and the configurations shown in the drawings are only the most desirable embodiments of the present invention, and do not represent all the technical ideas of the present invention. It should be understood that there may be equivalents and variations.

In addition, terms including ordinal numbers such as first and second are used to describe various elements, and are used only for the purpose of distinguishing one element from other elements, and to limit the elements. Not used. For example, without departing from the scope of the present invention, the second element may be referred to as the first element, and similarly, the first element may be referred to as the second element.

In addition, when a component is referred to as being "connected" or "connected" to another component, it means that it is logically or physically connected, or can be connected.

In other words, it should be understood that a component may be directly connected or connected to another component, but another component may exist in the middle, or may be indirectly connected or connected.

In addition, terms such as "comprises" or "have" described herein are intended to designate the existence of features, numbers, steps, actions, components, parts, or a combination thereof described in the specification. It is to be understood that the above other features or the possibility of the presence or addition of numbers, steps, actions, components, parts, or combinations thereof are not preliminarily excluded.

In addition, terms such as "... unit", "... group", and "module" described in the specification mean a unit that processes at least one function or operation, which can be implemented by hardware or software or a combination of hardware and software. have.

In addition, "a or an", "one", "the" and similar words are indicated otherwise in this specification in the context of describing the present invention (especially in the context of the following claims). It may be used in a sense encompassing both the singular and the plural, unless otherwise clearly contradicted by context.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

1 is a diagram showing the configuration of a computer 100 according to an embodiment of the present invention. As shown in FIG. 1, the computer 100 includes a communication unit 110, a control unit 120, and a storage unit 130.

The communication unit 110 is connected to enable communication with an external backup device. Here, the external backup device is a device that stores backup files for files stored in the computer 100, and the external backup device periodically receives and synchronizes files stored in the computer 100 to be the same as the computer 100. Saving the file. The communication unit 110 can perform communication through various wireless communication methods such as Bluetooth, Wi-Fi, short-range wireless communication (NFC), cellular, and LTE (Long-Term Evolution), and can communicate through wired communication such as a wired LAN. Of course it may be.

The control unit 120 controls the overall operation of the computer 100, for example, the central processing unit may correspond to this. The controller 120 performs the data backup method described later with reference to FIG. 2.

The storage unit 130 stores files. In addition, the storage unit 130 may store backup files for the stored files. In this case, the storage unit 130 periodically synchronizes the stored files and stores the backup files in the backup space. It goes without saying that the storage unit 130 may be implemented as a variety of storage devices, for example, a hard disk, a solid state drive (SSD), or the like.

The computer 100 having such a configuration performs a data backup method. Hereinafter, a data backup method will be described in detail with reference to the drawings.

2 is a flowchart illustrating a data backup method according to an embodiment of the present invention.

First, the computer 100 extracts a changed file from among a plurality of files (S210). Specifically, the computer 100 may periodically extract a changed file. In addition, the computer 100 may extract a file that is being changed into a changed file in real time.

Then, the computer 100 divides the changed file into a plurality of blocks (S220). In this case, the plurality of blocks may have different sizes of at least two of the blocks. That is, the computer 100 can variably set the size and number of blocks according to the current state of use of resources.

In addition, the computer 100 removes the unchanged block from among the plurality of blocks to generate a compare block composed of only the changed blocks (S230). Here, the compare block consists of only changed blocks, and is a block for synchronizing the backup file to the latest version.

In this case, the computer 100 divides one block into a plurality of partitions, checks whether to change it in units of partitions, and determines that a block including at least one changed partition is a changed block. That is, the computer 100 does not store and process a queue in a memory in block units, but stores and processes in a memory queue in a partition unit, which is a smaller unit. In this case, the computer 100 sets a plurality of partitions so that the size of each partition is the same. By processing in a partition unit smaller than a block unit, the computer 100 can use a memory for generating a compare block in a smaller unit, thereby facilitating real-time processing of backup data.

In addition, the computer 100 checks whether or not to change a preset number of partitions simultaneously in parallel. Specifically, by executing a plurality of precheckers, the computer 100 may simultaneously check whether or not to change a preset number of partitions in parallel. Here, the pre-checker is a module that performs a function of determining whether a partition has been changed, and the pre-checker corresponds to one program or processor, and a plurality of pre-checkers may be simultaneously executed in parallel.

Then, the computer 100 compresses the generated compare block (S240). The computer 100 may compress the compare block using various compression algorithms, and may apply the LZ4 algorithm, for example.

In this case, the computer 100 may calculate a compression rate for the compare block and perform compression only when the compression rate is greater than or equal to a specific value. Specifically, the computer 100 calculates the compression rate of the compare block in advance by using the compression rate formula corresponding to the compression algorithm, and performs compression only when the compression rate is higher than a specific value. Through this, the computer 100 does not perform unnecessary compression, so that the backup data processing speed can be improved. Here, the compression ratio is a value indicating the degree of compression, and the compression ratio increases as the compression increases.

Then, the computer 100 is synchronized by applying the compressed compare block to the corresponding existing backup file (S250). Specifically, the computer 100 stores the backup files in the storage unit 130, and by applying the compare block to the existing backup file corresponding to the changed file, synchronization is performed so that the contents of the changed file and the existing backup file become the same. Will perform. Through this, the computer 100 can synchronize the existing backup file with the changed file using only the compare block without comparing the entire changed file with the existing backup file, so that the backup file can be synchronized at a higher speed.

In addition, the computer 100 may be communicatively connected with the external backup device 700 or may transmit the compressed compare block to the external backup device 700. This will be described in more detail later with reference to FIG. 7.

Through this process, the computer 100 can provide a data backup method that is processed in a partition unit, and provides a data synchronization function through real-time block data redundancy transmission, thereby providing a 100% data recovery function in case of a primary system failure. When transmitting block data, it is possible to increase data compression efficiency and increase transmission efficiency by removing redundant blocks in advance.

3 is a schematic diagram of a data backup method according to an embodiment of the present invention. FIG. 3 is a schematic diagram of the above-described process, and shows a case in which four PreCheckers are configured. As shown in FIG. 3, the computer 100 may first search for a changed file and check a process of processing the changed file in parallel through four pre-checkers.

4 is a diagram illustrating an example of a block and a partition according to an embodiment of the present invention. FIG. 4 shows a case of dividing the changed file into 4 blocks of 3 GB, 4 GB, 1 GB, and 2 GB when the size of the changed file is 10 gigabytes (GB). In addition, FIG. 4 shows a case where one block is divided into partitions having a size of 640 KB. In FIG. 4, only 8 partitions from Part 1 to Part 8 are shown for Block 2 having a size of 4 GB, but these are only a part, and in reality, a partition having a size of 640 KB is divided into as many partitions as the total number of 4 GB.

5 is a diagram showing a memory processing method in block units according to a conventional method. According to FIG. 5, since the memory queue is loaded in block units and all data included in the block is accumulated in the memory, it can be seen that the amount of memory is very large.

On the other hand, FIG. 6 is a diagram illustrating a memory processing method in units of partitions according to an embodiment of the present invention. As shown in FIG. 6, the computer 100 stores and processes a memory queue in a partition unit. Then, as shown in FIG. 6, it can be seen that the memory usage becomes very small when the computer 100 processes data stored in the queue. In this way, when memory processing is performed in units of partitions, the computer 100 can process backup data faster with very little memory usage.

7 is a diagram showing the configuration of a backup system according to an embodiment of the present invention. As shown in FIG. 7, the backup system may be connected through a network so that the computer 100 and the external backup device 700 can communicate with each other. Then, the computer 100 transmits the compressed compare block to the external backup device 700. Then, the external backup device 700 stores the backup files, and by applying the received compare block to the existing backup file corresponding to the changed file, synchronization is performed so that the contents of the changed file and the existing backup file become the same. .

Meanwhile, the computer 100 including the above-described functions may be implemented as a physically independent device by itself, and may be implemented in a form included as a part of any device or system, and may be implemented as a smartphone, computer, or server. Of course, it may be implemented in the form of software such as a program installed on the back or a framework or application. In addition, each component of the computer 100 may be implemented as a physical component or a component in the form of a software function.

On the other hand, it goes without saying that the technical idea of the present invention can be applied to a computer-readable recording medium containing a computer program for performing the functions and methods of the apparatus according to the present embodiment. Further, the technical idea according to various embodiments of the present disclosure may be implemented in the form of a computer-readable programming language code recorded on a computer-readable recording medium. The computer-readable recording medium can be any data storage device that can be read by a computer and can store data. For example, a computer-readable recording medium may be ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, hard disk drive, flash memory, solid state disk (SSD), and the like. In addition, a computer-readable code or program stored in a computer-readable recording medium may be transmitted through a network connected between computers.

Although this specification and drawings describe exemplary device configurations, functional operations and implementations of the subject matter described herein are implemented as other types of digital electronic circuits, or include structures disclosed herein and structural equivalents thereof. It can be implemented with computer software, firmware, or hardware, or can be implemented by a combination of one or more of them.

Accordingly, although the present invention has been described in detail with reference to the above-described examples, modifications, changes, and modifications to the examples can be made without departing from the scope of the present invention by those of ordinary skill in the field to which the present invention belongs.

In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be understood individually from the technical spirit or prospect of the present invention.

100: computer
110: communication department
120: control unit
130: storage unit
700: external backup device

Claims (8)

In the data backup method performed by a computer,
Extracting a changed file from among a plurality of files;
Dividing the changed file into a plurality of blocks;
Removing an unchanged block from among the plurality of blocks to generate a compare block composed of only changed blocks; And
Compressing the compare block; data backup method comprising a.
The method according to claim 1,
The step of creating the compare block,
A method of backing up data, comprising: dividing one block into a plurality of partitions, checking whether to be changed in a partition unit, and determining that a block including at least one changed partition is a changed block.
The method according to claim 2,
The step of creating the compare block,
A method of backing up data, characterized in that checking whether or not to change a preset number of partitions in parallel and in parallel.
The method according to claim 2,
A plurality of blocks,
At least two of them have different sizes,
A plurality of partitions,
Data backup method, characterized in that the size of each partition is the same.
The method according to claim 1,
The step of compressing is,
A data backup method, comprising: calculating a compression rate for a compare block, and performing compression only when the compression rate is greater than or equal to a specific value.
The method according to claim 1,
And synchronizing by applying the compressed compare block to a corresponding existing backup file.
The method according to claim 1,
And transmitting the compressed compare block to an external backup device.
In a computer-readable recording medium containing a computer program for performing a data backup method performed by a computer,
Extracting a changed file from among a plurality of files at regular time intervals;
Dividing the changed file into a plurality of blocks;
Removing an unchanged block from among the plurality of blocks to generate a compare block composed of only changed blocks; And
Compressing the compare block; A computer-readable recording medium containing a computer program for performing a data backup method including.

Images