US20190065067A1 - Backup files to a disk image - Google Patents

Backup files to a disk image Download PDF

Info

Publication number
US20190065067A1
US20190065067A1 US16/171,193 US201816171193A US2019065067A1 US 20190065067 A1 US20190065067 A1 US 20190065067A1 US 201816171193 A US201816171193 A US 201816171193A US 2019065067 A1 US2019065067 A1 US 2019065067A1
Authority
US
United States
Prior art keywords
block
data
level write
write request
disk image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US16/171,193
Inventor
Chuanqi Sun
Zhiye Wang
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
CA Inc
Original Assignee
CA Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by CA Inc filed Critical CA Inc
Priority to US16/171,193 priority Critical patent/US20190065067A1/en
Publication of US20190065067A1 publication Critical patent/US20190065067A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/065Replication mechanisms
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1446Point-in-time backing up or restoration of persistent data
    • G06F11/1448Management of the data involved in backup or backup restore
    • G06F11/1451Management of the data involved in backup or backup restore by selection of backup contents
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614Improving the reliability of storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0683Plurality of storage devices

Abstract

Performing a file backup includes receiving a file to backup from a source machine and performing a write operation to write the file to a mount point in a file system on a backup server. The backup also includes intercepting a block-level data block to be written which is generated by the write operation; and writing the block-level data block to a corresponding, respective block of a disk image file having a plurality of blocks.

Description

    BACKGROUND
  • The present disclosure relates to data backups, and more specifically, to file-level and block-level backups.
  • When backing up data, two different methods are typically used. One method is file-level backups which performs the backup task by copy files and directories to a separate folder (or directory) structure. The granularity of this type of backup allows a small subset of folders or files to be designated for backup. Also this method allows flexible backup policies for different data types on the same volume. For example, the backup policies for documents, photographs, videos, etc. can all be different even though they reside on the same volume. This method also has detriments such as being time consuming for large amounts of data and copying a whole file during a backup operation even though only a small portion of the file may have changed.
  • Another backup method is at the block-level. A block-level backup is defined at blocks which are generally a group of disk sectors. This type of backup is at a lower level than files and folders and is performed below the file system level. Generally, block-level backups are used to accomplish disk-to-disk copies or volume-to-volume copies. Block-level backups allow backing up or restoring an entire volume or disk as block units. Block-level backups utilize more sophisticated software than file-level backups and typically require kernel-level drivers such as a snapshot driver and a change tracking driver to perform their tasks.
  • BRIEF SUMMARY
  • According to one aspect of the present disclosure, a system includes a processor; a storage device coupled to the processor, and a memory coupled to the processor, the memory configured to store program code executable by the processor. In particular, the program code, when executed by the processor, is configured to receive a file to backup from a source machine and to perform a write operation to write the file to a mount point of a file system on the storage device. The program code, when executed by the processor, is also configured to intercept a block-level data block to be written which is generated by the write operation and to write the block-level data block to a corresponding, respective block of a disk image file having a plurality of blocks.
  • According to another aspect of the present disclosure, a method includes receiving, by a computer, a file to backup from a source machine and performing, by the computer, a write operation to write the file to a mount point of a file system on the computer. The method also includes intercepting a block-level data block to be written which is generated by the write operation; and writing the block-level data block to a corresponding, respective block of a disk image file having a plurality of blocks.
  • According to another aspect of the present disclosure, a computer program product includes a computer readable storage medium having computer readable program code embodied therewith. The computer readable program code includes computer readable program code configured to receive a file to backup from a source machine; to perform a write operation to write the file to a mount point of a file system on a backup server; intercept a block-level data block to be written which is generated by the write operation; and to write the block-level data block to a corresponding, respective block of a disk image file having a plurality of blocks.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • Aspects of the present disclosure are illustrated by way of example and are not limited by the accompanying figures with like references indicating like elements.
  • FIG. 1 illustrates an example computing environment for performing data backups in accordance with the principles of the present disclosure.
  • FIG. 2A is a flowchart of an example process for backing up files to a disk image in accordance with the principles of the present disclosure.
  • FIG. 2B illustrates a logical diagram of how a disk image is constructed in accordance with the principles of the present disclosure.
  • FIG. 3A is a flowchart of an example process for performing a full backup to a disk image in accordance with the principles of the present disclosure.
  • FIG. 3B illustrates a logical diagram of how further details of how a disk image is constructed in accordance with the principles of the present disclosure.
  • FIG. 3C is a flowchart of an example process for performing an incremental backup to a disk image in accordance with the principles of the present disclosure.
  • FIG. 3D illustrates a logical diagram of how further details of how an incremental disk image is constructed in accordance with the principles of the present disclosure.
  • FIG. 4 illustrates a block diagram of a data processing system in accordance with the principles of the present disclosure.
  • DETAILED DESCRIPTION
  • As will be appreciated by one skilled in the art, aspects of the present disclosure may be illustrated and described herein in any of a number of patentable classes or context including any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof. Accordingly, aspects of the present disclosure may be implemented entirely hardware, entirely software (including firmware, resident software, micro-code, etc.) or combining software and hardware implementation that may all generally be referred to herein as a “circuit,” “module,” “component,” or “system.” Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable media having computer readable program code embodied thereon.
  • Any combination of one or more computer readable media may be utilized. The computer readable media may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an appropriate optical fiber with a repeater, a portable compact disc read-only memory (CORaM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
  • A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable signal medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
  • Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including MAINFRAME Assembler, an object oriented programming language such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, CII, VB.NET, Python or the like, conventional procedural programming languages, such as the “c” programming language, Visual Basic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP, dynamic programming languages such as Python, Ruby and Groovy, or other programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider) or in a cloud computing environment or offered as a service such as a Software as a Service (SaaS).
  • Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatuses (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable instruction execution apparatus, create a mechanism for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
  • These computer program instructions may also be stored in a computer readable medium that when executed can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions when stored in the computer readable medium produce an article of manufacture including instructions which when executed, cause a computer to implement the function/act specified in the flowchart and/or block diagram block or blocks. The computer program instructions may also be loaded onto a computer, other programmable instruction execution apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatuses or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
  • As used herein, a “block” is a uniformly sized unit of storage for a file system (e.g., a block can be 4 KB in size). Thus, a file system that is 400 KB in size would be comprised of 100 blocks. The file system includes methods and data structures that an operating system uses to keep track of files on a disk. It allows the user to assign symbolic names to files and directories while maintaining a record of the specific blocks that contain the data within those files and directories. Typically, the actual data stored on the disk will be organized according to a file system which is implemented within an operating system. User applications executing within that operating system access data on the disk according to the file system's organization.
  • When a read or write request for data is received, the operating system determines the blocks of the file system that are needed to satisfy the request and then requests those blocks from the disk. At a disk device driver level, or possibly, at a disk hardware interface level, a block number is translated into a physical location and data stored at that physical location is retrieved from, or written to, the disk.
  • A “disk image” is a single file containing the complete contents and structure representing a data storage medium or device, such as a hard drive. A disk image is usually created by creating a complete sector-by-sector copy of the source medium and thereby perfectly replicating the structure and contents of a storage device. Also, as used herein, the terms “folder” and “directory” are intended to be synonymous as they both refer to a way to organize groups of files in a hierarchical tree-like structure. A “volume” is a single accessible storage area with a single file system and is typically resident on a single partition of a hard disk. A volume exists at the logical operating system level and partitions exist at the physical media-specific level. Thus, an operating system can recognize a partition without necessarily recognizing any volume associated with it (e.g., when the operating system cannot interpret the file system type stored on the partition). A hard disk may, for example, have two partitions “hda1” and “hda2” and on each partition resides a respective file system (e.g., files, directories, and organizational meta data). The operating system can assign a label to each partition's file system such as “C:” and “D:” to create two volumes known as “C:” and “D:” in the operating system.
  • FIG. 1 illustrates an example computing environment for performing data backups in accordance with the principles of the present disclosure. In the example environment there is a source machine 102 where the source files to be backed up are located. For example, the source files and folders 112 may be stored in a disk drive or other storage device. One of the applications that can execute on the source machine 102 is a backup client 106. Using the backup client 106, a user can define flexible backup policies for the files and folders 112 that involve full or incremental backups of some or all of these files and folders 112. In particular, the user can use the back-up client 106 to define the backup policies at the file level (as opposed to the block level).
  • The environment of FIG. 1 also includes a backup server 104 on which is executing a backup application 108. The backup application 108 and the backup client can communicate with one another over a network connection 110. The backup client 106 determines the files and folders that need to be backed up according to the user defined backup policies. This is performed at the file level and a stream of data representing the files is sent to the backup application 108. Once the backup application 108 receives the data then it can launch routines to store the received data to the destination 114. As described more fully below, the writing of the data to the destination 114 can be performed at the block-level. As a result, copies of the files and folders 112 of the source machine 102 are created and stored at a destination storage device 114. As described more fully below, the copies that are created are done so at the block level such that the copies on the storage device 114 resemble disk images.
  • FIG. 2A is a flowchart of an example process for backing up files to a disk image in accordance with the principles of the present invention. Before the backup application 108 initiates, or causes the client 106 to initiate, any backup jobs, the backup application 108 and the backup client 106 exchange information about the storage devices where the files and folders 112 are located attached to the source machine 102. In the description below, operation of various backup processes and functions are described with respect to a single disk drive and its associated single disk image. One of ordinary skill will recognize that the described processes and functions can be repeated for additional disk drives without departing from the scope of the present disclosure.
  • Thus, in step 202, the backup application 108 receives from the backup client 106 a description of the disk and volume layout of the disk drive with the files and folders 112. The disk information includes the number of logical blocks on the disk. The volume layout information would include which blocks are associated with each volume. For example on a disk having n blocks, a “C:” volume may utilize blocks 1 through x and the “D:” volume utilize blocks x+1 through n. In general, the backup application 108 obtains information from the backup client 106 about: the number of different disks, each disk's size (in blocks), the number of different volumes, each volume's size, and the file system type of each volume.
  • Once the backup application 108 receives this information, a disk image can be created, in step 204, for the disk drive with the files and folders 112. While there is no data besides its metadata including the disk size and the parent disk image if it exists at the present time in the disk image, the backup application 108 can define the data structure and label that will become the disk image.
  • FIG. 2B illustrates a logical diagram of how a disk image is constructed in accordance with the principles of the present disclosure. In the example of FIG. 2B, there are three volumes of one disk drive shown. Because there are three volumes on the disk, the disk image can be considered to have three different sections. Each of the three sections representing one of the volumes. In step 206, the backup application 108 virtualizes each volume device on the disk image and creates a file system for each of the volumes.
  • The backup application 108, within the file system of the backup server 104, creates a respective folder for each of the three different volumes. For example, the folders could be named:
      • /backup/source/vol1
      • /backup/source/vol2
      • /backup/source/vol3
        and each of the folders can be treated as a mount point within the file system of the server 104 and its operating system.
  • For each of the three volumes, the backup application 108 can use the respective volume size and file system type to create a corresponding file system. Each of the three file systems can then be mounted to one of the appropriate mount points by the operating system of the backup server 104. Thus, the three volumes that make up the disk image are virtualized within the file system of the server 104. In other words, if an application on the server 104 were to write a file to one of the mount points, the backup application 108 could recognize that data as being written to one of the three volumes that are part of the disk that is being backed up to a disk image. In FIG. 2B there are three mount points 220, 222, 224 that correspond to the three volumes on an example disk drive of the source machine 102. Each of the mount point 220, 222, 224 correspond to an emulated volume (or section) 226, 228, 230 of the disk image 232.
  • Thus, when file contents are received from the source machine 102 a write operation to the mount-point folder acts as a normal file system write to a physical disk drive. The operating system will convert the file system write to a block-level write request to one of the file systems created and controlled by the backup application 108. Thus, the backup application 108 can intercept the block-level data in the file system write request and process the data before writing it to the disk image 232 as one or more blocks 234.
  • Returning back to the flowchart of FIG. 2A, once the volumes and file systems are created, a backup job can be initiated which, in step 208, causes the source machine 102 to discover folders and files that are to be backed up in accordance with the predefined policies and rules. In step 210, the file stream is received by the backup application 108 where the backup application 108 processes the file data and writes one or more blocks to the disk image 232. The actual disk image file is a single file written to a hard drive by the backup server's operating system, thus the disk image 232 of FIG. 2B is conceptual in nature in that on the hard drive of the server 104, the first block of the disk image file is not likely located at a logical block address of “1” for that hard drive.
  • Backups can be classified according to whether it is a full backup or an incremental backup. A full backup assumes that no previous files have been backed up and the backup process will make a copy of all the files and folders received from the source machine. An incremental backup, on the other hand, backs up only files and folders that have changed since the last full or incremental backup. Therefore a restore from an incremental backup will utilize the most recent full backup relative to the used incremental backup and all the incremental backups until the time of the used incremental backup.
  • FIG. 3A is a flowchart of an example process for performing a full backup to a disk image in accordance with the principles of the present disclosure. The full backup process of FIG. 3A is presented with respect to one file or folder from the source machine; the same process would be repeated for all the source files and folders that are to be copied.
  • A backup application 116 on the backup server 104 would receive the file contents from the source machine 102 of a file to be backed up. Afterwards, in step 302, the backup application 108 can write the received file contents to the mount point folder.
  • The writing of the file to the mount-point folder causes the operating system of the backup server 104 to generate block level write requests from the file system that the backup application 108 has virtualized at that mount point. The usual file system metadata maintains structures such as inodes and superblocks to track what blocks a file occupies within a file system. Thus, when a file system write occurs to one of the mount points, file names, inodes, and block addresses are all created or used to locate the blocks of the “virtual volume” where that file is to reside or already resides. However, the block data does not actually get physically written to blocks of that virtual volume; instead it is intercepted below the file system by the backup application 108 and processed as described herein.
  • The block-level data includes blocks that have an address which is an offset value from the start of their respective virtual volume. Thus, block “8” is the eighth sequential block starting from the beginning of the virtual volume. However, as mentioned above, each “virtual volume” is associated with a different contiguous portion of the disk image. Thus, if the second virtual volume, for example, started at block “100” of the disk image, then block “8” in that virtual volume is actually block “108” of the disk image.
  • Therefore, when the block-level data is intercepted by the backup application 108, a determination has been made by the operating system and file system, in step 306, as to what blocks of the disk image are involved in the write request. When a file has never been copied before, then the file system can write the file to any open blocks; however, if the file has previously been written to the disk image, then the inode tables and other file system data structures and functions can determine the blocks occupied by that file (relative to the beginning of the virtual volume).
  • In accordance with the principles of the present disclosure, a hash table of the blocks written to the disk image is maintained by the backup application 108. For example, a disk image may have 100 blocks with each block size being 4 K bytes. A hash table can be created, and stored separately from the disk image, that is 100 bytes in length. Each byte of the hash table represents a hash value of one of the blocks of the disk image. In other words, the 45th byte of the hash table includes 8-bits that represent a hash value for the 45th block of the disk image. One of ordinary skill will recognize that different size hash tables and different hash calculation algorithms can be used without departing from the scope of the present disclosure. The purpose of the hash process is to generate a value that changes whenever any part of its corresponding disk image block changes. The much-shorter size of the hash value allows it to be tested easier than evaluating all bits of the corresponding data block.
  • Thus, in step 308, for every block that is to be written to the disk image (all blocks for a full backup) a hash value is calculated and the hash table is updated. The backup application 108 then, in step 310, writes the blocks to the disk image. As mentioned above, the block address in the disk image takes into account the block number within a virtual volume and the offset value of that virtual volume relative to the starting block of the disk image.
  • FIG. 3B illustrates a logical diagram of how further details of how a disk image is constructed in accordance with the principles of the present disclosure. For example, a write of “File A” to one of the mount points can be performed by an application executing on the server 104. At this stage, the application is writing a file data stream 320. This write operation results in the operating system of the server 104 to generate block-level write data 322 which is intercepted by the backup application 108. For example, the backup application 108 determines that blocks 0, 1, 2, 4, 5, and 6 were chosen by the operating system to be used to store “File A”. Thus, the block-level write requests for “File A” may involve blocks: 0, 1, 2, 4, 5, and 6 which have the respective values: A, B, C, E, F, and G. These block numbers are relative to the first block of the virtual volume where “File A” is located. The “File A” is just used to describe the block write process, but it isn't used by the backup application 108 here. The backup application 108 focuses on the data blocks, not the file. In a full backup, the backup application 108 will write all the blocks to disk image blocks 324. The address of the blocks within the disk image is a combination of both where the virtual volume starts within the disk image and the block offset value of the file's data blocks within the virtual volume.
  • Initially the hash table 326 for the disk image is empty. Once the backup application 108 determines which of the blocks 322 are to be backed up in the disk image 324, the hash values for these blocks are calculated. The hash values are then used to produce an updated hash table 326. Similar to before, if block “8” of a virtual volume is to be backed up and that virtual volume starts at block “100” of the disk image, then hash value “108” of the disk image hash table is updated.
  • FIG. 3C is a flowchart of an example process for performing an incremental backup to a disk image in accordance with the principles of the present disclosure. An incremental backup can generate its own disk image file separate from the full backup disk image file described above. The disk image can also record its parent disk image (the last incremental backup disk image or the full backup disk image) in its metadata. Then they can be used to determine where the data are located in the disk image link when performing a restore job. However, the block numbers, offsets, and mount points remain consistent between the two disk image files. When performing an incremental back, the source machine, in steps 340 and 342, determines that “File A” (the example file from FIG. 3B) has changed and sends “File A” to the backup server 104. In step 344, the backup application 108 writes “File A” to the appropriate mount point which generates block-level write requests at the operating system. The backup application 108 intercepts the block-level write requests in step 346 and a respective hash value is calculated for each block that is included with the block-level write requests, in step 348. For example, the block-level write requests for “File A” may involve blocks: 0, 1, 2, 4, 5, 6, and 7 (element 360 of FIG. 3D) which have the respective values: A, B′, C, E, F′, G, and H.
  • In step 350, the backup application 108 compares each of the calculated seven hash values to their respective counterpart in the disk image hash table 328. In this example, the hash values BH′, FH′ are different from BH, FH and HH is a new added value, so the backup application 108 could determine that of the seven blocks 360 associated with “File A”, only blocks 1, 5 and 7 have changed since “File A” was previously copied. Thus, in step 352, the backup application 108 writes only blocks 1, 5, and 7 to the blocks of the incremental backup disk image file (element 362 of FIG. 3D). Also, in step 352, the backup application updates the hash values of entries 1, 5, and 7 of the hash table 328 to generate a new hash table 364. The other values in the hash table 328 are not changed and remain the same in the new hash table 364.
  • Compared to a file-level backup, a backup application in accordance with the principles of the present disclosure backs up only changed blocks of a file rather than the entire file. Also, because the backup form is that of a disk image, it can be used like a volume restore procedure which typically uses simpler logic and is faster than other types of recovery operations. Compared to a volume-level, or block-level, backup no kernel-level drivers need to be used on a source machine; and because the backup appears to be at the file-level on the source machine the defining of backup policies is easy and flexible.
  • Referring to FIG. 4, a block diagram of a data processing system is depicted in accordance with the present disclosure. A data processing system 400, such as may be utilized to implement the hardware platform 104 or aspects thereof, e.g., as set out in greater detail in FIG. 1-FIG. 3D, may comprise a symmetric multiprocessor (SMP) system or other configuration including a plurality of processors 402 connected to system bus 404. Alternatively, a single processor 402 may be employed. Also connected to system bus 404 is memory controller/cache 406, which provides an interface to local memory 408. An I/O bridge 410 is connected to the system bus 404 and provides an interface to an I/O bus 412. The I/O bus may be utilized to support one or more busses and corresponding devices 414, such as bus bridges, input output devices (I/O devices), storage, network adapters, etc. Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks.
  • Also connected to the I/O bus may be devices such as a graphics adapter 416, storage 418 and a computer usable storage medium 420 having computer usable program code embodied thereon. The computer usable program code may be executed to execute any aspect of the present disclosure, for example, to implement aspect of any of the methods, computer program products and/or system components illustrated in FIG. 1-FIG. 3D.
  • The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various aspects of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
  • The terminology used herein is for the purpose of describing particular aspects only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
  • The corresponding structures, materials, acts, and equivalents of any means or step plus function elements in the claims below are intended to include any disclosed structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the disclosure. The aspects of the disclosure herein were chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure with various modifications as are suited to the particular use contemplated.

Claims (21)

1-30. (canceled)
31. A method, comprising:
receiving, by an application executing on a first computer system, a request from a second computer system to backup a particular file, wherein the request includes the particular file;
causing, by the application, an operating system of the first computer system to generate a set of block-level write requests based on the particular file, wherein each of the set of block-level write requests includes data to be written to a respective block of a storage medium accessible to the first computer system;
for each block-level write request in the set, the application:
intercepting that block-level write request from the operating system;
determining whether that block-level write request includes data that is not stored in the respective block associated with that block-level write request; and
writing the data included in that block-level write request to the respective block only in response to the determining indicating that that block-level write request includes data that is not stored in the respective block.
32. The method of claim 31, further comprising:
creating, by the application, a disk image on the storage medium, wherein the disk image comprises a plurality of blocks for storing data, and wherein the particular file is written to a set of the plurality of blocks.
33. The method of claim 32, further comprising:
maintaining, by the application, a table having a respective entry for each of the plurality of blocks of the disk image, wherein a given entry is operable to store a hash value that is calculated by hashing data stored in a block corresponding to that given entry.
34. The method of claim 33, wherein the determining of whether that block-level write request includes data that is not stored in the respective block includes:
calculating, by the application, a first hash value based on the data included in that block-level write request; and
comparing, by the application, the first hash value with a second hash value stored in an entry of the table that corresponds to the respective block associated with that block-level write request, wherein the first hash value not matching the second hash value indicates that the data included in that block-level write request is not stored in the respective block.
35. The method of claim 33, wherein the determining of whether that block-level write request includes data that is not stored in the respective block includes:
determining, by the application, whether an entry of the table that corresponds to the respective block associated with that block-level write request includes a hash value.
36. The method of claim 33, further comprising:
for each block-level write request in the set, the application further:
calculating a hash value based on the data included in that block-level write request; and
storing the calculated hash value in an entry of the table that corresponds to the respective block associated with that block level write request only in response to the determining indicating that that block-level write request includes data that is not stored in the respective block.
37. The method of claim 31, wherein the causing of the operating system to generate the set of block-level write requests includes:
sending, to the operating system by the application, a request to write the particular file to a mount point of a file system created by the application.
38. A non-transitory computer readable medium having program instructions stored thereon that are capable of causing a computer system to implement an application capable of performing operations comprising:
receiving, from another computer system, a request to backup a particular file included in the request;
causing an operating system of the computer system to generate a set of block-level write requests for writing data of the particular file to a disk image accessible to the computer system, wherein each of the set of block-level write requests indicates one or more blocks of the disk image and includes data to be written to the one or more indicated blocks; and
for a given block-level write request in the set:
intercepting the given block-level write request generated by the operating system;
determining whether the given block-level write request includes data that is not stored in the one or more blocks indicated by the given block-level write request; and
writing data included in the given block-level write request to the disk image for only blocks indicated by the given block-level write request that are determined to not store data, included in the given block-level write request, that is to be written to those blocks.
39. The medium of claim 38, wherein the operations further comprise:
maintaining information that provides, for each one of the one or more blocks indicated by the given block-level write request, an indication of data stored in that block.
40. The medium of claim 39, wherein the determining of whether the given block-level write request includes data that is not stored in the one or more blocks includes:
for each one of the one or more blocks:
calculating a hash value based on data included in the given block-level write request that corresponds to that block; and
comparing the hash value and the indication provided in the information that corresponds to that block.
41. The medium of claim 39, wherein the operations further comprise:
updating indications provided in the information for only those blocks for which data that is included in the given block-level write request is written to the disk image.
42. The medium of claim 38, wherein the disk image is associated with one or more volumes of a storage device, and wherein the operations further comprise creating a file system for each one of the one or more volumes.
43. The medium of claim 42, wherein the causing of the operating system to generate the set of block-level write requests includes:
performing a write operation to write the particular file to a mount point of a file system created for particular one of the one or more volumes.
44. The medium of claim 38, wherein the operations further comprise:
receiving, from the other computer system, disk information defining a number of blocks of a storage device; and
creating the disk image based on the received disk information.
45. A method comprising:
receiving, by an application executing on a computer system, a request to backup data that is included in the request, wherein the data included in the request comprises a set of portions;
causing, by the application, an operating system of the computer system to generate block-level data that identifies, for each one of the set of portions, a respective block of a disk image for storing that portion;
intercepting, by the application, the block-level data from the operating system; and
writing, to the disk image by the application, only portions in the set that include data that is not stored in the respective blocks of those portions.
46. The method of claim 45, further comprising:
maintaining, by the application, a table that includes, for each block of the disk image, an entry for storing a value indicative of data stored in that corresponding block.
47. The method of claim 46, further comprising:
determining, by the application, the portions in the set to write to the disk image by:
for each portion in the set:
calculating a hash value by hashing that portion; and
comparing the hash value to a value in an entry of the table that corresponds to the respective block associated with that portion, wherein the hash value matching the value is indicative that the data included in that portion is stored in the respective block associated with that portion.
48. The method of claim 46, further comprising:
calculating, by the application, a hash value for each one of the portions in the set that are written to the disk image; and
storing the hash value, for each one of those portions that are written to the disk image, in an entry of the table that corresponds to the respective block associated with that portion.
49. The method of claim 45, wherein the disk image comprises a set of sections, each of which corresponds to a volume, and wherein the method further comprises:
creating, by the application, a file system for the corresponding volume of each one of the set of sections.
50. The method of claim 49, wherein the causing of the operating system to generate the block-level data includes:
requesting, by the application, the operating system to write the data included in the request to a mount point of the file system corresponding to a particular one of the set of sections.
US16/171,193 2013-10-10 2018-10-25 Backup files to a disk image Abandoned US20190065067A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/171,193 US20190065067A1 (en) 2013-10-10 2018-10-25 Backup files to a disk image

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US14/050,518 US10140051B1 (en) 2013-10-10 2013-10-10 Backup files to a disk image
US16/171,193 US20190065067A1 (en) 2013-10-10 2018-10-25 Backup files to a disk image

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US14/050,518 Continuation US10140051B1 (en) 2013-10-10 2013-10-10 Backup files to a disk image

Publications (1)

Publication Number Publication Date
US20190065067A1 true US20190065067A1 (en) 2019-02-28

Family

ID=64315459

Family Applications (2)

Application Number Title Priority Date Filing Date
US14/050,518 Active 2036-05-30 US10140051B1 (en) 2013-10-10 2013-10-10 Backup files to a disk image
US16/171,193 Abandoned US20190065067A1 (en) 2013-10-10 2018-10-25 Backup files to a disk image

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US14/050,518 Active 2036-05-30 US10140051B1 (en) 2013-10-10 2013-10-10 Backup files to a disk image

Country Status (1)

Country Link
US (2) US10140051B1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9348703B1 (en) 2013-12-23 2016-05-24 Emc Corporation Image recovery from volume image files
US11144401B2 (en) * 2018-11-16 2021-10-12 Vmware, Inc. Component aware incremental backup, restore, and reconciliation solution

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8484427B1 (en) * 2006-06-28 2013-07-09 Acronis International Gmbh System and method for efficient backup using hashes
US8595460B2 (en) * 2011-08-26 2013-11-26 Vmware, Inc. Configuring object storage system for input/output operations
US9098452B2 (en) * 2011-12-19 2015-08-04 International Business Machines Corporation Selecting files to backup in a block level backup
US8527479B2 (en) * 2011-12-21 2013-09-03 International Business Machines Corporation Determining whether a selected backup set satisfies a retention policy
US9152659B2 (en) * 2011-12-30 2015-10-06 Bmc Software, Inc. Systems and methods for migrating database data
US20140181443A1 (en) * 2012-12-21 2014-06-26 Commvault Systems, Inc. Archiving using data obtained during backup of primary storage

Also Published As

Publication number Publication date
US10140051B1 (en) 2018-11-27

Similar Documents

Publication Publication Date Title
US11080232B2 (en) Backup and restoration for a deduplicated file system
US10705919B2 (en) Data backup using metadata mapping
US10705917B2 (en) Consolidated full backup of a restored virtual machine
EP2731013B1 (en) Backing up method, device, and system for virtual machine
US9870151B1 (en) Backup time deduplication of common virtual disks from virtual machine backup images
US11226866B2 (en) Policy driven data updates
US9547655B1 (en) Filesystem independent snapshot driver
US9081792B1 (en) Optimizing backup of whitelisted files
US10725966B1 (en) Block level incremental backup for QCOW2 virtual disks
US9749193B1 (en) Rule-based systems for outcome-based data protection
US20190155534A1 (en) Methods and systems for improving efficiency in cloud-as-backup tier
US8966207B1 (en) Virtual defragmentation of a storage
US9317538B1 (en) Methods for generating data sets using catalog entries
US10776321B1 (en) Scalable de-duplication (dedupe) file system
US20190065067A1 (en) Backup files to a disk image
US11836050B2 (en) Methods and systems for differential based backups
US9804926B1 (en) Cataloging file system-level changes to a source storage between image backups of the source storage
US10303556B1 (en) Modifiable volume snapshots
US10853185B1 (en) Virtual machine image backup to exclude system page file
JP2017531892A (en) Improved apparatus and method for performing a snapshot of a block level storage device
US10496493B1 (en) Method and system for restoring applications of particular point in time
WO2016080949A1 (en) Snapshot backup
US9830471B1 (en) Outcome-based data protection using multiple data protection systems
US11640339B2 (en) Creating a backup data set
US20210365587A1 (en) Data masking in a microservice architecture

Legal Events

Date Code Title Description
STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION