CN112579357B - Snapshot difference obtaining method, device, equipment and storage medium - Google Patents
Snapshot difference obtaining method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112579357B CN112579357B CN202011522534.2A CN202011522534A CN112579357B CN 112579357 B CN112579357 B CN 112579357B CN 202011522534 A CN202011522534 A CN 202011522534A CN 112579357 B CN112579357 B CN 112579357B
- Authority
- CN
- China
- Prior art keywords
- file
- snapshot
- target
- compressed
- difference
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
- G06F11/1453—Management of the data involved in backup or backup restore using de-duplication of the data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a snapshot delta obtaining method, a device, equipment and a storage medium, wherein the method comprises the following steps: compressing the snapshot difference file to generate a compressed file; storing the compressed file through a compressed file system; when a file calling instruction is received, acquiring a target compressed file from the compressed file system according to the file calling instruction; and decompressing the target compressed file to obtain a target snapshot differential file. In the invention, the snapshot difference file is compressed to generate a compressed file and stored in the compressed file system, and then when the file needs to be called, the target compressed file is obtained from the compressed file system and decompressed to obtain the target snapshot difference file, so that the physical disk occupation of the stored snapshot difference file is reduced, and the performance overhead of the acquisition snapshot difference file is reduced.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a snapshot delta obtaining method, apparatus, device, and storage medium.
Background
At present, snapshot delta files are generally stored by using a sparse file system, but the occupation of a physical disk is particularly large due to the storage of the snapshot delta files by the sparse file system, and when the snapshot delta files need to be called, the snapshot delta files need to be called from the sparse file system, so that the performance overhead is increased.
The above is only for the purpose of assisting understanding of the technical aspects of the present invention, and does not represent an admission that the above is prior art.
Disclosure of Invention
The invention mainly aims to provide a snapshot delta acquisition method, a snapshot delta acquisition device, snapshot delta acquisition equipment and a storage medium, and aims to solve the technical problems that a mode of storing and acquiring a snapshot delta file in the prior art occupies a large physical disk and the performance cost is too large.
In order to achieve the above object, the present invention provides a snapshot delta obtaining method, including the following steps:
compressing the snapshot delta file to generate a compressed file;
storing the compressed file through a compressed file system;
when a file calling instruction is received, acquiring a target compressed file from the compressed file system according to the file calling instruction;
and decompressing the target compressed file to obtain a target snapshot differential file.
Optionally, when the file call instruction is received, acquiring the target compressed file from the compressed file system according to the file call instruction includes:
when a file calling instruction is received, determining target file information according to the file calling instruction;
determining a target file identifier according to the target file information;
and acquiring a target compressed file from the compressed file system according to the target file identifier.
Optionally, the obtaining a target compressed file from the compressed file system according to the target file identifier includes:
matching the target file identification with the file identification in the first mapping list;
determining a target compressed file corresponding to the target file identifier according to the matching result;
and searching a target file address corresponding to the target compressed file, and acquiring the target compressed file from the compressed file system according to the target file address.
Optionally, before the matching the target file identifier with the file identifier in the first mapping list, the method further includes:
file identifications are distributed for the compressed files, a first mapping list is established according to the file identifications and the compressed files, and the corresponding relation between the file identifications and the compressed files is recorded in the first mapping list.
Optionally, the determining a target file identifier according to the target file information includes:
determining differential file information according to the target file information, and extracting a target differential file name from the differential file information;
matching the target differential file name with the differential file name in a second mapping list;
and determining a target file identifier corresponding to the target difference file name according to the matching result.
Optionally, before the matching the target differential file name with the differential file name in the second mapping list, the method further includes:
acquiring a difference file name of each snapshot difference quantity file;
and establishing a second mapping list according to the difference file name and the file identifier of the compressed file corresponding to the snapshot difference amount file, wherein the second mapping list records the corresponding relation between the difference file name and the file identifier.
Optionally, the determining the difference file information according to the target file information includes:
determining a file to be called according to the target file information, and detecting whether the file to be called has a difference file;
and when the file to be called has a difference file, searching the difference file information corresponding to the file to be called.
Optionally, after the decompressing the target compressed file to obtain the target snapshot delta file, the method further includes:
and carrying out data processing on the file to be called according to the target snapshot difference file so as to obtain a target file.
Optionally, before the compressing the snapshot delta file to generate a compressed file, the method further includes:
when a snapshot file is received, detecting the snapshot file to determine a file type corresponding to the snapshot file;
and generating a snapshot difference quantity file according to the file type and the snapshot file.
Optionally, the generating a snapshot delta file according to the file type and the snapshot file includes:
judging whether the file type is a pointer type snapshot or not;
and when the file type is a pointer type snapshot, taking the snapshot file as a snapshot differential file.
Optionally, after determining whether the file type is a pointer-type snapshot, the method further includes:
when the file type is not the pointer snapshot, judging whether the file type is the mirror snapshot or not;
when the file type is a mirror image type snapshot, searching an original file corresponding to the snapshot file;
and generating a snapshot difference file according to the snapshot file and the original file.
Optionally, the compressing the snapshot delta file to generate a compressed file includes:
splitting the snapshot difference file into a plurality of files to be compressed according to a sequence, and recording a corresponding arrangement sequence of the files to be compressed;
and compressing the files to be compressed according to the corresponding arrangement sequence of the files to be compressed to generate compressed files.
Optionally, the decompressing the target compressed file to obtain a target snapshot delta file includes:
decompressing the target compressed file to obtain a plurality of decompressed files;
detecting the arrangement sequence of the decompressed files;
and sequencing the decompressed files according to the arrangement sequence of the decompressed files to obtain a target snapshot differential file.
In addition, to achieve the above object, the present invention further provides a snapshot delta obtaining apparatus, including:
the file compression module is used for compressing the snapshot difference quantity file to generate a compressed file;
the file storage module is used for storing the compressed file through a compressed file system;
the file calling module is used for acquiring a target compressed file from the compressed file system according to a file calling instruction when receiving the file calling instruction;
and the file decompression module is used for decompressing the target compressed file to obtain a target snapshot differential file.
Optionally, the file calling module is further configured to determine target file information according to the file calling instruction when the file calling instruction is received; determining a target file identifier according to the target file information; and acquiring a target compressed file from the compressed file system according to the target file identifier.
Optionally, the file calling module is further configured to match the target file identifier with a file identifier in a first mapping list;
determining a target compressed file corresponding to the target file identifier according to the matching result; and searching a target file address corresponding to the target compressed file, and acquiring the target compressed file from the compressed file system according to the target file address.
Optionally, the file calling module is further configured to allocate a file identifier for each compressed file, and establish a first mapping list according to the file identifier and the compressed file, where a corresponding relationship between the file identifier and the compressed file is recorded in the first mapping list.
Optionally, the file calling module is further configured to determine difference file information according to the target file information, and extract a target difference file name from the difference file information; matching the target differential file name with a differential file name in a second mapping list; and determining a target file identifier corresponding to the target difference file name according to the matching result.
In addition, to achieve the above object, the present invention further provides a snapshot delta obtaining apparatus, including: the snapshot delta acquisition program is stored on the memory and can run on the processor, and when being executed by the processor, the snapshot delta acquisition program realizes the steps of the snapshot delta acquisition method.
In addition, in order to achieve the above object, the present invention further provides a storage medium, on which a snapshot delta acquisition program is stored, and the snapshot delta acquisition program implements the steps of the snapshot delta acquisition method as described above when executed by a processor.
The snapshot delta acquisition method provided by the invention compresses the snapshot delta file to generate a compressed file; storing the compressed file through a compressed file system; when a file calling instruction is received, acquiring a target compressed file from the compressed file system according to the file calling instruction; and decompressing the target compressed file to obtain a target snapshot differential file. In the invention, the snapshot difference amount file is compressed to generate a compressed file and is stored in the compressed file system, and then when the file needs to be called, the target compressed file is obtained from the compressed file system and is decompressed to obtain the target snapshot difference amount file, so that the physical disk occupation of the stored snapshot difference amount file is reduced, and the performance overhead when the snapshot difference amount file is obtained is reduced.
Drawings
FIG. 1 is a schematic diagram of a snapshot delta acquisition device in a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a snapshot delta acquisition method according to a first embodiment of the present invention;
FIG. 3 is a schematic diagram of file compression storage according to an embodiment of a snapshot delta obtaining method of the present invention;
FIG. 4 is a flowchart illustrating a snapshot delta acquisition method according to a second embodiment of the present invention;
FIG. 5 is a flowchart illustrating a snapshot delta acquisition method according to a third embodiment of the present invention;
FIG. 6 is a flowchart illustrating a snapshot delta acquisition method according to a fourth embodiment of the present invention;
fig. 7 is a functional block diagram of a snapshot delta acquisition device according to a first embodiment of the invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a snapshot delta obtaining device in a hardware operating environment according to an embodiment of the present invention.
As shown in fig. 1, the snapshot delta acquisition device may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may comprise a Display screen (Display), an input unit such as keys, and the optional user interface 1003 may also comprise a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The Memory 1005 may be a Random Access Memory (RAM) Memory or a non-volatile Memory (e.g., a magnetic disk Memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the device configuration shown in fig. 1 does not constitute a limitation of the snapshot delta acquisition device, and may include more or fewer components than those shown, or some components in combination, or a different arrangement of components.
As shown in fig. 1, the memory 1005, which is a storage medium, may include therein an operating system, a network communication module, a user interface module, and a snapshot delta acquisition program.
In the snapshot delta acquisition device shown in fig. 1, the network interface 1004 is mainly used for connecting to an external network and performing data communication with other network devices; the user interface 1003 is mainly used for connecting a user equipment and performing data communication with the user equipment; the device of the present invention calls the snapshot delta acquisition program stored in the memory 1005 through the processor 1001, and executes the snapshot delta acquisition method provided by the embodiment of the present invention.
Based on the above hardware structure, an embodiment of the snapshot delta obtaining method of the present invention is provided.
Referring to fig. 2, fig. 2 is a flowchart illustrating a snapshot delta obtaining method according to a first embodiment of the present invention.
In a first embodiment, the snapshot delta acquisition method includes the following steps:
and step S10, compressing the snapshot difference quantity file to generate a compressed file.
It should be noted that the execution main body of this embodiment may be a snapshot delta obtaining device, such as a terminal device, and may also be other devices that can achieve the same or similar functions. The terminal device may include a mobile phone, an IOT device, and other devices, which is not limited in this embodiment.
It should be understood that, at present, the snapshot delta file is generally stored by a sparse file system, but this approach may occupy a large physical disk, and in this embodiment, to overcome this problem, a compressed file system is used to store the snapshot delta file, so as to reduce the occupation of the physical disk.
It can be understood that, since the snapshot delta file is stored by using a compressed file system, the compressed file system is used for storing the compressed file, and in order to save storage space, the snapshot delta file may be compressed first to generate the compressed file. Because the compressed file has a smaller volume compared with the original snapshot difference file, the occupied physical disk is also smaller.
And step S20, storing the compressed file through a compressed file system.
It should be understood that after the compressed file is generated, the compressed file may be stored by the compressed file system, and specifically, the compressed file may be hung on the compressed file system to obtain an effect of storing the compressed file.
It can be understood that, since there may be multiple snapshot delta files, the snapshot delta files may be compressed separately to obtain multiple compressed files, and the compressed files are stored by the compressed file system.
And step S30, when a file calling instruction is received, acquiring a target compressed file from the compressed file system according to the file calling instruction.
It should be understood that, when receiving the file call instruction, the terminal device may determine the target compressed file to be called according to the file call instruction, and may further obtain the target compressed file from the compressed file system.
In a specific implementation, as shown in fig. 3, fig. 3 is a schematic diagram of file compression storage, and it is assumed that there are 3 snapshot delta files, which are respectively: the delta file 1, the delta file 2, and the delta file 3 may be compressed for the 3 snapshot delta files, respectively, to generate 3 compressed files, which are: compressing the file 1, the file 2 and the file 3, storing the file 1, the file 2 and the file 3 in a compressed file system, and then compressing the file from the compressed file system according to a file calling instruction when receiving the file calling instruction.
And S40, decompressing the target compressed file to obtain a target snapshot differential file.
It should be understood that after the target compressed file is obtained from the compressed file system, the target compressed file may be decompressed, and then the target snapshot delta file may be obtained. In a specific implementation, for example, when it is determined that a target compressed file to be acquired is a compressed file 2 according to a file call instruction, the compressed file 2 may be acquired from a compressed file system, and decompression processing is performed on the compressed file 2 to obtain a delta file 2, and the delta file 2 is used as a target snapshot delta file.
In the embodiment, a snapshot difference file is compressed to generate a compressed file; storing the compressed file through a compressed file system; when a file calling instruction is received, acquiring a target compressed file from the compressed file system according to the file calling instruction; and decompressing the target compressed file to obtain a target snapshot differential file. In this embodiment, the snapshot difference file is compressed to generate a compressed file and stored in the compressed file system, and then when the file needs to be called, the target compressed file is obtained from the compressed file system and decompressed to obtain the target snapshot difference file, so that the physical disk occupation of the stored snapshot difference file is reduced, and thus the performance overhead of obtaining the snapshot difference file is reduced.
In an embodiment, as shown in fig. 4, a second embodiment of the snapshot delta obtaining method according to the present invention is proposed based on the first embodiment, and the step S30 includes:
step S301, when a file calling instruction is received, determining target file information according to the file calling instruction.
It should be understood that when a file call instruction is received, target file information may be determined according to the file call instruction, where the target file information is file information that the file call instruction needs to call.
Step S302, determining the target file identification according to the target file information.
It can be understood that, after the target file information is determined, the target file identifier may be determined according to the target file information, and then the target compressed file may be obtained from the compressed file system according to the target file identifier.
Step S303, obtaining the target compressed file from the compressed file system according to the target file identifier.
It should be understood that after the snapshot delta file is compressed to obtain compressed files, a file identifier may be allocated to each compressed file, because the file identifier is unique, after the target file identifier is determined, the target compressed file may be obtained from the compressed file system according to the target file identifier.
Further, in order to determine a corresponding compressed file more quickly according to a file identifier, and conveniently acquire the compressed file from a compressed file system, and improve file acquisition efficiency, the acquiring a target compressed file from the compressed file system according to the target file identifier includes:
matching the target file identification with the file identification in the first mapping list; determining a target compressed file corresponding to the target file identifier according to the matching result; and searching a target file address corresponding to the target compressed file, and acquiring the target compressed file from the compressed file system according to the target file address.
It should be understood that a first mapping list may be established in advance according to the file identifier and the compressed file, after the target file identifier is determined, the target file identifier may be matched with the file identifier in the first mapping list, and then the target compressed file corresponding to the target file identifier may be determined according to the matching result.
Further, before the matching the target file identifier with the file identifier in the first mapping list, the method further includes:
distributing file identifications for each compressed file, and establishing a first mapping list according to the file identifications and the compressed files, wherein the first mapping list records the corresponding relation between the file identifications and the compressed files.
It is understood that after obtaining the compressed files, file identifiers may be allocated to the compressed files, and a first mapping list may be established according to the file identifiers and the compressed files. In a specific implementation, for example, 3 compressed files are generated, which are respectively a compressed file 1, a compressed file 2, and a compressed file 3, a file identifier 1 may be allocated to the compressed file 1, a file identifier 2 may be allocated to the compressed file 2, and a file identifier 3 may be allocated to the compressed file 3, so that a first mapping list may be established according to the compressed files and the file identifiers, where a correspondence relationship exists between the compressed file 1 and the file identifier 1, a correspondence relationship exists between the compressed file 2 and the file identifier 2, and a correspondence relationship exists between the compressed file 3 and the file identifier 3. If the target file id is file id 2, the compressed file 2 corresponding to file id 2 can be used as the target compressed file.
It should be understood that, since the compressed file is stored by the compressed file system, the compressed file is generally stored in a block space of the compressed file system, and in order to distinguish the storage spaces of different compressed files, the file address of the compressed file in the compressed file system may be recorded when the compressed file is stored.
In a specific implementation, for example, when storing the compressed file 1 in the compressed file system, the file address 1 corresponding to the compressed file 1 may be recorded, when storing the compressed file 2 in the compressed file system, the file address 2 corresponding to the compressed file 2 may be recorded, and when storing the compressed file 3 in the compressed file system, the file address 3 corresponding to the compressed file 3 may be recorded.
It can be understood that, after determining the target compressed file corresponding to the target file identifier, the target file address corresponding to the target compressed file may be searched, specifically, a third mapping list is established according to the compressed file and the file address, a corresponding relationship between the compressed file and the file address is recorded in the third mapping list, and after determining the target compressed file, the target file address corresponding to the target compressed file is searched according to the third mapping list.
In a specific implementation, for example, when the target compressed file is the compressed file 2, the target file address corresponding to the compressed file 2 may be searched for as the file address 2 according to the third mapping list.
Further, in order to improve the efficiency of determining the target file identifier according to the target file information and improve the differential quantity obtaining speed, the determining the target file identifier according to the target file information includes:
determining difference file information according to the target file information, and extracting a target difference file name from the difference file information; matching the target differential file name with the differential file name in a second mapping list; and determining a target file identifier corresponding to the target difference file name according to the matching result.
It should be understood that after determining the target file information, the differential file information may be determined according to the target file information, and specifically may be: determining a file to be called according to the target file information, and detecting whether the file to be called has a difference file; and when the file to be called has a difference file, searching the difference file information corresponding to the file to be called.
It can be understood that, when a file is called, some files have a difference file, and therefore, a target compressed file needs to be acquired from a compressed file system to decompress to obtain a target snapshot difference file, but some files do not have a difference file, in this case, if the step of acquiring the target compressed file from the compressed file system is still performed, the target compressed file cannot be acquired, so that the operating efficiency of the system becomes low, and a failure easily occurs.
In this embodiment, in order to improve the operating efficiency of the system and reduce the occurrence probability of a failure, after determining the file to be called, whether the file to be called has a difference file may be detected, and only when the file to be called has the difference file, the step of searching for the difference file information corresponding to the file to be called is performed. And if the file to be called has no difference file, directly feeding back the file to be called as a target file to the calling process.
It should be understood that, after determining the differential file information corresponding to the file to be called through the above steps, the differential file name may be extracted from the differential file information, and then the target file identifier corresponding to the differential file name may be searched according to the second mapping list.
Further, before the matching the target differential file name with the differential file name in the second mapping list, the method further includes:
acquiring a difference file name of each snapshot difference quantity file; and establishing a second mapping list according to the difference file name and the file identifier of the compressed file corresponding to the snapshot difference amount file, wherein the second mapping list records the corresponding relation between the difference file name and the file identifier.
It should be understood that, since each snapshot difference file has a corresponding file name, different snapshot difference files can be distinguished through the file names, so that a difference file name of each snapshot difference file can be obtained, a second mapping list is established according to the difference file name and a file identifier of a compressed file corresponding to the snapshot difference file, and then after a target difference file name is determined, a target file identifier corresponding to the target difference file name can be searched according to the second mapping list.
In a specific implementation, for example, the three snapshot delta files are respectively: difference file 1 corresponds to a first difference file name, difference file 2 corresponds to a second difference file name, and difference file 3 corresponds to a third difference file name. The difference file 1 corresponds to the compressed file 1, the compressed file 1 corresponds to the file identifier 1, the difference file 2 corresponds to the compressed file 2, the compressed file 2 corresponds to the file identifier 2, the difference file 3 corresponds to the compressed file 3, and the compressed file 3 corresponds to the file identifier 3. Thus, it can be determined that the correspondence between the differential file name and the file identifier of the compressed file corresponding to the snapshot differential file is: the first differential file name corresponds to file identification 1, the second differential file name corresponds to file identification 2, and the third differential file name corresponds to file identification 3. Therefore, the second mapping list may be established according to the correspondence between the differential file name and the file identifier.
In a specific implementation, when the target differential file name is the second differential file name, the file identifier 2 corresponding to the second differential file name may be searched from the second mapping class table, and the file identifier 2 is used as the target file identifier corresponding to the target differential file name.
Further, since the file call instruction may be for calling a complete file, not a delta file, after the decompressing the target compressed file to obtain a target snapshot delta file, the method further includes:
and carrying out data processing on the file to be called according to the target snapshot difference file so as to obtain a target file.
It can be understood that the data processing may be data reduction processing, and since the snapshot delta file stores delta data of a file, after the target snapshot delta file is obtained, target delta data may be extracted from the target snapshot delta file, and data reduction processing may be performed on the file to be called according to the target delta data to obtain a target file, and then the target file is fed back to the calling process.
In the embodiment, when a file calling instruction is received, target file information is determined according to the file calling instruction; determining a target file identifier according to the target file information; and acquiring the target compressed file from the compressed file system according to the target file identifier, so that the target file identifier can be determined according to the calling instruction in a mode of distributing file identifiers for the compressed files, the target compressed file can be accurately acquired from the compressed file system according to the target file identifier, and the acquisition efficiency of the target compressed file is improved.
In an embodiment, as shown in fig. 5, a third embodiment of the snapshot delta obtaining method according to the present invention is provided based on the first embodiment or the second embodiment, and in this embodiment, the description is based on the first embodiment, before the step S10, the method further includes:
step S01, when the snapshot file is received, detecting the snapshot file to determine a file type corresponding to the snapshot file.
It should be understood that snapshots are roughly classified into 2 types, one type is called copy-on-write (copy-on-write) snapshot and is also commonly called pointer-type snapshot, VSS belongs to this type, and the other type is called split-mirror snapshot and is often called mirror-type snapshot. The pointer type snapshot occupies small space and has small influence on the system performance, but if the original data disk is damaged without backup, the data cannot be recovered; the mirror image type snapshot is actually a full mirror image of the data at that time, and will cause a certain load on the system performance, but even if the original data is damaged, the system will not have much influence, but will occupy the space with the same capacity.
Therefore, in this embodiment, the file types may be divided into a pointer type snapshot and a mirror type snapshot, and when the snapshot file is received, the snapshot file may be detected first to determine whether the file type corresponding to the snapshot file is a pointer type snapshot or a mirror type snapshot.
And S02, generating a snapshot difference file according to the file type and the snapshot file.
It will be appreciated that after determining the file type of the snapshot file, the snapshot delta file may be generated in different ways depending on the different file types.
Further, the generating a snapshot delta file according to the file type and the snapshot file includes:
judging whether the file type is a pointer type snapshot or not; and when the file type is a pointer type snapshot, taking the snapshot file as a snapshot differential file.
It should be understood that whether the file type is a pointer snapshot or not may be determined first, and when the file type is a pointer snapshot, since the snapshot file itself stores the delta in this case, the snapshot file may be directly used as the snapshot delta file.
Further, after determining whether the file type is a pointer-type snapshot, the method further includes:
when the file type is not the pointer snapshot, judging whether the file type is the mirror snapshot or not; when the file type is a mirror image type snapshot, searching an original file corresponding to the snapshot file; and generating a snapshot difference file according to the snapshot file and the original file.
It should be understood that when the file type is not a pointer-type snapshot, it may be further determined whether the file type is a mirror-image snapshot, and when the file type is a mirror-image snapshot, it indicates that a complete mirror-image file is recorded in the snapshot file at this time.
It can be understood that, if the snapshot file is determined to be neither a pointer snapshot nor a mirror snapshot after the above determining step is performed, it indicates that there may be a detection error. Therefore, when the file type is not the mirror image type snapshot, the step of judging whether the file type is the pointer type snapshot is returned to be executed, and when the situation is repeated for a plurality of times, the failure of the file type detection can be judged, the file type detection step can be suspended, and the error information can be fed back.
In this embodiment, when a snapshot file is received, the snapshot file is detected to determine a file type corresponding to the snapshot file, the snapshot differential quantity file is generated according to the file type and the snapshot file, and the snapshot file can be processed according to a mode adapted to the file type to obtain the snapshot differential quantity file, so that the generation efficiency of the snapshot differential quantity file is improved.
In an embodiment, as shown in fig. 6, a fourth embodiment of the snapshot delta obtaining method according to the present invention is provided based on the first embodiment, the second embodiment, or the third embodiment, in this embodiment, the description is made based on the first embodiment, and the step S10 includes:
step S101, splitting the snapshot difference file into a plurality of files to be compressed according to a sequence, and recording the corresponding arrangement sequence of the files to be compressed.
It should be understood that since the volume of the snapshot delta file may be large, it may not be good to compress the snapshot delta file directly, and in order to maintain the order of the delta, the snapshot delta file may be split into a plurality of files to be compressed in sequence.
In a specific implementation, when the snapshot difference file is a file 2, the file 2 may be split into 5 files to be compressed, which are: file 21, file 22, file 23, file 24, and file 25, and then file 21, file 22, file 23, file 24, and file 25 may be recorded in the order file 21-file 25.
And S102, compressing the files to be compressed according to the corresponding arrangement sequence of the files to be compressed to generate compressed files.
It should be understood that the compressed file may be compressed according to the arrangement order to obtain a plurality of sub-compressed files, and then the compressed file may be generated according to the sub-compressed files.
In a specific implementation, the file 21, the file 22, the file 23, the file 24, and the file 25 may be compressed to obtain the sub compressed file 21, the sub compressed file 22, the sub compressed file 23, the sub compressed file 24, and the sub compressed file 25, and then the sub compressed files may be compressed for the second time to obtain the compressed file 2.
Further, in order to accurately obtain a target snapshot delta file, the decompressing the target compressed file to obtain the target snapshot delta file includes:
decompressing the target compressed file to obtain a plurality of decompressed files; detecting the arrangement sequence of the decompressed files; and sequencing the decompressed files according to the arrangement sequence of the decompressed files to obtain a target snapshot differential file.
It should be understood that, the target compressed file may be decompressed to obtain a plurality of target sub-compressed files, and then each target sub-compressed file is decompressed to obtain a plurality of decompressed files, and then the arrangement order of the decompressed files is detected, and the decompressed files are sorted according to the arrangement order to obtain the target snapshot delta file.
In a specific implementation, when the target compressed file is the compressed file 2, the compressed file 2 may be decompressed to obtain a sub-compressed file, a sub-compressed file 22, a sub-compressed file 23, a sub-compressed file 24, and a sub-compressed file 25, then the sub-compressed files are respectively decompressed to obtain a file 21, a file 22, a file 23, a file 24, and a file 25, the file 21, the file 22, the file 23, the file 24, and the file 25 are used as decompressed files, and the decompressed files are sorted according to a sorting order to obtain the target snapshot difference file.
In the embodiment, the snapshot difference file is sequentially split into a plurality of files to be compressed, and the corresponding arrangement sequence of the files to be compressed is recorded; and compressing the file to be compressed according to the arrangement sequence corresponding to the file to be compressed to generate a compressed file, wherein the snapshot delta file can be split into a plurality of files for compression because the snapshot delta file is possibly large, so that the compression efficiency is improved, and the occupation of a physical disk is further reduced.
In addition, an embodiment of the present invention further provides a storage medium, where a snapshot delta obtaining program is stored on the storage medium, and when being executed by a processor, the snapshot delta obtaining program implements the steps of the snapshot delta obtaining method as described above.
Since the storage medium adopts all technical solutions of all the embodiments, at least all the beneficial effects brought by the technical solutions of the embodiments are achieved, and no further description is given here.
In addition, referring to fig. 7, an embodiment of the present invention further provides a snapshot delta obtaining apparatus, where the snapshot delta obtaining apparatus includes:
the file compression module 10 is configured to perform compression processing on the snapshot delta file to generate a compressed file.
And the file storage module 20 is used for storing the compressed file through a compressed file system.
And the file calling module 30 is configured to, when receiving a file calling instruction, obtain a target compressed file from the compressed file system according to the file calling instruction.
And a file decompression module 40, configured to decompress the target compressed file to obtain a target snapshot delta file.
In the embodiment, a snapshot difference file is compressed to generate a compressed file; storing the compressed file through a compressed file system; when a file calling instruction is received, acquiring a target compressed file from the compressed file system according to the file calling instruction; and decompressing the target compressed file to obtain a target snapshot differential file. In this embodiment, the snapshot difference amount file is compressed to generate a compressed file and stored in the compressed file system, and when the file needs to be called, the target compressed file is obtained from the compressed file system and decompressed to obtain the target snapshot difference amount file, so that the physical disk occupation of the stored snapshot difference amount file is reduced, and the performance overhead when the snapshot difference amount file is obtained is reduced.
In an embodiment, the file calling module 30 is further configured to obtain a difference file name of each snapshot difference amount file; and establishing a second mapping list according to the difference file name and the file identifier of the compressed file corresponding to the snapshot difference amount file, wherein the second mapping list records the corresponding relation between the difference file name and the file identifier.
In an embodiment, the file calling module 30 is further configured to determine a file to be called according to the target file information, and detect whether a difference file exists in the file to be called; and when the file to be called has a difference file, searching for difference file information corresponding to the file to be called.
In an embodiment, the file calling module 30 is further configured to perform data processing on the file to be called according to the target snapshot difference file, so as to obtain a target file.
In an embodiment, the snapshot delta obtaining apparatus further includes a delta file module, configured to detect the snapshot file when the snapshot file is received, so as to determine a file type corresponding to the snapshot file; and generating a snapshot difference quantity file according to the file type and the snapshot file.
In an embodiment, the delta file module is further configured to determine whether the file type is a pointer-type snapshot; and when the file type is a pointer snapshot, taking the snapshot file as a snapshot differential file.
In an embodiment, the delta file module is further configured to determine whether the file type is a mirror snapshot when the file type is not a pointer snapshot; when the file type is a mirror image type snapshot, searching an original file corresponding to the snapshot file; and generating a snapshot difference file according to the snapshot file and the original file.
In an embodiment, the file compression module 10 is further configured to split the snapshot difference file into a plurality of files to be compressed in sequence, and record an arrangement sequence corresponding to the files to be compressed; and compressing the files to be compressed according to the corresponding arrangement sequence of the files to be compressed to generate compressed files.
In an embodiment, the file decompression module 40 is further configured to decompress the target compressed file to obtain a plurality of decompressed files; detecting the arrangement sequence of the decompressed files; and sequencing the decompressed files according to the arrangement sequence of the decompressed files to obtain a target snapshot differential file.
In other embodiments or specific implementation methods of the snapshot delta acquisition device of the present invention, reference may be made to the above method embodiments, which are not described herein again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a computer-readable storage medium (such as ROM/RAM, magnetic disk, optical disk) as described above, and includes instructions for enabling an intelligent device (such as a mobile phone, an estimator, a snapshot difference obtaining device, or a network snapshot difference obtaining device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (17)
1. A snapshot delta acquisition method is characterized by comprising the following steps:
compressing the snapshot delta file to generate a compressed file;
storing the compressed file through a compressed file system;
when a file calling instruction is received, acquiring a target compressed file from the compressed file system according to the file calling instruction;
decompressing the target compressed file to obtain a target snapshot differential file;
before compressing the snapshot delta file to generate a compressed file, the method further includes:
when a snapshot file is received, detecting the snapshot file to determine a file type corresponding to the snapshot file;
generating a snapshot difference quantity file according to the file type and the snapshot file;
the generating a snapshot delta file according to the file type and the snapshot file comprises:
judging whether the file type is a pointer type snapshot or not;
when the file type is a pointer snapshot, the snapshot file is used as a snapshot difference file;
after the step of judging whether the file type is the pointer type snapshot, the method further comprises the following steps:
when the file type is not the pointer snapshot, judging whether the file type is the mirror snapshot or not;
when the file type is a mirror image type snapshot, searching an original file corresponding to the snapshot file;
and generating a snapshot difference file according to the snapshot file and the original file.
2. The snapshot delta acquisition method as recited in claim 1, wherein said acquiring a target compressed file from said compressed file system according to a file call instruction when receiving said file call instruction comprises:
when a file calling instruction is received, determining target file information according to the file calling instruction;
determining a target file identifier according to the target file information;
and acquiring a target compressed file from the compressed file system according to the target file identifier.
3. The snapshot delta acquisition method as recited in claim 2 wherein said acquiring a target compressed file from said compressed file system based on said target file identification comprises:
matching the target file identification with the file identification in the first mapping list;
determining a target compressed file corresponding to the target file identifier according to the matching result;
and searching a target file address corresponding to the target compressed file, and acquiring the target compressed file from the compressed file system according to the target file address.
4. The snapshot delta acquisition method as recited in claim 3, wherein prior to said matching said target file identification with a file identification in a first mapping list, further comprising:
distributing file identifications for each compressed file, and establishing a first mapping list according to the file identifications and the compressed files, wherein the first mapping list records the corresponding relation between the file identifications and the compressed files.
5. The snapshot delta acquisition method as recited in claim 4 wherein said determining a target file identification based on said target file information comprises:
determining difference file information according to the target file information, and extracting a target difference file name from the difference file information;
matching the target differential file name with the differential file name in a second mapping list;
and determining a target file identifier corresponding to the target difference file name according to the matching result.
6. The snapshot delta acquisition method as recited in claim 5 wherein prior to said matching said target differential file name to a differential file name in a second mapping list, further comprising:
acquiring a difference file name of each snapshot difference quantity file;
and establishing a second mapping list according to the difference file name and the file identifier of the compressed file corresponding to the snapshot difference amount file, wherein the second mapping list records the corresponding relation between the difference file name and the file identifier.
7. The snapshot delta acquisition method as recited in claim 5 wherein said determining differential file information based on said target file information comprises:
determining a file to be called according to the target file information, and detecting whether the file to be called has a difference file;
and when the file to be called has a difference file, searching the difference file information corresponding to the file to be called.
8. The snapshot delta acquisition method according to claim 7, wherein after said decompressing the target compressed file to obtain the target snapshot delta file, further comprising:
and carrying out data processing on the file to be called according to the target snapshot difference file so as to obtain a target file.
9. The snapshot delta acquisition method according to any one of claims 1 to 8, wherein said compressing the snapshot delta file to generate a compressed file comprises:
splitting the snapshot difference file into a plurality of files to be compressed according to a sequence, and recording a corresponding arrangement sequence of the files to be compressed;
and compressing the files to be compressed according to the arrangement sequence corresponding to the files to be compressed to generate compressed files.
10. The snapshot delta acquisition method as claimed in claim 9, wherein said decompressing said target compressed file to obtain a target snapshot delta file comprises:
decompressing the target compressed file to obtain a plurality of decompressed files;
detecting the arrangement sequence of the decompressed files;
and sequencing the decompressed files according to the arrangement sequence of the decompressed files to obtain a target snapshot differential file.
11. A snapshot delta acquisition device, comprising:
the file compression module is used for compressing the snapshot difference quantity file to generate a compressed file;
the file storage module is used for storing the compressed file through a compressed file system;
the file calling module is used for acquiring a target compressed file from the compressed file system according to a file calling instruction when the file calling instruction is received;
the file decompression module is used for decompressing the target compressed file to obtain a target snapshot differential file;
wherein, the snapshot delta obtaining device further comprises:
the snapshot file detection module is used for detecting the snapshot file when the snapshot file is received so as to determine a file type corresponding to the snapshot file; generating a snapshot difference quantity file according to the file type and the snapshot file;
the delta file module is also used for judging whether the file type is a pointer snapshot or not; when the file type is a pointer snapshot, taking the snapshot file as a snapshot differential file;
the delta file module is further used for judging whether the file type is a mirror image snapshot or not when the file type is not a pointer snapshot; when the file type is a mirror image type snapshot, searching an original file corresponding to the snapshot file; and generating a snapshot difference file according to the snapshot file and the original file.
12. The snapshot delta acquisition device as recited in claim 11, wherein the file call module is further configured to determine target file information according to a file call instruction when receiving the file call instruction; determining a target file identifier according to the target file information; and acquiring a target compressed file from the compressed file system according to the target file identifier.
13. The snapshot delta acquisition device as recited in claim 12 wherein said file call module is further configured to match said target file identifier with a file identifier in a first mapping list; determining a target compressed file corresponding to the target file identifier according to the matching result; and searching a target file address corresponding to the target compressed file, and acquiring the target compressed file from the compressed file system according to the target file address.
14. The snapshot delta acquisition device as recited in claim 13, wherein the file calling module is further configured to allocate a file identifier for each compressed file, and establish a first mapping list according to the file identifier and the compressed file, where a corresponding relationship between the file identifier and the compressed file is recorded in the first mapping list.
15. The snapshot delta acquisition device as claimed in claim 14, wherein said file calling module is further configured to determine difference file information according to said target file information, and extract a target difference file name from said difference file information; matching the target differential file name with the differential file name in a second mapping list; and determining a target file identifier corresponding to the target difference file name according to the matching result.
16. A snapshot delta acquisition device, comprising: a memory, a processor, and a snapshot delta acquisition program stored on the memory and executable on the processor, the snapshot delta acquisition program configured with steps to implement the snapshot delta acquisition method of any of claims 1-10.
17. A storage medium having stored thereon a snapshot delta acquisition program which, when executed by a processor, implements the steps of the snapshot delta acquisition method as recited in any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011522534.2A CN112579357B (en) | 2020-12-23 | 2020-12-23 | Snapshot difference obtaining method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011522534.2A CN112579357B (en) | 2020-12-23 | 2020-12-23 | Snapshot difference obtaining method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112579357A CN112579357A (en) | 2021-03-30 |
| CN112579357B true CN112579357B (en) | 2022-11-04 |
Family
ID=75136505
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011522534.2A Active CN112579357B (en) | 2020-12-23 | 2020-12-23 | Snapshot difference obtaining method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112579357B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116701238A (en) * | 2022-09-14 | 2023-09-05 | 荣耀终端有限公司 | Drive file processing method and related device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101714089A (en) * | 2008-10-07 | 2010-05-26 | 英业达股份有限公司 | Method for installing external members and method for manufacturing mapping files of external members |
| CN103999034A (en) * | 2011-12-20 | 2014-08-20 | Netapp股份有限公司 | System, method and computer program product for providing sparse snapshots |
| CN104781791A (en) * | 2011-12-05 | 2015-07-15 | 持续电信解决方案公司 | Universal pluggable cloud disaster recovery system |
| CN105162869A (en) * | 2015-09-18 | 2015-12-16 | 久盈世纪（北京）科技有限公司 | Data backup management method and equipment |
| CN106598785A (en) * | 2016-12-16 | 2017-04-26 | 广东美晨通讯有限公司 | File system backup and restoration method and device |
| US20170357551A1 (en) * | 2016-06-08 | 2017-12-14 | Red Hat Israel, Ltd. | Snapshot version control |
| CN108959614A (en) * | 2018-07-17 | 2018-12-07 | 郑州云海信息技术有限公司 | A kind of snapshot management method, system, device, equipment and readable storage medium storing program for executing |
| CN110114751A (en) * | 2017-01-06 | 2019-08-09 | 甲骨文国际公司 | The ZFS cloud storage of the End to End Encryption of compression and safety |
-
2020
- 2020-12-23 CN CN202011522534.2A patent/CN112579357B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101714089A (en) * | 2008-10-07 | 2010-05-26 | 英业达股份有限公司 | Method for installing external members and method for manufacturing mapping files of external members |
| CN104781791A (en) * | 2011-12-05 | 2015-07-15 | 持续电信解决方案公司 | Universal pluggable cloud disaster recovery system |
| CN103999034A (en) * | 2011-12-20 | 2014-08-20 | Netapp股份有限公司 | System, method and computer program product for providing sparse snapshots |
| CN105162869A (en) * | 2015-09-18 | 2015-12-16 | 久盈世纪（北京）科技有限公司 | Data backup management method and equipment |
| US20170357551A1 (en) * | 2016-06-08 | 2017-12-14 | Red Hat Israel, Ltd. | Snapshot version control |
| CN106598785A (en) * | 2016-12-16 | 2017-04-26 | 广东美晨通讯有限公司 | File system backup and restoration method and device |
| CN110114751A (en) * | 2017-01-06 | 2019-08-09 | 甲骨文国际公司 | The ZFS cloud storage of the End to End Encryption of compression and safety |
| CN108959614A (en) * | 2018-07-17 | 2018-12-07 | 郑州云海信息技术有限公司 | A kind of snapshot management method, system, device, equipment and readable storage medium storing program for executing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112579357A (en) | 2021-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106936441B (en) | Data compression method and device | |
| US20190146770A1 (en) | Method for Identifying File and Mobile Terminal | |
| CN113127125B (en) | Page automatic adaptation method, device, equipment and storage medium | |
| CN110808738A (en) | Data compression method, device, equipment and computer readable storage medium | |
| CN104182519A (en) | File scanning method and device | |
| CN116340367B (en) | Data query method, device, equipment and storage medium | |
| CN112579357B (en) | Snapshot difference obtaining method, device, equipment and storage medium | |
| CN110069406B (en) | Automatic triggering TPC-DS test method and system | |
| CN109246280B (en) | Address book cloud processing method and device, computer equipment and readable storage medium | |
| CN107169057B (en) | Method and device for detecting repeated pictures | |
| CN110647753B (en) | Method, device and equipment for acquiring kernel file and storage medium | |
| CN111090623A (en) | Data auditing method and device, electronic equipment and storage medium | |
| CN115795066A (en) | Method and related device for interconversion of iBootim image and PNG image | |
| CN110727565A (en) | Network equipment platform information collection method and system | |
| CN112257106B (en) | Data detection method and device | |
| CN110888686B (en) | Application program starting method, device and storage medium | |
| CN110413603B (en) | Method and device for determining repeated data, electronic equipment and computer storage medium | |
| CN112541182B (en) | Kernel VFS layer system repairing method, device, equipment and storage medium | |
| CN111563008A (en) | Data storage method, device, storage system and computer readable storage medium | |
| CN111538651A (en) | Interface testing method, device, server and storage medium | |
| CN111723229B (en) | Data comparison method, device, computer readable storage medium and electronic equipment | |
| CN112597109B (en) | Data storage method, device, electronic equipment and storage medium | |
| CN115543227B (en) | Cross-system data migration method, system, electronic device and storage medium | |
| CN112084199B (en) | General parameter maintenance method and device based on scene | |
| CN112733210B (en) | Equipment identifier obtaining method and device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |