CN115657969B - Method, device, equipment and medium for acquiring file system difference data - Google Patents
Method, device, equipment and medium for acquiring file system difference data Download PDFInfo
- Publication number
- CN115657969B CN115657969B CN202211660069.8A CN202211660069A CN115657969B CN 115657969 B CN115657969 B CN 115657969B CN 202211660069 A CN202211660069 A CN 202211660069A CN 115657969 B CN115657969 B CN 115657969B
- Authority
- CN
- China
- Prior art keywords
- snapshot
- file system
- disk partition
- file
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000005192 partition Methods 0.000 claims abstract description 199
- 230000004044 response Effects 0.000 claims description 26
- 238000004590 computer program Methods 0.000 claims description 10
- 230000000694 effects Effects 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Abstract
The invention discloses a method, a device, equipment and a medium for acquiring file system difference data, wherein the method comprises the following steps: responding to the formatting of a disk partition file system of the virtual machine, creating a snapshot system file, and starting a reflink module; sequentially traversing each disk partition file system in all disk partition file systems of the virtual machine; triggering a reflink module based on a triggering condition to snapshot the disk partition file system traversed this time; recording the traversed disk partition file system snapshot to a snapshot system file; and acquiring the disk partition file system snapshot traversed this time from the snapshot system file, comparing the information with the disk partition file system snapshot traversed last time, and returning to the step of sequentially traversing each disk partition file system in all the disk partition file systems of the virtual machine after the information comparison is finished. By the scheme of the invention, the speed of acquiring the data difference of the whole file system is greatly improved.
Description
Technical Field
The present invention relates to the field of virtualization technologies, and in particular, to a method, an apparatus, a device, and a medium for obtaining file system differential data.
Background
At present, the calculation virtualization degree of each large enterprise is very high, a plurality of key services run on a virtual machine, the importance of data is self-evident, and therefore a virtualization product must have certain disaster recovery capability. The conventional disaster recovery usually adopts means of data backup and periodic update, that is, data is backed up or updated to another host at night or every fixed period every day, so that if the system is abnormally operated due to external disasters or other reasons, and particularly data is damaged, the latest backup data can be acquired from another host with the backup data stored, and the influence caused by data damage is reduced to the greatest extent.
In a cloud computing and virtualization scene, a traditional data backup scheme mainly backs up data periodically, and before data backup operation is performed, difference information of all files in the period needs to be acquired first, and only the data with the difference needs to be backed up. However, although the traditional data backup scheme performs backup of differential data, in the comparison process of obtaining differential data, especially when a plurality of different file systems exist in a virtual machine and the amount of data in the file systems is large, obtaining differential data of the file systems is a long time-consuming task, which is expressed in that obtaining differential data of the file systems is slow, and the backup performance of system data is affected. Therefore, how to quickly obtain the difference of the file system of the virtual machine and improve the overall response capability of the file system, thereby improving the data backup capability of the file system, is a technical problem that needs to be solved by those skilled in the art.
Disclosure of Invention
In view of this, the present invention provides a method, an apparatus, a computer device, and a computer readable medium for obtaining file system difference data, which can directly compare metadata of a file system of a virtual machine on a file system level, rather than comparing each original data in the file system, thereby greatly increasing a speed of obtaining a data difference of the entire file system, and improving an overall response capability of the file system, and thus improving a data backup capability of the file system, and particularly, in a scenario where multiple file systems coexist in a virtual machine, a data scale is large, and data modification is infrequent, an effect of obtaining the file system difference data is better.
Based on the above object, an aspect of the embodiments of the present invention provides a method for obtaining file system difference data, which specifically includes the following steps:
responding to the formatting of a disk partition file system of the virtual machine, creating a snapshot system file, and starting a reflink module;
sequentially traversing each disk partition file system in all disk partition file systems of the virtual machine;
triggering the reflink module based on a triggering condition to snapshot the traversed disk partition file system;
recording the traversed disk partition file system snapshot to the snapshot system file;
and acquiring the disk partition file system snapshot traversed this time from the snapshot system file, comparing the information with the disk partition file system snapshot traversed last time, and returning to the step of sequentially traversing each disk partition file system in all the disk partition file systems of the virtual machine after the information comparison is finished.
In some embodiments, prior to the creating a snapshot system file in response to the virtual machine disk partition file system formatting, the method further comprises:
constructing a file system snapshot module;
creating a snapshot system file in response to the virtual machine disk partition file system formatting comprises:
and responding to the formatting of the disk partition file system of the virtual machine, and starting the snapshot module of the file system to create a snapshot system file.
In some embodiments, creating the snapshot system file and enabling the reflink module comprises:
creating a snapshot system file, starting a reflink module, and configuring a disk partition file system snapshot threshold.
In some embodiments, triggering the reflink module based on a trigger condition to snapshot the disk partition file system traversed this time includes:
judging whether the traversed disk partition file system receives a file system snapshot command issued by an upper application or not;
and responding to the received file system snapshot command, and triggering the reflink module to snapshot the disk partition file system traversed this time based on the file system snapshot command.
In some embodiments, the method further comprises:
and in response to the fact that the file system snapshot command is not received, triggering the reflink module to snapshot the disk partition file system traversed this time based on the file system snapshot threshold.
In some embodiments, the snapshotting the disk partition file system traversed this time based on the file system snapshooting threshold includes:
judging whether the time of the traversed disk partition file system reaches the file system snapshot threshold value or not;
and responding to the time reaching the file system snapshot threshold value, and carrying out snapshot on the disk partition file system traversed this time.
In some embodiments, the method further comprises:
and responding to the condition that the time does not reach the snapshot threshold of the file system, and returning to the step of sequentially traversing each disk partition file system in all the disk partition file systems of the virtual machine.
In some embodiments, the snapshotting the disk partition file system traversed this time based on the file system snapshooting threshold includes:
judging whether the input and output data volume of the traversed disk partition file system reaches the file system snapshot threshold value or not;
and responding to the fact that the input and output data volume reaches the file system snapshot threshold value, and carrying out snapshot on the disk partition file system traversed this time.
In some embodiments, the method further comprises:
and responding to the situation that the input and output data volume does not reach the file system snapshot threshold value, and returning to the step of sequentially traversing each disk partition file system in all the disk partition file systems of the virtual machine.
In some embodiments, the file system snapshot threshold comprises any one of the following thresholds: a time threshold and an input-output data volume threshold.
In some embodiments, the file system snapshot module is configured to create a snapshot system file in a system file of a disk partition file system.
In some embodiments, the snapshot system file is configured to store source system file data information of the disk partition file system.
In some embodiments, creating a snapshot system file and enabling a reflink module, and configuring a disk partition file system snapshot threshold comprises:
and creating a snapshot system file based on the formatting parameters, starting a reflink module, and configuring a disk partition file system snapshot threshold.
In some embodiments, obtaining the disk partition file system snapshot traversed this time from the snapshot system file, and comparing the information with the disk partition file system snapshot traversed last time includes:
and acquiring the disk partition file system snapshot traversed this time from the snapshot system file, and comparing the data information of the source system file with the disk partition file system snapshot traversed last time.
In another aspect of the embodiments of the present invention, a device for obtaining file system difference data is further provided, including:
the formatting module is configured to respond to the formatting of a disk partition file system of the virtual machine, create a snapshot system file and start the reflink module;
the traversing module is configured to sequentially traverse each disk partition file system in all the disk partition file systems of the virtual machine;
the snapshot module is configured to trigger the reflink module based on a trigger condition so as to snapshot the disk partition file system traversed this time;
the snapshot module is also configured to record the disk partition file system snapshot traversed this time to the snapshot system file;
and the snapshot module is also configured to acquire the disk partition file system snapshot traversed this time from the snapshot system file, compare the information with the disk partition file system snapshot traversed last time, and return to the step of sequentially traversing each disk partition file system in all the disk partition file systems of the virtual machine after the information comparison is completed.
In some embodiments, the apparatus further comprises:
a build module configured to build a file system snapshot module;
the formatting module is specifically configured to open the file system snapshot module to create a snapshot system file in response to a virtual machine disk partition file system formatting.
In some embodiments, the formatting module is specifically configured to create a snapshot system file, enable the reflink module, and configure a disk partition file system snapshot threshold.
In some embodiments, the snapshot module is configured to:
judging whether the traversed disk partition file system receives a file system snapshot command issued by an upper application or not;
in response to receiving the file system snapshot command, triggering the reflink module to snapshot the disk partition file system traversed this time based on the file system snapshot command;
and in response to not receiving the file system snapshot command, triggering the reflink module to snapshot the disk partition file system traversed this time based on the file system snapshot threshold.
In another aspect of the embodiments of the present invention, there is also provided a computer device, including: at least one processor; and a memory storing a computer program executable on the processor, the computer program when executed by the processor implementing the steps of the method as above.
In a further aspect of the embodiments of the present invention, a computer-readable storage medium is also provided, in which a computer program for implementing the above method steps is stored when the computer program is executed by a processor.
The invention has at least the following beneficial technical effects: by the scheme of the invention, the metadata of the file system of the virtual machine can be directly compared on the file system level instead of comparing each original data in the file system, so that the speed of acquiring the data difference of the whole file system is greatly increased, the integral response capability of the file system is improved, and the data backup capability of the file system is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
FIG. 1 is a flowchart of an embodiment of a method for obtaining file system difference data according to the present invention;
FIG. 2 is a flowchart illustrating a method for obtaining file system difference data according to another embodiment of the present invention;
FIG. 3 is a diagram illustrating an embodiment of adding snapshot system files provided by the present invention;
FIG. 4 is a diagram illustrating an embodiment of an apparatus for obtaining file system difference data according to the present invention;
FIG. 5 is a schematic structural diagram of an embodiment of a computer device provided in the present invention;
fig. 6 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two entities with the same name but different names or different parameters, and it should be noted that "first" and "second" are merely for convenience of description and should not be construed as limitations of the embodiments of the present invention, and they are not described in any more detail in the following embodiments.
In view of the above, a first aspect of the embodiments of the present invention proposes an embodiment of a method for obtaining file system difference data. As shown in fig. 1, it includes the following steps:
s10, responding to the formatting of a disk partition file system of the virtual machine, creating a snapshot system file, and starting a reflink module;
s20, traversing each disk partition file system in all the disk partition file systems of the virtual machine in sequence;
s30, triggering the reflink module based on a trigger condition to snapshot the traversed disk partition file system;
s40, recording the disk partition file system snapshot traversed this time to the snapshot system file;
and S50, acquiring the disk partition file system snapshot traversed this time from the snapshot system file, comparing the information with the disk partition file system snapshot traversed last time, and returning to the step of sequentially traversing each disk partition file system in all the disk partition file systems of the virtual machine after the information comparison is completed.
Specifically, when a disk partition file system of the virtual machine is formatted, a snapshot system file snap is added in a system file of the file system, a file system reflink module is started, and a file system snapshot threshold value is configured; traversing all disk partition file systems of the virtual machine; the method comprises the steps that a reflink module is triggered, a traversed disk partition file system is subjected to snapshot based on the reflink module after triggering, and two modes of external triggering file system snapshot characteristics and periodical automatic triggering file system snapshot characteristics are supported in the embodiment; recording a file system reflink snapshot to a snapshot system file snap; and acquiring and comparing previous and next file system reflink snapshots from the snapshot system snap so as to acquire the data difference information of the previous and next file systems.
It should be noted that reflink refers to a file flashcopy technology.
By the embodiment of the invention, the metadata of the file system of the virtual machine can be directly compared on the file system level, the speed of acquiring the data difference of the whole file system is greatly improved, and the integral response capability of the file system is improved, so that the data backup capability of the file system is improved, and the effect of acquiring the difference data of the file system is better especially under the scenes that multiple file systems coexist in the virtual machine, the data scale is large, and the data modification is not frequent.
In some embodiments, prior to the creating a snapshot system file in response to the virtual machine disk partition file system formatting, the method further comprises:
constructing a file system snapshot module;
creating a snapshot system file in response to the virtual machine disk partition file system formatting comprises:
and responding to the formatting of the disk partition file system of the virtual machine, and starting the snapshot module of the file system to create a snapshot system file.
In a specific embodiment, a file system snap module is constructed, namely the file system snap module represents a file system snapshot module, when the disk partition file system is formatted, the file system snap module is started, and a snap system file snap is added through the file system snap module and is used for storing reflink snapshots.
In some embodiments, creating the snapshot system file and enabling the reflink module comprises:
creating a snapshot system file, starting a reflink module, and configuring a disk partition file system snapshot threshold.
In one embodiment, when the file system of the disk partition is formatted, the snap module of the file system may be enabled by a formatting parameter, where the formatting parameter includes: when the module is started, a snapshot system file snap is newly added on the basis of the original system file of the formatted disk partition file system, and one or more blocks (data blocks) are occupied according to the maximum number of storable file system reflink in the formatting parameters, so that the module is used for reflink operation of the system file, storing superblock (superblock), allocator (classifier) and other key element information, wherein the key element information refers to source system file data information of the disk partition file system. When a plurality of disk partitions exist in the virtual machine, each disk partition opens a snap module of the file system.
When the disk partition file system is formatted, the file system reflink module can be started through the formatting parameters, and the file system reflink module can realize the basis of quickly acquiring the file system difference data.
When the disk partition file system is formatted, the file system snapshot threshold value can be configured through the formatting parameters, and the file system snapshot threshold value can include two configuration modes of time and data volume. For example, when the snapshot threshold of the configuration file system is 1min, a reflink operation is automatically performed on the disk partition file system every 1 min. When the snapshot threshold of the configured file system is 100MB, the file system automatically performs a reflink operation on the file system once when processing 100MB of IO (Input Output) data. When a plurality of disk partitions exist, different file system snapshot thresholds are allowed to be used by each disk partition, and configuration can be carried out according to actual service conditions.
In some embodiments, triggering the reflink module based on a trigger condition to snapshot the disk partition file system traversed this time includes:
judging whether the traversed disk partition file system receives a file system snapshot command issued by an upper application or not;
and in response to receiving the file system snapshot command, triggering the reflink module to snapshot the disk partition file system traversed this time based on the file system snapshot command.
In some embodiments, the method further comprises:
and in response to not receiving the file system snapshot command, triggering the reflink module to snapshot the disk partition file system traversed this time based on the file system snapshot threshold.
In some embodiments, the snapshot of the file system of the disk partition traversed this time based on the file system snapshot threshold includes:
judging whether the time of the traversed disk partition file system reaches the file system snapshot threshold or not;
and responding to the time reaching the file system snapshot threshold value, and carrying out snapshot on the disk partition file system traversed this time.
In some embodiments, the method further comprises:
and responding to the condition that the time does not reach the snapshot threshold of the file system, and returning to the step of sequentially traversing each disk partition file system in all the disk partition file systems of the virtual machine.
In some embodiments, the snapshot of the file system of the disk partition traversed this time based on the file system snapshot threshold includes:
judging whether the input and output data volume of the traversed disk partition file system reaches the file system snapshot threshold value or not;
and responding to the fact that the input and output data volume reaches the file system snapshot threshold value, and carrying out snapshot on the disk partition file system traversed this time.
In some embodiments, the method further comprises:
and responding to the situation that the input and output data volume does not reach the file system snapshot threshold value, and returning to the step of sequentially traversing each disk partition file system in all the disk partition file systems of the virtual machine.
In a specific embodiment, since a plurality of disks can be installed on the virtual machine, each disk can be divided into a plurality of partitions, and each separate partition can use a file system, when traversing file systems of all disk partitions of the virtual machine, the file systems of the disk partitions are used as granularity for traversing, and the following processing is sequentially performed:
judging whether an upper application has an instruction for issuing a file system snapshot, and if the disk partition file system receives the instruction, immediately performing reflink snapshot operation on key data information of the disk partition file system; if the disk partition file system does not receive the instruction, judging whether the time of last snapshot of the disk partition file system or the IO data volume reaches a snapshot threshold value, if so, immediately performing reflink operation on key metadata such as superblock, allocator and the like of the disk partition file system, storing the reflink operation to a snapshot system file snap, and if not, traversing the next disk partition file system.
And obtaining and comparing the reflink snapshots of the file system before and after so as to obtain the data difference information of the file system. When the file system data difference information needs to be acquired, taking out the specified twice reflink snapshot records from the snapshot system file snap, generally comparing the key element information in the traversed reflink snapshot record of this time with the reflink snapshot record of the last time, thereby acquiring the file system data difference information quickly.
In a specific embodiment, since the number of file system reflink snapshots that can be stored by the system file snap is limited when the disk partition file system is formatted, in order to prevent data overflow, the earliest reflink snapshot record can be overwritten in an automatic overwriting manner.
In some embodiments, the file system snapshot threshold comprises any one of the following thresholds: a time threshold and an input-output data amount threshold.
In some embodiments, the file system snapshot module is configured to create a snapshot system file in a system file of a disk partition file system.
In some embodiments, the snapshot system file is configured to store source system file data information of the disk partition file system.
In some embodiments, creating a snapshot system file and enabling a reflink module, and configuring a disk partition file system snapshot threshold comprises:
and creating a snapshot system file based on the formatting parameters, starting a reflink module, and configuring a snapshot threshold of the disk partition file system.
In some embodiments, obtaining the disk partition file system snapshot traversed this time from the snapshot system file, and comparing the information with the disk partition file system snapshot traversed last time includes:
and acquiring the disk partition file system snapshot traversed this time from the snapshot system file, and comparing the data information of the source system file with the disk partition file system snapshot traversed last time.
In one embodiment, as shown in FIG. 2, a method for obtaining file system difference data is disclosed.
In this embodiment, the virtual machine has 2 disk dista and disk b, where the disk dista is divided into 3 partitions, each partition uses ext4 (Fourth generation extended file system), and the disk distb is divided into 4 partitions, each partition uses ocfs2 (a file system).
When the disk partition file system is formatted, a file system snap module is started, and a snapshot system file snap is added based on the file system snap module. Specifically, when a disk partition file system is formatted, a formatting parameter of 'fs _ features fs-snap' is added, a file system snap module is started, when the file system snap module is started, a snap system file snap is added on the basis of an original system file of the file system, and one or more blocks are occupied according to the formatting parameter 'the maximum storable file system snap number', so that key element information such as reflink snapshot operation, superblock, allocate and the like can be stored for the system file. When multiple disks or partitions are present, each disk or partition requires the module to be turned on.
As shown in fig. 3, a disk diskB is a physical hard disk that is started by using an MBR (Master Boot Record), a partition table is used to accommodate disk partition information, the disk diskB includes four disk partitions, which are part0 to part3 respectively, and a part1 partition is taken as an example to describe a location where a newly added snapshot system file snap is located. In an ocfs2 file system, except for a leading block of a disk partition part1, the rest of block blocks are divided into a plurality of cluster groups for management, the cluster groups are respectively cluster group 0-cluster group n, and source system file data information such as super blocks (super _ blocks) and system files is uniformly stored in the cluster group 0. In cluster group 0, super _ block (superblock) is the first; then cluster group description information (group _ desc) and a root directory (root _ dir) are carried out; then, system files are generated, wherein the system files comprise bad block statistics (bad _ blocks), global index node allocation (global _ inode), slot bitmaps (slot _ maps), heartbeats (heartbeat) and snapshots (snap), and when a snapshot module of a file system is started, a snapshot system file is newly added and used for performing reflink snapshot operation on other system files and storing data information of source system files such as superblocks and allocators; and the data area is used for storing data blocks (data _ blocks).
And when the file system is formatted, starting a file system reflink module. Specifically, a file system reflink module is enabled through a formatting parameter '-fs _ features reflink', and the file system reflink module is a basis for realizing quick acquisition of file system difference data.
A default file system snapshot threshold is configured. Specifically, a snapshot threshold of the file system is configured through a formatting parameter, and two configuration modes of time and data volume are supported. In the embodiment, the time is selected as the default snapshot threshold of the file system, and during formatting, the reflink snapshot operation is automatically performed on the file system every 1 hour through the formatting parameter "— default _ snap _ threshold 3600 s". All disk partitions of the virtual machine may use the same parameters.
And traversing all disk partition file systems of the virtual machine. Specifically, 3 partitions of the disk A and 4 partitions of the disk B on the virtual machine are polled in sequence, and reflink operation is performed on each partition file system according to a strategy.
The strategy supports two modes of externally triggering the snapshot characteristic of the file system and periodically and automatically triggering the snapshot characteristic of the file system. Specifically, whether an instruction for issuing a file system snapshot exists in the upper layer application is judged, and if the file system receives the instruction, reflink snapshot operation is immediately performed on the file data information of a source system of the file system; if the file system does not receive the instruction, judging whether the time or the IO data volume reaches a snapshot threshold value, if so, immediately performing reflink snapshot operation on file data information of a source system of the file system, and if not, retrieving the next disk or partition file system.
And recording the snapshot of the file system reflink to the snapshot system file snap. Specifically, reflink is performed on key element information such as superblock, allocator and the like of the file system, and the reflink is stored in a system file snap.
And acquiring and comparing previous and subsequent file system reflink snapshots from a snapshot system snap to acquire file system data difference information. Specifically, when the file system data difference information needs to be acquired, the designated two reflink snapshot records are taken out from the snapshot system file snap, and the file system data difference information is acquired quickly by comparing the source system file data information of the file system in the reflink snapshot records.
By the embodiment of the invention, the metadata of the file system of the virtual machine can be directly compared on the file system level, the speed of acquiring the data difference of the whole file system is greatly improved, and the integral response capability of the file system is improved, so that the data backup capability of the file system is improved, and the effect of acquiring the difference data of the file system is better especially under the scenes that multiple file systems coexist in the virtual machine, the data scale is large, and the data modification is not frequent.
Based on the same inventive concept, according to another aspect of the present invention, as shown in fig. 4, an embodiment of the present invention further provides an apparatus for obtaining file system difference data, including:
a formatting module 110, said formatting module 110 configured to create a snapshot system file and enable a reflink module in response to a virtual machine disk partition file system formatting;
a traversing module 120, wherein the traversing module 120 is configured to sequentially traverse each disk partition file system of all disk partition file systems of the virtual machine;
a snapshot module 130, where the snapshot module 130 is configured to trigger the reflink module based on a trigger condition to snapshot the disk partition file system traversed this time;
the snapshot module 130 is further configured to record the disk partition file system snapshot traversed this time to the snapshot system file;
the snapshot module 130 is further configured to obtain the disk partition file system snapshot traversed this time from the snapshot system file, compare the information with the disk partition file system snapshot traversed last time, and return to the step of sequentially traversing each disk partition file system in all the disk partition file systems of the virtual machine after the information comparison is completed.
In some embodiments, the apparatus further comprises:
a build module configured to build a file system snapshot module;
the formatting module is specifically configured to open the file system snapshot module to create a snapshot system file in response to a virtual machine disk partition file system formatting.
In some embodiments, the formatting module is specifically configured to create a snapshot system file, enable the reflink module, and configure a disk partition file system snapshot threshold.
In some embodiments, the snapshot module is configured to:
judging whether the traversed disk partition file system receives a file system snapshot command issued by an upper application or not;
in response to receiving the file system snapshot command, triggering the reflink module to snapshot the disk partition file system traversed this time based on the file system snapshot command;
and in response to not receiving the file system snapshot command, triggering the reflink module to snapshot the disk partition file system traversed this time based on the file system snapshot threshold.
Based on the same inventive concept, according to another aspect of the present invention, as shown in fig. 5, the embodiment of the present invention further provides a computer device 30, in which the computer device 30 comprises a processor 310 and a memory 320, the memory 320 stores a computer program 321 that can run on the processor, and the processor 310 executes the program to perform the steps of the above method.
The memory, as a non-volatile computer-readable storage medium, may be used to store a non-volatile software program, a non-volatile computer-executable program, and modules, such as program instructions/modules corresponding to the method for acquiring file system difference data in the embodiments of the present application. The processor executes various functional applications and data processing of the device by running the nonvolatile software programs, instructions and modules stored in the memory, namely, the method for acquiring file system difference data of the above method embodiment is realized.
The memory may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the device, and the like. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, the memory optionally includes memory located remotely from the processor, and such remote memory may be coupled to the local module via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Based on the same inventive concept, according to another aspect of the present invention, as shown in fig. 6, an embodiment of the present invention further provides a computer-readable storage medium 40, where the computer-readable storage medium 40 stores a computer program 410, which when executed by a processor, performs the above method.
Finally, it should be noted that, as will be understood by those skilled in the art, all or part of the processes of the methods of the above embodiments may be implemented by a computer program, which may be stored in a computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. The storage medium of the program may be a magnetic disk, an optical disk, a Read Only Memory (ROM), a Random Access Memory (RAM), or the like. The embodiments of the computer program may achieve the same or similar effects as any of the above-described method embodiments.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant only to be exemplary, and is not intended to intimate that the scope of the disclosure, including the claims, is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.
Claims (20)
1. A method of obtaining file system difference data, comprising:
responding to the formatting of a disk partition file system of the virtual machine, creating a snapshot system file in a system file of the file system, and starting a reflink module;
sequentially traversing each disk partition file system in all disk partition file systems of the virtual machine;
triggering the reflink module based on a triggering condition to snapshot the disk partition file system traversed this time;
recording the traversed disk partition file system snapshot to the snapshot system file;
and acquiring the disk partition file system snapshot traversed this time from the snapshot system file, comparing the information with the disk partition file system snapshot traversed last time, and returning to the step of traversing each disk partition file system in all the disk partition file systems of the virtual machine in sequence after the information comparison is finished.
2. The method of claim 1, further comprising, prior to the step of creating a snapshot system file in response to the virtual machine disk partition file system formatting:
constructing a file system snapshot module;
creating a snapshot system file in response to the virtual machine disk partition file system formatting comprises:
and responding to the file system formatting of the disk partition of the virtual machine, and starting the file system snapshot module to create a snapshot system file.
3. The method of claim 1, wherein creating a snapshot system file and enabling a reflink module comprises:
creating a snapshot system file, starting a reflink module, and configuring a disk partition file system snapshot threshold.
4. The method of claim 3, wherein triggering the reflink module to snapshot the disk partition file system traversed this time based on a triggering condition comprises:
judging whether the traversed disk partition file system receives a file system snapshot command issued by an upper application or not;
and in response to receiving the file system snapshot command, triggering the reflink module to snapshot the disk partition file system traversed this time based on the file system snapshot command.
5. The method of claim 4, further comprising:
and in response to not receiving the file system snapshot command, triggering the reflink module to snapshot the disk partition file system traversed this time based on the file system snapshot threshold.
6. The method of claim 5, wherein snapshotting the disk partition file system traversed this time based on the file system snapshot threshold comprises:
judging whether the time of the traversed disk partition file system reaches the file system snapshot threshold value or not;
and responding to the time reaching the file system snapshot threshold value, and carrying out snapshot on the disk partition file system traversed this time.
7. The method of claim 6, further comprising:
and responding to the condition that the time does not reach the snapshot threshold of the file system, and returning to the step of sequentially traversing each disk partition file system in all the disk partition file systems of the virtual machine.
8. The method of claim 5, wherein snapshotting the disk partition file system traversed this time based on the file system snapshot threshold comprises:
judging whether the input and output data volume of the traversed disk partition file system reaches the file system snapshot threshold value or not;
and responding to the fact that the input and output data volume reaches the file system snapshot threshold value, and carrying out snapshot on the disk partition file system traversed this time.
9. The method of claim 8, further comprising:
and responding to the situation that the input and output data volume does not reach the file system snapshot threshold value, and returning to the step of sequentially traversing each disk partition file system in all the disk partition file systems of the virtual machine.
10. The method of claim 3, wherein the file system snapshot threshold comprises any one of the following thresholds: a time threshold and an input-output data volume threshold.
11. The method of claim 2, wherein the file system snapshot module is configured to create a snapshot system file in a system file of a disk partitioned file system.
12. The method of claim 11, wherein the snapshot system file is configured to store source system file data information of the disk partition file system.
13. The method of claim 3, wherein creating a snapshot system file and enabling a reflink module and configuring disk partition file system snapshot thresholds comprises:
and creating a snapshot system file based on the formatting parameters, starting a reflink module, and configuring a snapshot threshold of the disk partition file system.
14. The method of claim 1, wherein obtaining the snapshot of the disk partition file system traversed this time from the snapshot system file, and comparing the information with the snapshot of the disk partition file system traversed last time comprises:
and acquiring the disk partition file system snapshot traversed this time from the snapshot system file, and comparing the data information of the source system file with the disk partition file system snapshot traversed last time.
15. An apparatus for obtaining file system difference data, comprising:
the formatting module is configured to respond to the formatting of a disk partition file system of the virtual machine, create a snapshot system file in a system file of the file system, and start the reflink module;
the traversing module is configured to sequentially traverse each disk partition file system in all the disk partition file systems of the virtual machine;
the snapshot module is configured to trigger the reflink module based on a trigger condition so as to snapshot the disk partition file system traversed this time;
the snapshot module is also configured to record the disk partition file system snapshot traversed this time to the snapshot system file;
and the snapshot module is also configured to acquire the disk partition file system snapshot traversed this time from the snapshot system file, compare the information with the disk partition file system snapshot traversed last time, and return to the step of sequentially traversing each disk partition file system in all the disk partition file systems of the virtual machine after the information comparison is completed.
16. The apparatus of claim 15, further comprising:
a build module configured to build a file system snapshot module;
the formatting module is specifically configured to open the file system snapshot module to create a snapshot system file in response to a virtual machine disk partition file system formatting.
17. The apparatus of claim 15, wherein the formatting module is specifically configured to create a snapshot system file and enable a reflink module and configure a disk partition file system snapshot threshold.
18. The apparatus of claim 17, wherein the snapshot module is configured to:
judging whether the traversed disk partition file system receives a file system snapshot command issued by an upper application or not;
in response to receiving the file system snapshot command, triggering the reflink module to snapshot the disk partition file system traversed this time based on the file system snapshot command;
and in response to not receiving the file system snapshot command, triggering the reflink module to snapshot the disk partition file system traversed this time based on the file system snapshot threshold.
19. A computer device, comprising:
at least one processor; and
memory storing a computer program operable on the processor, wherein the processor executes the program to perform the steps of the method according to any of claims 1 to 14.
20. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 14.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211660069.8A CN115657969B (en) | 2022-12-23 | 2022-12-23 | Method, device, equipment and medium for acquiring file system difference data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211660069.8A CN115657969B (en) | 2022-12-23 | 2022-12-23 | Method, device, equipment and medium for acquiring file system difference data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115657969A CN115657969A (en) | 2023-01-31 |
| CN115657969B true CN115657969B (en) | 2023-03-10 |
Family
ID=85023539
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211660069.8A Active CN115657969B (en) | 2022-12-23 | 2022-12-23 | Method, device, equipment and medium for acquiring file system difference data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115657969B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112181589A (en) * | 2020-09-25 | 2021-01-05 | 苏州浪潮智能科技有限公司 | Optimization method for realizing post-snapshot migration of KVM (Kernel-based virtual machine) based on reflink |
| CN115185891A (en) * | 2022-09-14 | 2022-10-14 | 联想凌拓科技有限公司 | Data management method and device of file system, electronic equipment and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7567991B2 (en) * | 2003-06-25 | 2009-07-28 | Emc Corporation | Replication of snapshot using a file system copy differential |
| US10534759B1 (en) * | 2018-08-23 | 2020-01-14 | Cohesity, Inc. | Incremental virtual machine metadata extraction |
| CN113282538A (en) * | 2021-07-06 | 2021-08-20 | 中国工商银行股份有限公司 | File system management method, device, equipment, storage medium and program product |
-
2022
- 2022-12-23 CN CN202211660069.8A patent/CN115657969B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112181589A (en) * | 2020-09-25 | 2021-01-05 | 苏州浪潮智能科技有限公司 | Optimization method for realizing post-snapshot migration of KVM (Kernel-based virtual machine) based on reflink |
| CN115185891A (en) * | 2022-09-14 | 2022-10-14 | 联想凌拓科技有限公司 | Data management method and device of file system, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115657969A (en) | 2023-01-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11741048B2 (en) | Distributed write journals that support fast snapshotting for a distributed file system | |
| US20220188003A1 (en) | Distributed Storage Method and Device | |
| US11249940B2 (en) | Snapshot archive management | |
| US10628378B2 (en) | Replication of snapshots and clones | |
| US7257690B1 (en) | Log-structured temporal shadow store | |
| US10872017B2 (en) | Restoring a file system object | |
| CN107451013B (en) | Data recovery method, device and system based on distributed system | |
| JP2007179551A (en) | Method and apparatus for backup and recovery using storage based journaling | |
| WO2019020081A1 (en) | Distributed system and fault recovery method and apparatus thereof, product, and storage medium | |
| WO2017050064A1 (en) | Memory management method and device for shared memory database | |
| CN113377292B (en) | Single machine storage engine | |
| CN111033471A (en) | Method, system, and medium for controlling appending only files | |
| WO2014190143A1 (en) | Efficient storage of small random changes to data on disk | |
| JP7215971B2 (en) | METHOD AND APPARATUS FOR PROCESSING DATA LOCATION IN STORAGE DEVICE, COMPUTER DEVICE AND COMPUTER-READABLE STORAGE MEDIUM | |
| CN106709014B (en) | File system conversion method and device | |
| CN109753381B (en) | Continuous data protection method based on object storage | |
| CN113885809A (en) | Data management system and method | |
| CN115657969B (en) | Method, device, equipment and medium for acquiring file system difference data | |
| CN113419897A (en) | File processing method and device, electronic equipment and storage medium thereof | |
| JP2023531751A (en) | Vehicle data storage method and system | |
| CN109960602B (en) | Information management method, device, equipment and medium | |
| CN114153392A (en) | Object storage data storage management method, device and equipment | |
| CN113590380A (en) | Database recovery method and system | |
| CN115840662A (en) | Data backup system and device | |
| CN114258532A (en) | Apparatus and method for merging backup policies |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |