CN116401096A - Metadata backup method and device, electronic equipment and storage medium - Google Patents
Metadata backup method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116401096A CN116401096A CN202310377190.8A CN202310377190A CN116401096A CN 116401096 A CN116401096 A CN 116401096A CN 202310377190 A CN202310377190 A CN 202310377190A CN 116401096 A CN116401096 A CN 116401096A
- Authority
- CN
- China
- Prior art keywords
- metadata
- file
- storage
- storage file
- backup
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000013524 data verification Methods 0.000 claims abstract description 19
- 230000015654 memory Effects 0.000 claims description 34
- 238000004590 computer program Methods 0.000 claims description 13
- 238000013500 data storage Methods 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000006978 adaptation Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 4
- 238000012217 deletion Methods 0.000 description 3
- 230000037430 deletion Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000011084 recovery Methods 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 230000003442 weekly effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
Abstract
The application provides a metadata backup method, a metadata backup device, electronic equipment and a storage medium, and relates to the technical field of data storage, wherein the metadata backup method comprises the following steps: under the condition that a preset operation for a storage file is received, acquiring metadata of the storage file and backing up the metadata before responding to the preset operation; wherein the preset operation includes any one of the following: after the data writing operation is carried out on the storage file, closing operation is carried out on the storage file after the writing operation; deleting the storage file; and carrying out data verification on the storage file. In this embodiment, under the condition that a preset operation for storing a file is received, the operation is intercepted, metadata is acquired and backed up, and the IO number of times of a disk is not additionally increased, so that the cost of metadata backup is reduced, and the availability of backup service is improved.
Description
Technical Field
The present application relates to the field of data storage technologies, and in particular, to a metadata backup method, a metadata backup device, an electronic device, and a storage medium.
Background
For storage services, it is necessary to ensure that the stored data is not lost. In actual data storage, the need of retrieving after deleting data for various reasons, such as file deletion caused by software failure of a storage system, file deletion caused by operation and maintenance operation or file system damage, file deletion caused by misoperation of a user and the like, is faced. To ensure that data is not lost, the storage system needs to recover these erroneously deleted files.
For solid state disks (Solid State Drive, SSD), the integrity of the stored data is ensured by way of a full disk scan. However, for mechanical Hard disks (HDD) with larger and larger storage space and less variation in input/output IO capability, the time of full Disk scan needs to be calculated on a weekly basis, which seriously affects the timeliness of storage service restoration. In order to accelerate the data recovery speed, metadata of files stored on the HDD needs to be backed up, however, a metadata backup scheme is generally adopted, files stored on the HDD need to be read, more disk IO capability is consumed, and the performance of backup service is affected.
Disclosure of Invention
The embodiment of the application provides a metadata backup method, a metadata backup device, electronic equipment and a storage medium, which avoid consuming more disk IO capacity and improve the performance of backup service.
In a first aspect, an embodiment of the present application provides a metadata backup method, where the method includes:
under the condition that a preset operation for a storage file is received, acquiring metadata of the storage file and backing up the metadata before responding to the preset operation;
wherein the preset operation includes any one of the following:
after the data writing operation is carried out on the storage file, closing operation is carried out on the storage file after the writing operation;
deleting the storage file;
and carrying out data verification on the storage file.
In a second aspect, an embodiment of the present application provides a metadata backup apparatus, including:
the acquisition module is used for acquiring metadata of the storage file before responding to the preset operation under the condition that the preset operation aiming at the storage file is received;
the backup module is used for backing up the metadata;
wherein the preset operation includes any one of the following:
after the data writing operation is carried out on the storage file, closing operation is carried out on the storage file after the writing operation;
deleting the storage file;
and carrying out data verification on the storage file.
In a third aspect, embodiments of the present application provide an electronic device including a memory, a processor, and a computer program stored on the memory, the processor implementing the method of any one of the above when the computer program is executed.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium having a computer program stored therein, the computer program, when executed by a processor, implementing a method according to any one of the above.
Compared with the prior art, the application has the following advantages:
the application provides a metadata backup method, a metadata backup device, electronic equipment and a storage medium, wherein under the condition that a preset operation for a storage file is received, metadata of the storage file is acquired and backed up before the preset operation is responded; wherein the preset operation includes any one of the following: after the data writing operation is carried out on the storage file, closing operation is carried out on the storage file after the writing operation; deleting the storage file; and carrying out data verification on the storage file. In this embodiment, under the condition that a preset operation for a storage file is received, intercepting the operation, acquiring metadata and backing up the metadata, wherein the storage file after a write operation is closed, and the storage file is subjected to data verification, before the operations, the metadata is also in a kernel memory, and at this time, the metadata is acquired, so that the IO capability of a disk is not consumed; for the operation of deleting the storage file, the deleting operation is also used for loading metadata, so that the metadata acquired before deleting the storage file belongs to the preheating operation, the IO frequency of a disk is not additionally increased, the expenditure of metadata backup is reduced, and the availability of backup service is improved.
The foregoing description is merely an overview of the technical solutions of the present application, and in order to make the technical means of the present application more clearly understood, it is possible to implement the present application according to the content of the present specification, and in order to make the above and other objects, features and advantages of the present application more clearly understood, the following detailed description of the present application will be given.
Drawings
In the drawings, the same reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily drawn to scale. It is appreciated that these drawings depict only some embodiments according to the application and are not to be considered limiting of its scope.
Fig. 1 is a schematic diagram of an application scenario provided in the present application;
FIG. 2 is a flowchart of a metadata backup method according to an embodiment of the present application;
FIG. 3 is a block diagram illustrating a metadata backup apparatus according to an embodiment of the present application; and
fig. 4 is a block diagram of an electronic device used to implement an embodiment of the present application.
Detailed Description
Hereinafter, only certain exemplary embodiments are briefly described. As will be recognized by those of skill in the pertinent art, the described embodiments may be modified in various different ways without departing from the spirit or scope of the present application. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.
In order to facilitate understanding of the technical solutions of the embodiments of the present application, the following describes related technologies of the embodiments of the present application. The following related technologies may be optionally combined with the technical solutions of the embodiments of the present application, which all belong to the protection scope of the embodiments of the present application.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region, and provide corresponding operation entries for the user to select authorization or rejection.
Fig. 1 is a schematic diagram of an application scenario of the technical scheme of the present application. As shown in fig. 1, in this embodiment, the backup of metadata is completed through two channels of the data service and the metadata service. In the storage service, the data service is a service responsible for writing data, which is a base of the storage system, and user data is written into the HDD through the service.
After a data service channel performs a data writing operation on a storage file, before performing a closing operation on the storage file after the writing operation (before closing the writing and opening the file as shown in fig. 1), before deleting the storage file (before deleting the file as shown in fig. 1) and before performing a data verification on the storage file (before data verification as shown in fig. 1), intercepting the operations, acquiring metadata by using a metadata backup engine through a standard application programming interface provided by an operating system, and persisting the acquired metadata to a metadata backup warehouse. That is, when corresponding operation tasks are performed, metadata backup of a single task is performed, data is not required to be read through full-disk scanning, files in a disk are not required to be read, closing operation is performed on storage files after writing operation, and data verification is performed on the storage files, before the operations, metadata is still in a memory, at the moment, the metadata is acquired, and IO capacity of the disk is not consumed; for the operation of deleting the storage file, metadata is loaded in the deleting operation, so that the metadata acquired before deleting the storage file belongs to the preheating operation, and the IO frequency of the disk is not additionally increased.
In actual operation, the data service may suddenly fail or abnormally exit at a certain moment, so that metadata of a part of files are not normally backed up, and in this case, a metadata service channel is required to perform leak detection and defect repair on lost files. The method comprises the steps of periodically obtaining an effective file list and metadata of effective files on a disk, loading backed-up metadata obtained from a metadata backup warehouse, comparing the metadata of the effective files with backed-up metadata, wherein the metadata comprise file names, creation time and index node identification file lengths, respectively comparing to obtain an incremental file list, obtaining metadata of files in the incremental file list, if the metadata of the files in the incremental file list are metadata without backup, obtaining metadata of the files in the incremental file list, and backing up the metadata of the files in the incremental file list into the metadata backup warehouse.
In the related art, data loss or errors are avoided by scanning a file or backing up a metadata area and a Journal (Journal) area of a file system, and the two processing modes inevitably require reading data on the HDD, so that disk IO capacity is consumed, and further service performance is affected.
In another related art, in order to reduce disk IO operations, metadata backup is accomplished through an enhanced kernel. For example, through a kernel module and other schemes, the kernel function is enhanced, and after the metadata of the file is found to be modified in the kernel, the metadata of the modified file is persisted, so that backup is completed. However, this solution requires modification in kernel mode, so the development requirement is very high, and faults are likely to be introduced, resulting in system crashes. In addition, as the kernel version is changed and iterated, uncontrollable adaptation risks are introduced. Therefore, the adaptation needs to be performed on the cores with different architectures, different versions and different compiling options, the workload is huge, the stability risk is high, and if the related functions have faults, the cores are problematic, and the risks of downtime, outage and even data error are possibly caused.
According to the embodiment, metadata backup is completed at the important node of the service process, and then the metadata backup service performs the scheme of leak detection and deficiency detection, so that the corresponding metadata can be ensured to be searched at the time point of the kernel memory as much as possible, and the extra disk IO operation is avoided. In addition, the application programming interface supported by the operating system does not need the kernel to make adaptation, so that the risk of the kernel not adapting is avoided.
The embodiment of the application provides a metadata backup method, and the method in the embodiment can be applied to a computing device, and the computing device can include: server, user terminal, etc. FIG. 2 is a flowchart of a metadata backup method according to an embodiment of the present application, where the method includes:
in step S201, in the case of receiving a preset operation for a storage file, metadata of the storage file is acquired before responding to the preset operation.
Step S202, the metadata is backed up.
Wherein the preset operation includes any one of the following: after the data writing operation is carried out on the storage file, closing operation is carried out on the storage file after the writing operation; deleting the storage file; and carrying out data verification on the storage file.
The method in the present embodiment is applicable to any storage system, and is not limited to HDD, SSD, and the like.
In this embodiment, under the condition that a preset operation for a storage file is received, before the preset operation is processed, the preset operation is intercepted, metadata of the storage file corresponding to the preset operation is obtained, backup is performed, then the preset operation is processed, and a closing operation, a deleting operation or a data checking operation is performed continuously.
Metadata (Metadata), also called intermediate data and relay data, is data describing data, mainly describing information of data attributes, and is used for supporting functions such as indicating storage locations, historical data, resource searching, file recording, and the like.
Illustratively, the metadata includes at least one of: file name, file length, creation time, inode identification, storage location information. The inode identification may be an inode number, or the like. The storage location information may be interval (extension) information, one extension being defined by the starting block plus the length.
The file is stored on a hard disk, and the minimum storage unit of the hard disk is a Sector (Sector). Each sector stores 512 bytes. When an operating system reads a hard disk, a plurality of sectors are read continuously at a time, that is, one block at a time, instead of reading each sector. Such a block consisting of a plurality of sectors is a minimum unit of file access, and for example, the size of the block may be 4KB, that is, eight sectors in succession constitute a block in which file data is stored. The region storing metadata is an inode.
In order to better play the performance of the HDD disk, the storage service writes the file in a manner of writing in a large block as much as possible, and Ops (Operations) for calling a closing operation after the file is written are relatively low, and Ops for performing metadata backup operation is correspondingly reduced.
After the file is deleted on the file system, the metadata of the file is released, the important information of data recovery is the storage positions of the file on the disk, if the positions are not overwritten, the storage service can re-splice the data on the disk into the original user file, and in order to enhance the data reliability of the system and increase the possibility of recovering the data, the metadata is backed up before the file is deleted.
The purpose of data verification is to detect silent errors on the HDD disk, and during data verification operations, data is read from the disk and checked for integrity and consistency, and before the data verification is completed, metadata backup of the verified file is performed.
According to the metadata backup method provided by the embodiment of the application, under the condition that the preset operation for the storage file is received, before the preset operation is responded, metadata of the storage file is obtained, and the metadata is backed up; wherein the preset operation includes any one of the following: after the data writing operation is carried out on the storage file, closing operation is carried out on the storage file after the writing operation; deleting the storage file; and carrying out data verification on the storage file. In this embodiment, under the condition that a preset operation for a storage file is received, intercepting the operation, acquiring metadata and backing up the metadata, wherein the storage file after a write operation is closed, and the storage file is subjected to data verification, before the operations, the metadata is also in a kernel memory, and at this time, the metadata is acquired, so that the IO capability of a disk is not consumed; for the operation of deleting the storage file, the deleting operation is also used for loading metadata, so that the metadata acquired before deleting the storage file belongs to the preheating operation, the IO frequency of a disk is not additionally increased, the expenditure of metadata backup is reduced, and the availability of backup service is improved.
The specific way to obtain the metadata is as follows:
in one implementation, in step S201, obtaining metadata of a storage file includes: metadata of the storage file is obtained through an application programming interface provided by the operating system.
In the file system, an operating system acquires key information in file metadata through a stat interface: file name, file length, creation/modification time, inode identification, etc. And acquiring storage position information in the metadata through a fiemap interface.
In this embodiment, metadata information of the file is obtained through an application programming interface provided by the operating system, so that data backup is completed, no kernel is required to make adaptation, and the risk of system crash caused by kernel discomfort can be avoided.
In one implementation, the method further comprises: acquiring metadata of effective storage files in the storage service according to a preset time period, and comparing the metadata of the effective storage files with backed-up metadata; if the metadata of the effective storage file is not matched with the backed-up metadata, the metadata of the unmatched storage file is backed up.
In actual operation, the data service may suddenly fail or abnormally exit at a certain moment, so that metadata of a part of files are not normally backed up, and in this case, a metadata service channel is required to perform leak detection and defect repair on lost files. And (3) regularly acquiring the metadata of the effective storage file in the storage service, comparing the metadata with the backed-up metadata, and if the metadata are matched, not modifying the file and not needing to be backed up again. If the files are not matched, the files are modified after the last backup, and the incremental backup needs to be triggered to be corrected. The effective storage file may be a file meeting preset requirements, where the preset requirements include a format requirement, a time requirement, and the like.
In one implementation, the mismatch of the metadata of the active storage file and the backed up metadata includes a mismatch of the metadata of the active storage file and at least one of a file name, a file length, a creation time, and an inode identification in the backed up metadata.
In an example, if key information (file name, file length, creation time, inode identification) in metadata of a valid stored file is completely matched with key information in backed up metadata, then the metadata of the file is considered to be matched with backed up metadata, and no metadata modification exists at this time, and backup is not needed. If the metadata is not matched, the metadata is modified, and the storage position information needs to be read again for backup.
In one implementation, obtaining metadata of a storage file valid in a storage service according to a preset time period includes: and communicating with the storage service or reading a check point file of the storage service according to a preset time period in a remote procedure call mode, and acquiring metadata of an effective storage file in the storage service.
In practical application, a remote procedure call (Remote Procedure Call, RPC) mode or a checkpoint (checkpoint) file is used for acquiring a file list of a storage file effective in a storage service, a file name, a file length, creation time, an index node identifier and the like in metadata, and acquiring the information through the RPC without disk IO, wherein data to be read through the checkpoint file is less and concentrated, so that no loss is caused to the IO performance of the HDD disk.
In one implementation, backing up metadata includes: metadata is persisted into the backup database using the metadata backup engine.
Data backup is the basis of disaster recovery, which is the process of copying all or part of data sets from the hard disk or array of an application host to other storage media in order to prevent the data loss caused by misoperation or system failure of the system. When the data is backed up, a data backup engine is started, and metadata needing to be backed up are persisted into a backup database.
Corresponding to the application scenario and method of the method provided by the embodiment of the present application, the embodiment of the present application further provides a metadata backup device, where the device is applied to a client of an access platform, as shown in fig. 3, which is a structural block diagram of the metadata backup device according to an embodiment of the present application, and the device includes:
an obtaining module 301, configured to, in a case where a preset operation for a storage file is received, obtain metadata of the storage file before responding to the preset operation;
a backup module 302, configured to backup metadata;
wherein the preset operation includes any one of the following: after the data writing operation is carried out on the storage file, closing operation is carried out on the storage file after the writing operation; deleting the storage file; and carrying out data verification on the storage file.
In the metadata backup device provided by the embodiment of the application, under the condition that the preset operation for the storage file is received, before responding to the preset operation, the metadata of the storage file is obtained, and the metadata is backed up; wherein the preset operation includes any one of the following: after the data writing operation is carried out on the storage file, closing operation is carried out on the storage file after the writing operation; deleting the storage file; and carrying out data verification on the storage file. In this embodiment, under the condition that a preset operation for a storage file is received, intercepting the operation, acquiring metadata and backing up the metadata, wherein the storage file after a write operation is closed, and the storage file is subjected to data verification, before the operations, the metadata is also in a kernel memory, and at this time, the metadata is acquired, so that the IO capability of a disk is not consumed; for the operation of deleting the storage file, the deleting operation is also used for loading metadata, so that the metadata acquired before deleting the storage file belongs to the preheating operation, the IO frequency of a disk is not additionally increased, the expenditure of metadata backup is reduced, and the availability of backup service is improved.
In one implementation, the obtaining module 301, when obtaining metadata of a storage file, is configured to:
metadata of the storage file is obtained through an application programming interface provided by the operating system.
In one implementation, the apparatus further includes an alignment module configured to:
acquiring metadata of effective storage files in the storage service according to a preset time period, and comparing the metadata of the effective storage files with backed-up metadata;
if the metadata of the effective storage file is not matched with the backed-up metadata, the metadata of the unmatched storage file is backed up.
In one implementation, the mismatch of the metadata of the active storage file and the backed up metadata includes a mismatch of the metadata of the active storage file and at least one of a file name, a file length, a creation time, and an inode identification in the backed up metadata.
In one implementation, when the comparison module obtains metadata of the stored valid storage file according to a preset time period, the comparison module is configured to:
and communicating with the storage service or reading a check point file of the storage service according to a preset time period in a remote procedure call mode, and acquiring metadata of an effective storage file in the storage service.
In one implementation, the backup module 302 is configured to:
metadata is persisted into the backup database using the metadata backup engine.
In one implementation, the metadata includes at least one of: file name, file length, creation time, inode identification, storage location information.
The functions of each module in each device of the embodiments of the present application may be referred to the corresponding descriptions in the above methods, and have corresponding beneficial effects, which are not described herein.
Fig. 4 is a block diagram of an electronic device used to implement an embodiment of the present application. As shown in fig. 4, the electronic device includes: memory 410 and processor 420, memory 410 stores a computer program executable on processor 420. The processor 420, when executing the computer program, implements the methods of the above-described embodiments. The number of memories 410 and processors 420 may be one or more.
The electronic device further includes:
and the communication interface 430 is used for communicating with external equipment and carrying out data interaction transmission.
If the memory 410, the processor 420, and the communication interface 430 are implemented independently, the memory 410, the processor 420, and the communication interface 430 may be connected to each other and communicate with each other through buses. The bus may be an industry standard architecture (Industry Standard Architecture, ISA) bus, an external device interconnect (Peripheral Component Interconnect, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in fig. 4, but not only one bus or one type of bus.
Alternatively, in a specific implementation, if the memory 410, the processor 420, and the communication interface 430 are integrated on a chip, the memory 410, the processor 420, and the communication interface 430 may communicate with each other through internal interfaces.
The present embodiments provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods provided in the embodiments of the present application.
The embodiment of the application also provides a chip, which comprises a processor and is used for calling the instructions stored in the memory from the memory and running the instructions stored in the memory, so that the communication device provided with the chip executes the method provided by the embodiment of the application.
The embodiment of the application also provides a chip, which comprises: the input interface, the output interface, the processor and the memory are connected through an internal connection path, the processor is used for executing codes in the memory, and when the codes are executed, the processor is used for executing the method provided by the application embodiment.
It should be appreciated that the processor may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or any conventional processor or the like. It is noted that the processor may be a processor supporting an advanced reduced instruction set machine (Advanced RISC Machines, ARM) architecture.
Further alternatively, the memory may include a read-only memory and a random access memory. The memory may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), programmable ROM (PROM), erasable Programmable ROM (EPROM), electrically Erasable EPROM (EEPROM), or flash Memory, among others. Volatile memory can include random access memory (Random Access Memory, RAM), which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available. For example, static RAM (SRAM), dynamic RAM (Dynamic Random Access Memory, DRAM), synchronous DRAM (SDRAM), double Data Rate Synchronous DRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), and Direct RAM (DR RAM).
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions in accordance with the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. Computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
Any process or method described in flow charts or otherwise herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process. And the scope of the preferred embodiments of the present application includes additional implementations in which functions may be performed in a substantially simultaneous manner or in an opposite order from that shown or discussed, including in accordance with the functions that are involved.
Logic and/or steps described in the flowcharts or otherwise described herein, e.g., may be considered a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the methods of the embodiments described above may be performed by a program that, when executed, comprises one or a combination of the steps of the method embodiments, instructs the associated hardware to perform the method.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules described above, if implemented in the form of software functional modules and sold or used as a stand-alone product, may also be stored in a computer-readable storage medium. The storage medium may be a read-only memory, a magnetic or optical disk, or the like.
The foregoing is merely exemplary embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think of various changes or substitutions within the technical scope of the present application, which should be covered in the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A method of metadata backup, the method comprising:
under the condition that a preset operation for a storage file is received, before responding to the preset operation, acquiring metadata of the storage file, and backing up the metadata;
wherein the preset operation includes any one of the following:
after the data writing operation is carried out on the storage file, closing operation is carried out on the storage file after the writing operation;
deleting the storage file;
and carrying out data verification on the storage file.
2. The method of claim 1, wherein the obtaining metadata of the storage file comprises:
and acquiring the metadata of the storage file through an application programming interface provided by an operating system.
3. The method according to claim 1, wherein the method further comprises:
acquiring metadata of effective storage files in a storage service according to a preset time period, and comparing the metadata of the effective storage files with backed-up metadata;
and if the metadata of the effective storage file is not matched with the backed-up metadata, backing up the metadata of the unmatched storage file.
4. The method of claim 3, wherein the metadata of the active storage file not matching the backed up metadata includes the metadata of the active storage file not matching at least one of a file name, a file length, a creation time, and an inode identification in the backed up metadata.
5. A method according to claim 3, wherein the obtaining metadata of the stored valid storage file according to the preset time period includes:
and communicating with the storage service or reading a check point file of the storage service according to a preset time period in a remote procedure call mode, and acquiring metadata of an effective storage file in the storage service.
6. The method of claim 1, wherein the backing up the metadata comprises:
the metadata is persisted into a backup database using a metadata backup engine.
7. The method of any of claims 1-6, wherein the metadata comprises at least one of:
file name, file length, creation time, inode identification, storage location information.
8. A metadata backup apparatus, the apparatus comprising:
the device comprises an acquisition module, a storage file generation module and a storage file generation module, wherein the acquisition module is used for acquiring metadata of the storage file before responding to a preset operation under the condition that the preset operation aiming at the storage file is received;
the backup module is used for backing up the metadata;
wherein the preset operation includes any one of the following:
after the data writing operation is carried out on the storage file, closing operation is carried out on the storage file after the writing operation;
deleting the storage file;
and carrying out data verification on the storage file.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory, the processor implementing the method of any one of claims 1-7 when the computer program is executed.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when executed by a processor, implements the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310377190.8A CN116401096A (en) | 2023-04-10 | 2023-04-10 | Metadata backup method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310377190.8A CN116401096A (en) | 2023-04-10 | 2023-04-10 | Metadata backup method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116401096A true CN116401096A (en) | 2023-07-07 |
Family
ID=87011919
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310377190.8A Pending CN116401096A (en) | 2023-04-10 | 2023-04-10 | Metadata backup method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116401096A (en) |
-
2023
- 2023-04-10 CN CN202310377190.8A patent/CN116401096A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9268648B1 (en) | System and method for consistency verification of replicated data in a recovery system | |
| US10481988B2 (en) | System and method for consistency verification of replicated data in a recovery system | |
| CN106951345B (en) | Consistency test method and device for disk data of virtual machine | |
| US7487385B2 (en) | Apparatus and method for recovering destroyed data volumes | |
| CN102929750A (en) | Nonvolatile media dirty region tracking | |
| US7020805B2 (en) | Efficient mechanisms for detecting phantom write errors | |
| US20060200500A1 (en) | Method of efficiently recovering database | |
| US7631159B1 (en) | Faster recovery mechanism of validated continuous data protection (CDP) time image | |
| US20100017648A1 (en) | Complete dual system and system control method | |
| US9330153B2 (en) | System, method, and computer readable medium that coordinates between devices using exchange of log files | |
| CN114385418A (en) | Protection method, device, equipment and storage medium for communication equipment | |
| CN113312205B (en) | Data verification method and device, storage medium and computer equipment | |
| CN110222035A (en) | A kind of efficient fault-tolerance approach of database page based on exclusive or check and journal recovery | |
| CN113419680A (en) | Storage method and system | |
| CN116401096A (en) | Metadata backup method and device, electronic equipment and storage medium | |
| CN114138566B (en) | Data storage method and device of virtual machine, virtual machine and storage medium | |
| CN115495286A (en) | Test method, system, equipment and storage medium for timed backup of configuration file | |
| CN114461455A (en) | Method and device for repairing bad blocks of disk of stream replication cluster | |
| US11645333B1 (en) | Garbage collection integrated with physical file verification | |
| CN112307022A (en) | Metadata repairing method and related device | |
| CN113342751B (en) | Metadata processing method, device, equipment and readable storage medium | |
| CN115543225B (en) | Data processing method, system, device, computer storage medium and electronic equipment | |
| KR100204576B1 (en) | Structure of common block register built-in memory and shadow recovery structure & method | |
| CN112445761B (en) | File checking method and device and storage medium | |
| JP7180319B2 (en) | Information processing device and dump management method for information processing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |