CN113254273A - Method, system, device and medium for real-time recovery of principal metadata - Google Patents
Method, system, device and medium for real-time recovery of principal metadata Download PDFInfo
- Publication number
- CN113254273A CN113254273A CN202110646471.XA CN202110646471A CN113254273A CN 113254273 A CN113254273 A CN 113254273A CN 202110646471 A CN202110646471 A CN 202110646471A CN 113254273 A CN113254273 A CN 113254273A
- Authority
- CN
- China
- Prior art keywords
- data block
- linked list
- metadata
- block
- segment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000011084 recovery Methods 0.000 title description 8
- 230000004044 response Effects 0.000 claims abstract description 9
- 238000004590 computer program Methods 0.000 claims description 7
- 230000006835 compression Effects 0.000 description 7
- 238000007906 compression Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method, a system, equipment and a storage medium for recovering principal metadata in real time, wherein the method comprises the following steps: creating a first idle data block linked list, a first data block distribution linked list and a first section of first data block linked list; in response to the fact that any one of the read main metadata blocks is damaged, reading a user data block linked list and a segment metadata database linked list of each segment, and storing the read data of the user data blocks and the segment metadata blocks into a first data block distribution linked list and a first segment first data block linked list; marking each data block recorded by the main metadata block chain table as idle, and respectively storing each data block recorded by the main metadata block chain table to a first idle data block chain table and a second idle data block chain table corresponding to the main metadata block; and selecting an idle data block from the intersection of the first idle data block linked list and the second idle data block linked list to be allocated to the newly generated main metadata, and recording the identifier of the idle data block to the main metadata block linked list.
Description
Technical Field
The present invention relates to the field of data centers, and more particularly, to a method, a system, a computer device, and a readable medium for real-time recovery of principal metadata.
Background
In the modern times, the demand for storage capacity is increasing with the exponential growth of data. There are generally two ways to cope with the increase in storage capacity, one is to expand the storage capacity, but this leads to an increase in cost; and the other is to compress the data, so that the consumption of the storage capacity is reduced, and the cost is saved. The real-time compression storage system compresses data before the data are written into the storage device, so that the occupation of storage space is reduced, and the access to the storage device (such as a magnetic disk) can be further reduced, thereby improving the performance of the storage system.
In the prior art, when a real-time compression storage system is started, a main metadata block is read from a physical storage space, then a segment metadata block is read into a memory from the physical storage space according to an address of a first segment metadata block of a segment recorded by the main metadata block, and required user data can be read into the memory from the physical storage space according to a storage location of the user data recorded by the segment metadata block. When a real-time compression storage system receives user write data, the user write data is compressed in a memory, when the length of a plurality of compressed data can reach the length of one data block, the system allocates a free data block in a physical storage space, then the compressed data is written into the free data block from the memory, meanwhile, main metadata is updated, the state of the free data block which is just allocated is marked as 'allocated user data', section metadata is updated, and the storage position of the compressed user data in the physical storage space is recorded.
In the current real-time compression storage system, when a main metadata block is damaged, segment metadata blocks of all segments cannot be found, so that user data cannot be read, and the state of each data block in a physical storage space cannot be known, for example, whether the data block is idle or stored, whether the data block stores the user data, the segment metadata or the main metadata, so that each data block in the physical storage space needs to be traversed, segment metadata and the main metadata are reconstructed according to information such as the state of the data block recorded in each data block, the storage position of the user data, the data length and the like, and the performance of the real-time compression storage system is seriously affected.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method, a system, a computer device, and a computer readable storage medium for real-time restoring principal metadata, in which a newly added data block linked list is read and segment metadata blocks are traversed instead of all data blocks, so that principal metadata can be quickly restored, and performance of a real-time compressed storage system is improved.
Based on the above object, an aspect of the embodiments of the present invention provides a method for recovering principal metadata in real time, including the following steps: creating a first idle data block linked list, a first data block distribution linked list and a first section of first data block linked list; in response to the fact that any one of the read main metadata blocks is damaged, reading a user data block linked list and a segment metadata database linked list of each segment to which the user data block belongs, and storing the read data of the user data block and the segment metadata block into the first data block distribution linked list and the first segment first data block linked list; marking each data block recorded by the main metadata block chain table as idle, and respectively storing each data block recorded by the main metadata block chain table to the first idle data block chain table and a second idle data block chain table corresponding to the main metadata block; and selecting an idle data block from the intersection of the first idle data block linked list and the second idle data block linked list to be allocated to the newly generated main metadata, and recording the identifier of the idle data block to the main metadata block linked list.
In some embodiments, the selecting an idle data block from the intersection of the first and second linked lists of idle data blocks to assign to the newly generated primary metadata comprises: and reading the first idle data block linked list and assigning the read data to the second idle data block linked list.
In some embodiments, the saving the read data of the user data block and the segment metadata block to the first data block allocation linked list and the first segment metadata block linked list includes: storing the identifiers and the corresponding distribution states of the read user data blocks and the read segment metadata blocks to the first data block distribution linked list; and storing the identification of the first segment metadata block stored in the segment metadata database linked list of each belonging segment into the first segment head data block linked list.
In some embodiments, said recording the identity of the free data block to the master metadata block chaining table comprises: and deleting the identifier of the idle data block from the first idle data block linked list and the second idle data block linked list.
In some embodiments, said recording the identity of the free data block to the master metadata block chaining table comprises: and adding the identifier and the distribution state of the idle data block into a second data block distribution linked list corresponding to the main metadata block.
In some embodiments, the method further comprises: and responding to newly added user data or segment metadata, selecting a second free data block or a third free data block from the first free data block chain table to be distributed to the user data or the segment metadata.
In some embodiments, the method further comprises: and respectively recording the identifications of the second idle data block and the third idle data block into a user data block linked list and a segment metadata database linked list of the segment to which the identification belongs.
In another aspect of the embodiments of the present invention, a system for recovering principal metadata in real time is provided, including: the creating module is configured to create a first idle data block linked list, a first data block distribution linked list and a first segment first data block linked list; the first linked list module is configured to read the user data block linked list and the segment metadata database linked list of each segment in response to the fact that any one of the read main metadata blocks is damaged, and store the read data of the user data block and the segment metadata block into the first data block distribution linked list and the first segment first data block linked list; the second linked list module is configured to mark each data block recorded by the main metadata block linked list as idle, and store each data block recorded by the main metadata block linked list to the first idle data block linked list and a second idle data block linked list corresponding to the main metadata block; and the execution module is configured to select an idle data block from an intersection of the first idle data block linked list and the second idle data block linked list to allocate the idle data block to the newly generated main metadata, and record the identifier of the idle data block to the main metadata block linked list.
In another aspect of the embodiments of the present invention, there is also provided a computer device, including: at least one processor; and a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of the method as above.
In a further aspect of the embodiments of the present invention, a computer-readable storage medium is also provided, in which a computer program for implementing the above method steps is stored when the computer program is executed by a processor.
The invention has the following beneficial technical effects: the idle data block linked list, the data block distribution linked list and the segment head data block linked list which are required by restoring the main metadata are added, and in the writing process of user data, not only are the main metadata and the segment metadata updated, but also the added linked lists are updated. When the main metadata block is damaged and the main metadata needs to be recovered, the added idle data block linked list, the main metadata block linked list, the segment metadata block linked list, the user data block linked list corresponding to each segment and the like are sequentially read, so that the main metadata is reconstructed. By traversing the segment metadata blocks, the main metadata can be quickly recovered, and the performance of a real-time compression storage system is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
FIG. 1 is a diagram illustrating an embodiment of a method for real-time recovery of primary metadata according to the present invention;
FIG. 2 is a diagram illustrating a hardware structure of an embodiment of a computer device for real-time recovery of principal metadata according to the present invention;
FIG. 3 is a schematic diagram of an embodiment of a computer storage medium for real-time recovery of primary metadata according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two entities with the same name but different names or different parameters, and it should be noted that "first" and "second" are merely for convenience of description and should not be construed as limitations of the embodiments of the present invention, and they are not described in any more detail in the following embodiments.
In a first aspect of the embodiments of the present invention, an embodiment of a method for recovering principal metadata in real time is provided. Fig. 1 is a schematic diagram illustrating an embodiment of a method for recovering principal metadata in real time according to the present invention. As shown in fig. 1, the embodiment of the present invention includes the following steps:
s1, creating a first idle data block linked list, a first data block distribution linked list and a first section of first data block linked list;
s2, in response to the fact that any one of the read main metadata blocks is damaged, reading a user data block chain table and a segment metadata database chain table of each segment to which the main metadata block belongs, and storing the read data of the user data block and the segment metadata block into the first data block distribution chain table and the first segment first data block chain table;
s3, marking each data block recorded by the main metadata block chain table as idle, and respectively storing each data block recorded by the main metadata block chain table to a first idle data block chain table and a second idle data block chain table corresponding to the main metadata block; and
s4, selecting an idle data block from the intersection of the first idle data block linked list and the second idle data block linked list to be allocated to the newly generated main metadata, and recording the identifier of the idle data block to the main metadata block linked list.
The whole physical storage space can be divided into a plurality of data blocks with fixed length, when the physical storage space is large and the number of the data blocks is large, the data blocks are divided into a plurality of segments for convenient management, and each segment contains the same number of data blocks. The data blocks are divided into main metadata blocks, segment metadata blocks, and user data blocks according to usage. The main metadata block records main metadata, and the main metadata is used for recording the state (free or storage, and storing user data, segment metadata or main metadata) of each data block and the first address of the first segment metadata block of each segment; the segment metadata block is used for recording segment metadata, the segment metadata is used for recording storage positions of the compressed user data in the physical storage space, and the user data block is used for recording the compressed user data.
And creating a first idle data block linked list, a first data block distribution linked list and a first section of first data block linked list. The first free data block linked list is used for recording the identification (block ID) of free data blocks, the first data block distribution linked list is used for recording the identification and the state of distributed data blocks, and the first section first data block linked list is used for recording the identification of the first section metadata block of each section. In the embodiment of the invention, the First Free data Block linked List is Free _ Block _ List, the First data Block distribution linked List is Allocated _ Block _ List, and the First section of First data Block linked List is First _ Meta _ Block _ List.
In addition, the linked list corresponding to the main metadata includes: the First Free data Block linked List Master _ Free _ Block _ List, the second data Block distribution linked List Master _ Allocated _ Block _ List, the second segment First data Block linked List Master _ First _ Meta _ Block _ List, and the primary metadata Block linked List Master _ Block _ List. The linked list corresponding to the segment metadata comprises: the User data Block chain table User _ Block _ List and the Segment metadata Block chain table Segment _ Block _ List which form the Segment.
In some embodiments, the method further comprises: and responding to newly added user data or segment metadata, selecting a second free data block or a third free data block from the first free data block chain table to be distributed to the user data or the segment metadata. Different free data blocks can be allocated for different types of data, and new data can be written in the allocated free data blocks.
In some embodiments, the method further comprises: and respectively recording the identifications of the second idle data block and the third idle data block into a user data block linked list and a segment metadata database linked list of the segment to which the identification belongs. After writing corresponding data in the free data block, the identifier of the corresponding free data block can be recorded in the corresponding linked list of the segment to which the identifier corresponds, so as to know the condition of recording data in the data block. Meanwhile, the identifier and the state of the data block in which the data is written may also be recorded in the first data block allocation linked list and the second data block allocation linked list.
And after the idle data blocks are distributed, deleting the identifiers of the data blocks from the first idle data block linked list and the second idle data block linked list. If the data Block is used for storing User data, recording the Block ID of the distributed data Block on a User _ Block _ List of the segment to which the data Block belongs; if the data Block is used for storing Segment metadata, recording the Block ID of the distributed data Block on Segment _ Block _ List of the Segment to which the data Block belongs; if the data Block is used for storing the pivot metadata, the Block ID of the allocated data Block is recorded on the Master _ Block _ List.
And in response to the fact that any one of the read main metadata blocks is damaged, reading the user data block linked list and the segment metadata database linked list of each segment to which the user data block belongs, and storing the read data of the user data block and the segment metadata block into the first data block distribution linked list and the first segment first data block linked list.
In some embodiments, the saving the read data of the user data block and the segment metadata block to the first data block allocation linked list and the first segment metadata block linked list includes: storing the identifiers and the corresponding distribution states of the read user data blocks and the read segment metadata blocks to the first data block distribution linked list; and storing the identification of the first segment metadata block stored in the segment metadata database linked list of each belonging segment into the first segment head data block linked list. For each Segment, sequentially reading a User _ Block _ List and a Segment _ Block _ List of the Segment, and storing Block IDs and corresponding distribution states of User data blocks and Segment metadata blocks recorded by the Segment into a first data Block distribution linked List; and sequentially storing the Block ID of the first Segment metadata Block stored in the Segment _ Block _ List of each Segment on the first Segment head data Block linked List.
Marking each data block recorded by the main metadata block chain table as idle, and respectively storing each data block recorded by the main metadata block chain table to the first idle data block chain table and a second idle data block chain table corresponding to the main metadata block. Each data Block of the Master _ Block _ List record is marked as Free and saved to Free _ Block _ List and Master _ Free _ Block _ List, respectively. In order to recover the main metadata block, in the embodiment of the present invention, it is necessary that all data blocks that originally record data in the main metadata block chain table can record newly generated main metadata, so that each data block recorded in the main metadata block chain table is marked as free.
And selecting an idle data block from the intersection of the first idle data block linked list and the second idle data block linked list to be allocated to the newly generated main metadata, and recording the identifier of the idle data block to the main metadata block linked list. And allocating new Free data blocks for the newly generated main metadata and writing the data, recording the Block IDs of the data blocks on a Master _ Block _ List, and updating the Free _ Block _ List, the Master _ Free _ Block _ List and the Master _ Allocated _ Block _ List. One or more free data blocks are selected from the intersection of the first free data block linked list and the second free data block linked list as main metadata blocks to receive newly generated main metadata to replace the damaged main metadata blocks.
In some embodiments, said recording the identity of the free data block to the master metadata block chaining table comprises: and deleting the identifier of the idle data block from the first idle data block linked list and the second idle data block linked list.
In some embodiments, said recording the identity of the free data block to the master metadata block chaining table comprises: and adding the identifier and the distribution state of the idle data block into a second data block distribution linked list corresponding to the main metadata block.
In some embodiments, the selecting an idle data block from the intersection of the first and second linked lists of idle data blocks to assign to the newly generated primary metadata comprises: and reading the first idle data block linked list and assigning the read data to the second idle data block linked list. And reading a Free data Block linked List Free _ Block _ List, and assigning the Free data Block linked List Free _ Block _ List to a Master _ Free _ Block _ List.
In the embodiment, the idle data block linked list, the data block distribution linked list and the segment head data block linked list which are required by restoring the main metadata are added, and in the writing process of user data, not only are the main metadata and the segment metadata updated, but also the added linked lists are updated. When the main metadata block is damaged and the main metadata needs to be recovered, the added idle data block linked list, the main metadata block linked list, the segment metadata block linked list, the user data block linked list corresponding to each segment and the like are sequentially read, so that the main metadata is reconstructed. By traversing the segment metadata blocks, the main metadata can be quickly recovered, and the performance of a real-time compression storage system is improved.
It should be particularly noted that, the steps in the embodiments of the method for restoring principal metadata in real time can be mutually intersected, replaced, added, and deleted, so that these reasonable permutation and combination transformations as well as the method for restoring principal metadata in real time shall also belong to the scope of the present invention, and shall not limit the scope of the present invention to the embodiments.
In view of the above object, according to a second aspect of the embodiments of the present invention, there is provided a system for restoring principal metadata in real time, including: the creating module is configured to create a first idle data block linked list, a first data block distribution linked list and a first segment first data block linked list; the first linked list module is configured to read the user data block linked list and the segment metadata database linked list of each segment in response to the fact that any one of the read main metadata blocks is damaged, and store the read data of the user data block and the segment metadata block into the first data block distribution linked list and the first segment first data block linked list; the second linked list module is configured to mark each data block recorded by the main metadata block linked list as idle, and store each data block recorded by the main metadata block linked list to the first idle data block linked list and a second idle data block linked list corresponding to the main metadata block; and the execution module is configured to select an idle data block from an intersection of the first idle data block linked list and the second idle data block linked list to allocate the idle data block to the newly generated main metadata, and record the identifier of the idle data block to the main metadata block linked list.
In some embodiments, the execution module is configured to: and reading the first idle data block linked list and assigning the read data to the second idle data block linked list.
In some embodiments, the first linked list module is configured to: storing the identifiers and the corresponding distribution states of the read user data blocks and the read segment metadata blocks to the first data block distribution linked list; and storing the identification of the first segment metadata block stored in the segment metadata database linked list of each belonging segment into the first segment head data block linked list.
In some embodiments, the execution module is configured to: and deleting the identifier of the idle data block from the first idle data block linked list and the second idle data block linked list.
In some embodiments, the execution module is configured to: and adding the identifier and the distribution state of the idle data block into a second data block distribution linked list corresponding to the main metadata block.
In some embodiments, the system further comprises an assignment module configured to: and responding to newly added user data or segment metadata, selecting a second free data block or a third free data block from the first free data block chain table to be distributed to the user data or the segment metadata.
In some embodiments, the system further comprises a recording module configured to: and respectively recording the identifications of the second idle data block and the third idle data block into a user data block linked list and a segment metadata database linked list of the segment to which the identification belongs.
In view of the above object, a third aspect of the embodiments of the present invention provides a computer device, including: at least one processor; and a memory storing computer instructions executable on the processor, the instructions being executable by the processor to perform the steps of: s1, creating a first idle data block linked list, a first data block distribution linked list and a first section of first data block linked list; s2, in response to the fact that any one of the read main metadata blocks is damaged, reading a user data block chain table and a segment metadata database chain table of each segment to which the main metadata block belongs, and storing the read data of the user data block and the segment metadata block into the first data block distribution chain table and the first segment first data block chain table; s3, marking each data block recorded by the main metadata block chain table as idle, and respectively storing each data block recorded by the main metadata block chain table to a first idle data block chain table and a second idle data block chain table corresponding to the main metadata block; and S4, selecting an idle data block from the intersection of the first idle data block linked list and the second idle data block linked list to be allocated to the newly generated main metadata, and recording the identifier of the idle data block to the main metadata block linked list.
In some embodiments, the selecting an idle data block from the intersection of the first and second linked lists of idle data blocks to assign to the newly generated primary metadata comprises: and reading the first idle data block linked list and assigning the read data to the second idle data block linked list.
In some embodiments, the saving the read data of the user data block and the segment metadata block to the first data block allocation linked list and the first segment metadata block linked list includes: storing the identifiers and the corresponding distribution states of the read user data blocks and the read segment metadata blocks to the first data block distribution linked list; and storing the identification of the first segment metadata block stored in the segment metadata database linked list of each belonging segment into the first segment head data block linked list.
In some embodiments, said recording the identity of the free data block to the master metadata block chaining table comprises: and deleting the identifier of the idle data block from the first idle data block linked list and the second idle data block linked list.
In some embodiments, said recording the identity of the free data block to the master metadata block chaining table comprises: and adding the identifier and the distribution state of the idle data block into a second data block distribution linked list corresponding to the main metadata block.
In some embodiments, the steps further comprise: and responding to newly added user data or segment metadata, selecting a second free data block or a third free data block from the first free data block chain table to be distributed to the user data or the segment metadata.
In some embodiments, the steps further comprise: and respectively recording the identifications of the second idle data block and the third idle data block into a user data block linked list and a segment metadata database linked list of the segment to which the identification belongs.
Fig. 2 is a schematic hardware structure diagram of an embodiment of the computer device for recovering principal metadata in real time according to the present invention.
Taking the apparatus shown in fig. 2 as an example, the apparatus includes a processor 201 and a memory 202, and may further include: an input device 203 and an output device 204.
The processor 201, the memory 202, the input device 203 and the output device 204 may be connected by a bus or other means, and fig. 2 illustrates the connection by a bus as an example.
The memory 202, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions/modules corresponding to the method for recovering principal metadata in real time in the embodiments of the present application. The processor 201 executes various functional applications of the server and data processing by running nonvolatile software programs, instructions, and modules stored in the memory 202, that is, implements the method of restoring principal metadata in real time of the above-described method embodiments.
The memory 202 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the method of restoring the main metadata in real time, and the like. Further, the memory 202 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 202 may optionally include memory located remotely from processor 201, which may be connected to local modules via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 203 may receive information such as a user name and a password that are input. The output device 204 may include a display device such as a display screen.
Program instructions/modules corresponding to one or more methods of restoring the primary metadata in real-time are stored in the memory 202 and, when executed by the processor 201, perform the method of restoring the primary metadata in real-time in any of the above-described method embodiments.
Any embodiment of a computer device implementing the method for real-time recovery of primary metadata as described above may achieve the same or similar effects as any of the preceding method embodiments corresponding thereto.
The invention also provides a computer readable storage medium storing a computer program which, when executed by a processor, performs the method as above.
Fig. 3 is a schematic diagram of an embodiment of a computer storage medium for real-time recovery of main metadata according to the present invention. Taking the computer storage medium as shown in fig. 3 as an example, the computer readable storage medium 3 stores a computer program 31 which, when executed by a processor, performs the method as described above.
Finally, it should be noted that, as one of ordinary skill in the art can appreciate that all or part of the processes of the methods of the above embodiments can be implemented by a computer program to instruct related hardware, and the program of the method for recovering principal metadata in real time can be stored in a computer readable storage medium, and when executed, the program can include the processes of the embodiments of the methods as described above. The storage medium of the program may be a magnetic disk, an optical disk, a Read Only Memory (ROM), a Random Access Memory (RAM), or the like. The embodiments of the computer program may achieve the same or similar effects as any of the above-described method embodiments.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.
Claims (10)
1. A method for recovering principal metadata in real-time, comprising the steps of:
creating a first idle data block linked list, a first data block distribution linked list and a first section of first data block linked list;
in response to the fact that any one of the read main metadata blocks is damaged, reading a user data block linked list and a segment metadata database linked list of each segment to which the user data block belongs, and storing the read data of the user data block and the segment metadata block into the first data block distribution linked list and the first segment first data block linked list;
marking each data block recorded by the main metadata block chain table as idle, and respectively storing each data block recorded by the main metadata block chain table to the first idle data block chain table and a second idle data block chain table corresponding to the main metadata block; and
and selecting an idle data block from the intersection of the first idle data block linked list and the second idle data block linked list to be allocated to the newly generated main metadata, and recording the identifier of the idle data block to the main metadata block linked list.
2. The method of claim 1, wherein the selecting a free data block from the intersection of the first free data block linked list and the second free data block linked list to assign to the newly generated primary metadata comprises:
and reading the first idle data block linked list and assigning the read data to the second idle data block linked list.
3. The method of claim 1, wherein saving the read data of the user data block and the segment metadata block to the first data block allocation linked list and the first segment metadata block linked list comprises:
storing the identifiers and the corresponding distribution states of the read user data blocks and the read segment metadata blocks to the first data block distribution linked list; and
and storing the identification of the first segment metadata block stored in the segment metadata database linked list of each belonging segment into the first segment data block linked list.
4. The method of claim 1, wherein recording the identity of the free data block to the master metadata block chaining table comprises:
and deleting the identifier of the idle data block from the first idle data block linked list and the second idle data block linked list.
5. The method of claim 1, wherein recording the identity of the free data block to the master metadata block chaining table comprises:
and adding the identifier and the distribution state of the idle data block into a second data block distribution linked list corresponding to the main metadata block.
6. The method of claim 1, further comprising:
and responding to newly added user data or segment metadata, selecting a second free data block or a third free data block from the first free data block chain table to be distributed to the user data or the segment metadata.
7. The method of claim 6, further comprising:
and respectively recording the identifications of the second idle data block and the third idle data block into a user data block linked list and a segment metadata database linked list of the segment to which the identification belongs.
8. A system for recovering principal metadata in real-time, comprising:
the creating module is configured to create a first idle data block linked list, a first data block distribution linked list and a first segment first data block linked list;
the first linked list module is configured to read the user data block linked list and the segment metadata database linked list of each segment in response to the fact that any one of the read main metadata blocks is damaged, and store the read data of the user data block and the segment metadata block into the first data block distribution linked list and the first segment first data block linked list;
the second linked list module is configured to mark each data block recorded by the main metadata block linked list as idle, and store each data block recorded by the main metadata block linked list to the first idle data block linked list and a second idle data block linked list corresponding to the main metadata block;
and the execution module is configured to select an idle data block from an intersection of the first idle data block linked list and the second idle data block linked list to allocate the idle data block to the newly generated main metadata, and record the identifier of the idle data block to the main metadata block linked list.
9. A computer device, comprising:
at least one processor; and
a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of the method of any one of claims 1 to 7.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110646471.XA CN113254273A (en) | 2021-06-10 | 2021-06-10 | Method, system, device and medium for real-time recovery of principal metadata |
| PCT/CN2021/127407 WO2022257337A1 (en) | 2021-06-10 | 2021-10-29 | Method and system for recovering master metadata in real time, and device and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110646471.XA CN113254273A (en) | 2021-06-10 | 2021-06-10 | Method, system, device and medium for real-time recovery of principal metadata |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113254273A true CN113254273A (en) | 2021-08-13 |
Family
ID=77187423
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110646471.XA Withdrawn CN113254273A (en) | 2021-06-10 | 2021-06-10 | Method, system, device and medium for real-time recovery of principal metadata |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113254273A (en) |
| WO (1) | WO2022257337A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022257337A1 (en) * | 2021-06-10 | 2022-12-15 | 苏州浪潮智能科技有限公司 | Method and system for recovering master metadata in real time, and device and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106708665A (en) * | 2016-12-20 | 2017-05-24 | 华为技术有限公司 | Metadata restoration method and device |
| CN109683825A (en) * | 2018-12-24 | 2019-04-26 | 广东浪潮大数据研究有限公司 | A kind of storage system online data compression method, device and equipment |
| CN111274295A (en) * | 2020-01-12 | 2020-06-12 | 苏州浪潮智能科技有限公司 | Method, device, equipment and medium for rapidly loading data in database |
| US20200341849A1 (en) * | 2019-04-29 | 2020-10-29 | EMC IP Holding Company LLC | Method to recover metadata in a content aware storage system |
| CN112799592A (en) * | 2021-01-29 | 2021-05-14 | 苏州浪潮智能科技有限公司 | Multi-namespace allocation method, device, equipment and readable medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2369206B (en) * | 2000-11-18 | 2004-11-03 | Ibm | Method for rebuilding meta-data in a data storage system and a data storage system |
| CN103488710B (en) * | 2013-09-10 | 2018-04-24 | 广州巨杉软件开发有限公司 | The non-fixed-length data method of efficient storage in big data page |
| CN113254273A (en) * | 2021-06-10 | 2021-08-13 | 苏州浪潮智能科技有限公司 | Method, system, device and medium for real-time recovery of principal metadata |
-
2021
- 2021-06-10 CN CN202110646471.XA patent/CN113254273A/en not_active Withdrawn
- 2021-10-29 WO PCT/CN2021/127407 patent/WO2022257337A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106708665A (en) * | 2016-12-20 | 2017-05-24 | 华为技术有限公司 | Metadata restoration method and device |
| CN109683825A (en) * | 2018-12-24 | 2019-04-26 | 广东浪潮大数据研究有限公司 | A kind of storage system online data compression method, device and equipment |
| US20200341849A1 (en) * | 2019-04-29 | 2020-10-29 | EMC IP Holding Company LLC | Method to recover metadata in a content aware storage system |
| CN111274295A (en) * | 2020-01-12 | 2020-06-12 | 苏州浪潮智能科技有限公司 | Method, device, equipment and medium for rapidly loading data in database |
| CN112799592A (en) * | 2021-01-29 | 2021-05-14 | 苏州浪潮智能科技有限公司 | Multi-namespace allocation method, device, equipment and readable medium |
Non-Patent Citations (1)
| Title |
|---|
| 曹二山: "海量流数据存储系统元数据服务器的设计与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022257337A1 (en) * | 2021-06-10 | 2022-12-15 | 苏州浪潮智能科技有限公司 | Method and system for recovering master metadata in real time, and device and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022257337A1 (en) | 2022-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110018983B (en) | Metadata query method and device | |
| CN106446044B (en) | Storage space recovery method and device | |
| CN109240607B (en) | File reading method and device | |
| CN111198856B (en) | File management method, device, computer equipment and storage medium | |
| CN110147203B (en) | File management method and device, electronic equipment and storage medium | |
| CN104346357A (en) | File accessing method and system for embedded terminal | |
| CN106383666B (en) | Data storage method and device | |
| CN109976669B (en) | Edge storage method, device and storage medium | |
| EP3322155A1 (en) | Virtual disk processing method and apparatus | |
| CN112269665B (en) | Memory processing method and device, electronic equipment and storage medium | |
| CN104933051A (en) | File storage space recovery method and device | |
| CN113254273A (en) | Method, system, device and medium for real-time recovery of principal metadata | |
| CN114443598A (en) | Data writing method and device, computer equipment and storage medium | |
| CN113391916A (en) | Organization architecture data processing method, device, computer equipment and storage medium | |
| CN106980685B (en) | Data processing method and data processing device | |
| CN114675776A (en) | Resource storage method and device, storage medium and electronic equipment | |
| CN114185849A (en) | File operation method, file operation system, electronic device and storage medium | |
| CN113467997A (en) | Data recovery method and device, mobile device and storage medium | |
| CN108196790B (en) | Data management method, storage device, and computer-readable storage medium | |
| CN114020707B (en) | Storage space recovery method, storage medium, and program product | |
| CN113609076B (en) | File storage method and file reading method | |
| CN112997162A (en) | Method and device for deleting index entry in memory | |
| CN117539409B (en) | Query acceleration method and device based on data cache, medium and electronic equipment | |
| CN114356232B (en) | Data reading and writing method and device | |
| CN110018988B (en) | Snapshot deleting method, processing method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210813 |