CN112000623A - Metadata access method and device and computer readable storage medium - Google Patents
Metadata access method and device and computer readable storage medium Download PDFInfo
- Publication number
- CN112000623A CN112000623A CN202010791402.3A CN202010791402A CN112000623A CN 112000623 A CN112000623 A CN 112000623A CN 202010791402 A CN202010791402 A CN 202010791402A CN 112000623 A CN112000623 A CN 112000623A
- Authority
- CN
- China
- Prior art keywords
- log
- metadata
- file
- unit
- access offset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
- G06F16/164—File meta data generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a method, a device and a medium for accessing a metadata log, wherein when a metadata write-in request is received, metadata carried in the metadata write-in request is cached to a preset memory space, and a log file corresponding to a file identifier carried in the metadata write-in request is searched; and writing the log event corresponding to the metadata into the log file according to the access offset of the additional write file recorded in the log file. And executing a disk-dropping operation on the metadata cached in the memory space according to the set cycle time, and updating the outdated data file access offset recorded in the log header of the log file after completing the disk-dropping of the metadata. Through caching the metadata and then executing the disk-dropping operation, the processing efficiency of the metadata is greatly improved. And by recording the log file, the recovery of the metadata can be realized according to the recorded log file, and the smooth processing of the metadata request is effectively ensured.
Description
Technical Field
The present invention relates to the field of storage system technologies, and in particular, to a method and an apparatus for accessing a metadata log, and a computer-readable storage medium.
Background
With the development of the internet, a traditional centralized network storage system cannot meet the storage of massive PB-level data, and a storage server becomes a bottleneck of system performance and is also a focus of reliability and security. The distributed network storage system adopts an expandable system structure, and a plurality of storage servers are used for sharing storage load, so that the reliability, the availability and the access efficiency of the system are effectively improved.
There are metadata (Meta Date, MD) and data in the file system, the data refers to actual data in a general file, and the metadata is system data for describing a characteristic attribute of a file, etc. In a distributed storage system, metadata and data are stored separately, and a subsystem providing metadata services in a distributed file system may be referred to as a metadata Service system (MDS). In the distributed storage system, the operation on the metadata is a random IO request, and when the MDS receives the metadata IO request operation, the next metadata IO request can be executed only after the metadata IO request operation needs to be completed. When a system receives a plurality of metadata IO requests in a short time, the traditional processing method can cause the system to have an access bottleneck.
Therefore, how to improve the processing efficiency of metadata requests is a problem to be solved by those skilled in the art.
Disclosure of Invention
Embodiments of the present invention provide a method, an apparatus, and a computer-readable storage medium for accessing a metadata log, which can improve processing efficiency of a metadata request.
To solve the foregoing technical problem, an embodiment of the present invention provides a method for accessing metadata, including:
when a metadata writing request is received, caching metadata carried in the metadata writing request to a preset memory space, and searching a log file corresponding to a file identifier carried in the metadata writing request;
writing the log event corresponding to the metadata into the log file according to the access offset of the additional write file recorded in the log file;
and executing a disk-dropping operation on the metadata cached in the memory space according to a set cycle time, and updating the outdated data file access offset recorded in the log header of the log file after completing the disk-dropping of the metadata.
Optionally, after updating the outdated data file access offset recorded in the log header of the log file after completing the disk dropping of the metadata, the method further includes:
when the system is detected to be restarted, reading an access offset of an expired data file from a log header of the log file;
according to the outdated data file access offset, sequentially reading log entries from the log file, generating metadata according to log events in the log entries, and caching the generated metadata to the memory space;
and updating the access offset of the expired data file in the log header once every time one log entry is read, and ending the operation until all the log entries in the log file are read.
Optionally, before the sequentially reading the log entries from the log file according to the outdated data file access offset, the method further includes:
judging whether the check code carried in the log file is consistent with a preset check code or not;
and when the check code carried in the log file is consistent with the preset check code, executing the step of reading log entries from the log file in sequence according to the access offset of the expired data file.
Optionally, before the generating metadata according to the journal event in the journal entry and caching the generated metadata in the memory space, the method further includes:
judging whether a log flag bit carried in a currently read log entry is consistent with an access offset of an expired data file before updating;
when the log flag bit carried in the currently read log entry is consistent with the access offset of the expired data file before updating, executing the step of generating metadata according to the log event in the log entry and caching the generated metadata to the memory space;
and when the log flag bit carried in the currently read log entry is inconsistent with the access offset of the expired data file before updating, generating alarm prompt information.
Optionally, after updating the outdated data file access offset recorded in the log header of the log file after completing the disk dropping of the metadata, the method further includes:
and deleting the metadata of the completion of the disk drop from the memory space.
The embodiment of the invention also provides a metadata access device, which comprises a cache unit, a search unit, a write-in unit, a disk-dropping unit and an update unit;
the cache unit is used for caching the metadata carried in the metadata writing request to a preset memory space when the metadata writing request is received;
the searching unit is used for searching the log file corresponding to the file identifier carried in the metadata writing request;
the writing unit is configured to write the log event corresponding to the metadata into the log file according to an additional write file access offset recorded in the log file;
the disk dropping unit is used for executing disk dropping operation on the metadata cached in the memory space according to a set cycle time;
and the updating unit is used for updating the access offset of the expired data file recorded in the log header of the log file after the metadata is completely landed.
Optionally, the device further comprises a reading unit and a generating unit;
the reading unit is used for reading an access offset of an expired data file from a log header of the log file when the system restart is detected; sequentially reading log entries from the log file according to the access offset of the expired data file;
the generating unit is used for generating metadata according to the log events in the log entries and caching the generated metadata to the memory space;
the updating unit is further configured to update the outdated data file access offset in the log header once every time a log entry is read, and end the operation until all log entries in the log file are read.
Optionally, the system further comprises a first judging unit;
the first judging unit is used for judging whether the check code carried in the log file is consistent with a preset check code or not; and when the check code carried in the log file is consistent with the preset check code, triggering the reading unit to execute the step of sequentially reading the log entries from the log file according to the overdue data file access offset.
Optionally, the system further comprises a second judging unit and a prompting unit;
the second judging unit is used for judging whether a log flag bit carried in the currently read log entry is consistent with the access offset of the expired data file before updating; when the log flag bit carried in the currently read log entry is consistent with the access offset of the expired data file before updating, executing the step that the generating unit generates the metadata according to the log event in the log entry and caches the generated metadata to the memory space;
and the prompting unit is used for generating alarm prompting information when the log flag bit carried in the currently read log entry is inconsistent with the access offset of the expired data file before updating.
Optionally, a deleting unit is further included;
and the deleting unit is used for deleting the metadata of the completed disk drop from the memory space.
The embodiment of the present invention further provides an apparatus for accessing metadata, including:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the method for accessing metadata as described in any of the above.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements the steps of the metadata access method according to any one of the above.
According to the technical scheme, when the metadata writing request is received, the metadata carried in the metadata writing request is cached to the preset memory space, and the log file corresponding to the file identification carried in the metadata writing request is searched; and writing the log event corresponding to the metadata into the log file according to the access offset of the additional write file recorded in the log file. When a metadata write-in request is received, the metadata are cached firstly, and then the disk-dropping operation is executed on the cached metadata in the memory space according to the set cycle time, so that the response efficiency of the metadata write-in request is greatly improved. And updating an outdated data file access offset recorded in a log header of the log file after completing the destaging of the metadata. The log event of the metadata is recorded in a log file mode, the metadata can be tracked, even if the metadata in the memory space is lost due to abnormal exit of the metadata service system, the metadata can be recovered according to the recorded log file, and the smooth processing of the metadata request is effectively guaranteed.
Drawings
In order to illustrate the embodiments of the present invention more clearly, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained by those skilled in the art without inventive effort.
FIG. 1 is a flowchart illustrating a method for accessing metadata according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of an apparatus for accessing metadata according to an embodiment of the present invention;
fig. 3 is a schematic hardware structure diagram of an apparatus for accessing metadata according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without any creative work belong to the protection scope of the present invention.
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
Next, a method for accessing metadata according to an embodiment of the present invention will be described in detail. Fig. 1 is a flowchart of a method for accessing metadata according to an embodiment of the present invention, where the method includes:
s101: when a metadata writing request is received, caching metadata carried in the metadata writing request to a preset memory space, and searching a log file corresponding to a file identifier carried in the metadata writing request.
In the embodiment of the present invention, in order to ensure timely response to the metadata write request and avoid the problem of access bottleneck of the metadata service system caused by untimely response to the metadata write request, the metadata carried in the metadata write request may be cached first.
In the embodiment of the present invention, a log file may be constructed for each MDS, and the log file is used for recording log events of the metadata service system.
In order to distinguish the log files corresponding to the plurality of metadata service systems, a unique file identifier may be set for each log file. For example, the file identification of the log file of MDS 0 is set to 0x400, and the file identification of the log file of MDS 1 is set to 0x 401.
S102: and writing the log event corresponding to the metadata into the log file according to the access offset of the additional write file recorded in the log file.
In order to ensure the ordered recording of the log events in the log files, a log header is set for each log file, and an additional write file access offset and an expired data file access offset can be included in the log header.
The added write file access offset is used for indicating the write-in position of a newly added log event, and the expired data file access offset is used for indicating the initial position of the log event of which the metadata is not executed to be landed in the log file.
In an initial state, the value of the access offset of the additional write file and the access offset of the outdated data file is 0, the access offset of the additional write file can be changed continuously along with the continuous writing of log events in the log file, and the access offset of the outdated data file can be changed correspondingly after the disk-dropping operation of metadata corresponding to part of the log events is completed.
For example, three log entries are stored in one log file, the first log entry includes 100 log events, the second log entry includes 50 log events, and the third log entry includes 30 log events, where the additionally written file access offset is 181, that is, represents that a newly written log event is recorded from bit 181. Assuming that a disk-down operation has been performed on the metadata corresponding to the first journal entry, the access offset of the expired data file is 101, i.e. representing that the disk-down operation is performed from the metadata corresponding to the 101 st journal event.
The log events record the basic information and the way of operation needed to build the metadata. Each metadata has its corresponding log event, upon which the complete metadata can be generated.
S103: and executing a disk-dropping operation on the metadata cached in the memory space according to the set cycle time, and updating the outdated data file access offset recorded in the log header of the log file after completing the disk-dropping of the metadata.
The period time can be set according to actual requirements, and is not limited herein.
In the embodiment of the present invention, the metadata carried in the metadata write-in request is cached to the preset memory space, and for the actual disk-drop update of the metadata, the disk-drop operation of the metadata in the memory space needs to be executed periodically.
After completing the metadata landing, the outdated data file access offset recorded in the log header needs to be updated, so that the metadata service system can clearly know the starting position of the metadata which is not currently subjected to the metadata landing operation, and effective execution of subsequent metadata landing operation is ensured.
In order to avoid occupying the memory space for a long time, in the embodiment of the present invention, the metadata that completes the disk drop can be deleted from the memory space, so that the memory space can record the newly generated metadata, and the utilization rate of the memory space is improved.
According to the technical scheme, when the metadata writing request is received, the metadata carried in the metadata writing request is cached to the preset memory space, and the log file corresponding to the file identification carried in the metadata writing request is searched; and writing the log event corresponding to the metadata into the log file according to the access offset of the additional write file recorded in the log file. When a metadata write-in request is received, the metadata are cached firstly, and then the disk-dropping operation is executed on the cached metadata in the memory space according to the set cycle time, so that the response efficiency of the metadata write-in request is greatly improved. And updating an outdated data file access offset recorded in a log header of the log file after completing the destaging of the metadata. The log event of the metadata is recorded in a log file mode, the metadata can be tracked, even if the metadata in the memory space is lost due to abnormal exit of the metadata service system, the metadata can be recovered according to the recorded log file, and the smooth processing of the metadata request is effectively guaranteed.
In the embodiment of the present invention, the metadata that is not subjected to the disk-drop is cached in the preset memory space, but the memory space has the problem of data loss after the power failure, so when detecting that the system is restarted, the metadata in the memory space needs to be played back, that is, the metadata in the memory space is restored to the state before the power failure.
In practical application, when the metadata service system detects that the system is restarted, the access offset of the expired data file can be read from the log header of the log file; sequentially reading log entries from the log file according to the access offset of the expired data file, generating metadata according to log events in the log entries, and caching the generated metadata to a memory space; and updating the access offset of the expired data file in the log header once every time one log entry is read, and ending the operation until all the log entries in the log file are read.
The log events which are not subjected to the disk-dropping operation can be positioned to the initial positions of the log events depending on the access offset of the expired data file, so that the log events can be sequentially read from the log file, corresponding metadata can be generated depending on the log events, and the generated metadata is cached to a memory space, so that the metadata in the memory space can be recovered.
By setting the access offset of the expired data file in the log header of the log file, even if the metadata in the memory space is lost due to abnormal exit of the metadata service system, the metadata can be recovered from the log events recorded by the log file according to the access offset of the expired data file, so that the reliability of metadata storage is effectively ensured.
In the embodiment of the invention, when the system is restarted, the metadata in the memory space needs to be recovered by depending on the log file, and in order to ensure the correctness of the metadata, a verification mechanism can be set for the log file.
In practical application, a preset check code can be set for the log file when the log file is generated. In the embodiment of the present invention, the form of the check code is not limited. Before reading log entries from the log file in sequence according to the access offset of the expired data file, whether a check code carried in the log file is consistent with a preset check code or not can be judged.
When the check code carried in the log file is consistent with the preset check code, the log file is a file generated aiming at the metadata writing request, and at the moment, the steps of reading log entries from the log file in sequence according to the access offset of the expired data file can be executed.
To further ensure the integrity of the metadata, a corresponding log flag bit is set in the last log event of each log entry, and the log flag bit is used to indicate the start position of the log entry. Therefore, before generating metadata according to the log event in the log entry and caching the generated metadata in the memory space, whether the log flag bit carried in the currently read log entry is consistent with the access offset of the expired data file before updating can be judged.
When the log flag bit carried in the currently read log entry is consistent with the access offset of the expired data file before updating, it indicates that the metadata generated depending on the log event belongs to complete metadata, and at this time, a step of generating metadata according to the log event in the log entry and caching the generated metadata to a memory space may be performed.
When the log flag bit carried in the currently read log entry is inconsistent with the access offset of the expired data file before updating, it indicates that the metadata generated depending on the log event is incomplete, and at this time, alarm prompt information can be generated.
By means of the method, the integrity of the generated metadata is checked depending on the relation between the log flag bit and the access offset of the expired data file, the usability of the generated metadata can be guaranteed, the problem that the generated metadata is incomplete can be found in time, and the influence of the incomplete metadata on a metadata service system is reduced.
Fig. 2 is a schematic structural diagram of a metadata access apparatus according to an embodiment of the present invention, including a cache unit 21, a lookup unit 22, a write unit 23, a drop unit 24, and an update unit 25;
the cache unit 21 is configured to, when a metadata write request is received, cache metadata carried in the metadata write request to a preset memory space;
a searching unit 22, configured to search for a log file corresponding to the file identifier carried in the metadata write request;
a writing unit 23, configured to write a log event corresponding to the metadata into the log file according to an access offset of the additional write file recorded in the log file;
a disk dropping unit 24, configured to perform a disk dropping operation on the metadata cached in the memory space according to a set cycle time;
and an updating unit 25 for updating the outdated data file access offset recorded in the log header of the log file after completing the landed of the metadata.
Optionally, the device further comprises a reading unit and a generating unit;
the reading unit is used for reading the access offset of the expired data file from the log header of the log file when the system restart is detected; sequentially reading log entries from the log file according to the access offset of the expired data file;
the generating unit is used for generating metadata according to the log events in the log entries and caching the generated metadata to a memory space;
the updating unit is further configured to update the outdated data file access offset in the log header once every time one log entry is read, and end the operation until all log entries in the log file are read.
Optionally, the system further comprises a first judging unit;
the first judging unit is used for judging whether the check code carried in the log file is consistent with a preset check code or not; and when the check code carried in the log file is consistent with the preset check code, triggering the reading unit to execute the step of sequentially reading the log entries from the log file according to the access offset of the expired data file.
Optionally, the system further comprises a second judging unit and a prompting unit;
the second judging unit is used for judging whether the log flag bit carried in the currently read log entry is consistent with the access offset of the expired data file before updating; when the log flag bit carried in the currently read log entry is consistent with the access offset of the expired data file before updating, the execution generation unit executes the steps of generating metadata according to the log event in the log entry and caching the generated metadata to a memory space;
and the prompting unit is used for generating alarm prompting information when the log flag bit carried in the currently read log entry is inconsistent with the access offset of the expired data file before updating.
Optionally, a deleting unit is further included;
and the deleting unit is used for deleting the metadata of the completed disk drop from the memory space.
The description of the features in the embodiment corresponding to fig. 2 may refer to the related description of the embodiment corresponding to fig. 1, and is not repeated here.
According to the technical scheme, when the metadata writing request is received, the metadata carried in the metadata writing request is cached to the preset memory space, and the log file corresponding to the file identification carried in the metadata writing request is searched; and writing the log event corresponding to the metadata into the log file according to the access offset of the additional write file recorded in the log file. When a metadata write-in request is received, the metadata are cached firstly, and then the disk-dropping operation is executed on the cached metadata in the memory space according to the set cycle time, so that the response efficiency of the metadata write-in request is greatly improved. And updating an outdated data file access offset recorded in a log header of the log file after completing the destaging of the metadata. The log event of the metadata is recorded in a log file mode, the metadata can be tracked, even if the metadata in the memory space is lost due to abnormal exit of the metadata service system, the metadata can be recovered according to the recorded log file, and the smooth processing of the metadata request is effectively guaranteed.
Fig. 3 is a schematic hardware structure diagram of a metadata access apparatus 30 according to an embodiment of the present invention, including:
a memory 31 for storing a computer program;
a processor 32 for executing a computer program for implementing the steps of the method for accessing metadata as described in any of the embodiments above.
The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the metadata access method according to any of the above embodiments.
The above description details an access method, an access device, and a computer-readable storage medium for a metadata log according to embodiments of the present invention. The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Claims (10)
1. A method for accessing metadata, comprising:
when a metadata writing request is received, caching metadata carried in the metadata writing request to a preset memory space, and searching a log file corresponding to a file identifier carried in the metadata writing request;
writing the log event corresponding to the metadata into the log file according to the access offset of the additional write file recorded in the log file;
and executing a disk-dropping operation on the metadata cached in the memory space according to a set cycle time, and updating the outdated data file access offset recorded in the log header of the log file after completing the disk-dropping of the metadata.
2. The method of claim 1, further comprising, after updating the outdated data file access offset recorded in the log header of the log file after the completion of the destaging of the metadata:
when the system is detected to be restarted, reading an access offset of an expired data file from a log header of the log file;
according to the outdated data file access offset, sequentially reading log entries from the log file, generating metadata according to log events in the log entries, and caching the generated metadata to the memory space;
and updating the access offset of the expired data file in the log header once every time one log entry is read, and ending the operation until all the log entries in the log file are read.
3. The method of claim 2, further comprising, before the sequentially reading the journal entries from the journal file according to the outdated data file access offset, the steps of:
judging whether the check code carried in the log file is consistent with a preset check code or not;
and when the check code carried in the log file is consistent with the preset check code, executing the step of reading log entries from the log file in sequence according to the access offset of the expired data file.
4. The method according to claim 3, wherein before the generating metadata according to the journal event in the journal entry and caching the generated metadata in the memory space, the method further comprises:
judging whether a log flag bit carried in a currently read log entry is consistent with an access offset of an expired data file before updating;
when the log flag bit carried in the currently read log entry is consistent with the access offset of the expired data file before updating, executing the step of generating metadata according to the log event in the log entry and caching the generated metadata to the memory space;
and when the log flag bit carried in the currently read log entry is inconsistent with the access offset of the expired data file before updating, generating alarm prompt information.
5. The method according to any of claims 1-4, further comprising after updating the outdated data file access offset recorded in the log header of the log file after completing the landing of the metadata:
and deleting the metadata of the completion of the disk drop from the memory space.
6. The access device of metadata is characterized by comprising a cache unit, a search unit, a write-in unit, a disk-dropping unit and an updating unit;
the cache unit is used for caching the metadata carried in the metadata writing request to a preset memory space when the metadata writing request is received;
the searching unit is used for searching the log file corresponding to the file identifier carried in the metadata writing request;
the writing unit is configured to write the log event corresponding to the metadata into the log file according to an additional write file access offset recorded in the log file;
the disk dropping unit is used for executing disk dropping operation on the metadata cached in the memory space according to a set cycle time;
and the updating unit is used for updating the access offset of the expired data file recorded in the log header of the log file after the metadata is completely landed.
7. The metadata access device according to claim 6, further comprising a reading unit and a generating unit;
the reading unit is used for reading an access offset of an expired data file from a log header of the log file when the system restart is detected; sequentially reading log entries from the log file according to the access offset of the expired data file;
the generating unit is used for generating metadata according to the log events in the log entries and caching the generated metadata to the memory space;
the updating unit is further configured to update the outdated data file access offset in the log header once every time a log entry is read, and end the operation until all log entries in the log file are read.
8. The metadata access device according to claim 7, further comprising a first judgment unit;
the first judging unit is used for judging whether the check code carried in the log file is consistent with a preset check code or not; and when the check code carried in the log file is consistent with the preset check code, triggering the reading unit to execute the step of sequentially reading the log entries from the log file according to the overdue data file access offset.
9. An apparatus for accessing metadata, comprising:
a memory for storing a computer program;
a processor for executing said computer program to implement the steps of the method for accessing metadata according to any of claims 1 to 5.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method for accessing metadata according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010791402.3A CN112000623A (en) | 2020-08-07 | 2020-08-07 | Metadata access method and device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010791402.3A CN112000623A (en) | 2020-08-07 | 2020-08-07 | Metadata access method and device and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112000623A true CN112000623A (en) | 2020-11-27 |
Family
ID=73462925
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010791402.3A Withdrawn CN112000623A (en) | 2020-08-07 | 2020-08-07 | Metadata access method and device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112000623A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113806139A (en) * | 2021-06-15 | 2021-12-17 | 荣耀终端有限公司 | Operating system recovery method, operating system recovery device, storage medium and computer program product |
| CN113835643A (en) * | 2021-11-23 | 2021-12-24 | 苏州浪潮智能科技有限公司 | Data storage method and device, electronic equipment and readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103902479A (en) * | 2014-03-27 | 2014-07-02 | 浪潮电子信息产业股份有限公司 | Quick reconstruction mechanism for metadata cache on basis of metadata log |
| CN108427728A (en) * | 2018-02-13 | 2018-08-21 | 百度在线网络技术（北京）有限公司 | Management method, equipment and the computer-readable medium of metadata |
| CN108932158A (en) * | 2018-06-28 | 2018-12-04 | 郑州云海信息技术有限公司 | A kind of method, apparatus and MDS server of meta data server issued transaction |
| CN110046132A (en) * | 2019-04-15 | 2019-07-23 | 苏州浪潮智能科技有限公司 | A kind of metadata request processing method, device, equipment and readable storage medium storing program for executing |
| CN110647511A (en) * | 2019-09-27 | 2020-01-03 | 掌阅科技股份有限公司 | Data synchronization method, computing device and computer storage medium |
| CN111104254A (en) * | 2019-11-29 | 2020-05-05 | 北京浪潮数据技术有限公司 | Storage system data flashing method, device, equipment and readable storage medium |
-
2020
- 2020-08-07 CN CN202010791402.3A patent/CN112000623A/en not_active Withdrawn
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103902479A (en) * | 2014-03-27 | 2014-07-02 | 浪潮电子信息产业股份有限公司 | Quick reconstruction mechanism for metadata cache on basis of metadata log |
| CN108427728A (en) * | 2018-02-13 | 2018-08-21 | 百度在线网络技术（北京）有限公司 | Management method, equipment and the computer-readable medium of metadata |
| CN108932158A (en) * | 2018-06-28 | 2018-12-04 | 郑州云海信息技术有限公司 | A kind of method, apparatus and MDS server of meta data server issued transaction |
| CN110046132A (en) * | 2019-04-15 | 2019-07-23 | 苏州浪潮智能科技有限公司 | A kind of metadata request processing method, device, equipment and readable storage medium storing program for executing |
| CN110647511A (en) * | 2019-09-27 | 2020-01-03 | 掌阅科技股份有限公司 | Data synchronization method, computing device and computer storage medium |
| CN111104254A (en) * | 2019-11-29 | 2020-05-05 | 北京浪潮数据技术有限公司 | Storage system data flashing method, device, equipment and readable storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113806139A (en) * | 2021-06-15 | 2021-12-17 | 荣耀终端有限公司 | Operating system recovery method, operating system recovery device, storage medium and computer program product |
| CN113835643A (en) * | 2021-11-23 | 2021-12-24 | 苏州浪潮智能科技有限公司 | Data storage method and device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9183236B2 (en) | Low level object version tracking using non-volatile memory write generations | |
| US8612722B2 (en) | Determining an end of valid log in a log of write records | |
| US20150213100A1 (en) | Data synchronization method and system | |
| US9298625B2 (en) | Read and write requests to partially cached files | |
| JP4249719B2 (en) | Backup system, program, and backup method | |
| CN109542682B (en) | Data backup method, device, equipment and storage medium | |
| US6539402B1 (en) | Using periodic spaces of block ID to improve additional recovery | |
| CN109189608A (en) | A kind of method guaranteeing duplication transaction consistency and corresponding reproducing unit | |
| JP5012628B2 (en) | Memory database, memory database system, and memory database update method | |
| CN109241004B (en) | Metadata file size recovery method, system, device and readable storage medium | |
| JP2007188497A (en) | System and method for managing log information for transaction | |
| CN113596010B (en) | Data processing method, device, node equipment and computer storage medium | |
| CN112000623A (en) | Metadata access method and device and computer readable storage medium | |
| CN115145697A (en) | Database transaction processing method and device and electronic equipment | |
| JP5719083B2 (en) | Database apparatus, program, and data processing method | |
| CN111694806B (en) | Method, device, equipment and storage medium for caching transaction log | |
| CN114138192A (en) | Storage node online upgrading method, device, system and storage medium | |
| CN110866068B (en) | Advertisement data storage method and device based on HDFS | |
| CN111858173A (en) | Data recovery method, device, equipment and medium | |
| CN115905115A (en) | File storage method, file reading method and device, electronic equipment and storage medium | |
| CN113791934A (en) | Data recovery method, computing device and storage medium | |
| CN111861613B (en) | Electronic lock operation recording method and device | |
| JPH0816881B2 (en) | Database update method | |
| CN116483284B (en) | Method, device, medium and electronic equipment for reading and writing virtual hard disk | |
| CN115329391B (en) | Text database protection method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20201127 |
|
| WW01 | Invention patent application withdrawn after publication |