CN109614045B - Metadata dropping method and device and related equipment - Google Patents
Metadata dropping method and device and related equipment Download PDFInfo
- Publication number
- CN109614045B CN109614045B CN201811488967.3A CN201811488967A CN109614045B CN 109614045 B CN109614045 B CN 109614045B CN 201811488967 A CN201811488967 A CN 201811488967A CN 109614045 B CN109614045 B CN 109614045B
- Authority
- CN
- China
- Prior art keywords
- metadata
- disk
- memory
- pressure value
- merged
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000004590 computer program Methods 0.000 claims description 11
- 238000012544 monitoring process Methods 0.000 claims description 6
- 230000009286 beneficial effect Effects 0.000 abstract description 3
- 238000012545 processing Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000011010 flushing procedure Methods 0.000 description 5
- 230000001960 triggered effect Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a metadata tray-dropping method which comprises the steps of issuing a pressure acquisition instruction to a pressure acquisition device according to an acquired data tray-dropping instruction; receiving a system pressure value returned by the pressure acquisition device; judging whether the system pressure value exceeds a preset pressure value or not; if the pressure value does not exceed the preset pressure value, reading the disk metadata, combining the disk metadata and the memory metadata in the memory, and sending the combined metadata to the disk to complete the disk dropping; if the pressure value exceeds the preset pressure value, the memory metadata are sent to the disk, and the disk metadata and the memory metadata are merged in the disk to obtain merged metadata so as to complete the disk dropping; the metadata falling method can effectively improve the stability of the storage system and the reliability of the metadata on the premise of ensuring the metadata falling efficiency. The application also discloses a metadata tray dropping device, equipment, a system and a computer readable storage medium, which all have the beneficial effects.
Description
Technical Field
The present application relates to the field of storage technologies, and in particular, to a metadata destaging method, and further, to a metadata destaging apparatus, device, system, and computer-readable storage medium.
Background
Metadata is data describing data, mainly refers to information describing data attributes, and is used for supporting functions such as indicating storage locations, history data, resource searching, file recording and the like. Metadata landing refers to writing metadata onto a disk in a persistent form, so that data positioning can be performed quickly when a host computer performs data query, and data acquisition and related operations are realized.
When a host issues an IO (input/output) request, metadata is first stored in a memory, but due to the limitation of the memory of the storage system and the problem of data loss possibly caused by abnormal power failure of the storage system when the metadata resides in the memory, a disk-down is triggered when the metadata reaches a certain threshold in the memory, and the metadata in the memory is then printed onto a disk.
Conventional metadata flushing can generally be divided into two cases. One is that the metadata in the memory is directly printed on the disk, and the printed metadata is directly merged with the metadata in the disk, however, since the metadata is generally stored in the disk in the form of B + Tree, when the pressure of the original data in the disk is large, many complex operations of B + Tree, such as data splitting, etc., are easily generated when the amount of the printed metadata is too large; in addition, the B + Tree can be locked in the metadata refreshing process, normal IO read-write is seriously influenced, and IO efficiency is reduced. Another method for flushing metadata is that when the amount of metadata in the memory reaches a flushing threshold, an intermediate memory block is temporarily generated to merge the metadata in the memory and the metadata in the disk, and then the metadata is flushed to the disk at the same time, however, since the intermediate memory block is actually established in the memory, the flushing method occupies a larger memory space; meanwhile, when the system needs a large pressure for data processing, or when the number of metadata to be merged on the disk is large and the flushing speed is high, the situation of insufficient memory may occur, which may result in failure to normally respond to the host IO, thereby reducing the stability and reliability of the storage system.
Therefore, a need exists in the art for a method for providing a metadata destaging method that can effectively improve the stability of a storage system and the reliability of metadata thereof while ensuring the metadata destaging efficiency.
Disclosure of Invention
The metadata falling method can effectively improve the stability of a storage system and the reliability of metadata thereof on the premise of ensuring the metadata falling efficiency; it is another object of the present application to provide a metadata landing device, apparatus, system, and computer-readable storage medium, which also have the above-mentioned advantages.
In order to solve the technical problem, the present application provides a metadata destaging method, including:
issuing a pressure acquisition instruction to a pressure acquisition device according to the acquired data tray falling instruction;
receiving a system pressure value returned by the pressure acquisition device;
judging whether the system pressure value exceeds a preset pressure value or not;
if the pressure value does not exceed the preset pressure value, reading the disk metadata, merging the disk metadata and the memory metadata in the memory, and sending the merged metadata to the disk to complete the disk drop;
and if the pressure value exceeds the preset pressure value, sending the memory metadata to the disk, and merging the disk metadata and the memory metadata in the disk to obtain the merged metadata so as to complete the disk drop.
Preferably, before issuing the pressure acquisition command to the pressure acquisition device according to the acquired data landing command, the method further includes:
monitoring the quantity of the memory metadata in real time;
and when the number reaches a preset threshold value, triggering the data tray falling instruction.
Preferably, the merging the disk metadata and the memory metadata in the memory includes:
creating an intermediate memory block in the memory;
and merging the disk metadata and the memory metadata in the intermediate memory block.
Preferably, after sending the merged metadata to the disk to complete the disk dropping, the method further includes:
and deleting the intermediate memory block.
Preferably, the merged metadata and the form of the B + tree are stored in the disk.
In order to solve the above technical problem, the present application provides a metadata falling device, including:
the command issuing module is used for issuing a pressure acquisition command to the pressure acquisition device according to the acquired data tray-falling command;
the pressure acquisition module is used for receiving a system pressure value returned by the pressure acquisition device;
the pressure judging module is used for judging whether the system pressure value exceeds a preset pressure value or not;
the first disk dropping module is used for reading the disk metadata if the system pressure value does not exceed the preset pressure value, combining the disk metadata and the memory metadata in the memory, and sending the combined metadata to the disk to complete disk dropping;
and the second tray falling module is used for sending the memory metadata to the magnetic disc if the system pressure value exceeds the preset pressure value, merging the magnetic disc metadata and the memory metadata in the magnetic disc to obtain the merged metadata so as to finish the tray falling.
Preferably, the metadata dropping device further includes:
the instruction triggering module is used for monitoring the quantity of the memory metadata in real time; and when the number reaches a preset threshold value, triggering the data tray falling instruction.
In order to solve the above technical problem, the present application provides a metadata landing device, including:
a memory for storing a computer program;
and the processor is used for realizing the steps of any one metadata disk-dropping method when the computer program is executed.
In order to solve the above technical problem, the present application provides a metadata destaging system, including:
the data falling device is used for issuing a pressure acquisition instruction to a pressure acquisition device when the data falling instruction is received, receiving a system pressure value returned by the pressure acquisition device, and judging whether the system pressure value exceeds a preset pressure value; if not, reading the disk metadata, merging the disk metadata and the memory metadata in the memory, and sending the merged metadata to the disk to complete the disk drop; if so, sending the memory metadata to the disk, and merging the disk metadata and the memory metadata in the disk to obtain the merged metadata so as to complete the disk drop;
the pressure acquisition device is used for acquiring the system pressure value according to the pressure acquisition instruction;
the memory is used for storing the memory metadata;
and the disk is used for storing the disk metadata and the merged metadata.
In order to solve the above technical problem, the present application provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any one of the above metadata destaging methods.
The metadata tray-falling method comprises the steps of issuing a pressure acquisition instruction to a pressure acquisition device according to an acquired data tray-falling instruction; receiving a system pressure value returned by the pressure acquisition device; judging whether the system pressure value exceeds a preset pressure value or not; if the pressure value does not exceed the preset pressure value, reading the disk metadata, merging the disk metadata and the memory metadata in the memory, and sending the merged metadata to the disk to complete the disk drop; and if the pressure value exceeds the preset pressure value, sending the memory metadata to the disk, and merging the disk metadata and the memory metadata in the disk to obtain the merged metadata so as to complete the disk drop.
Therefore, the metadata landing method provided by the application can determine the metadata landing method according to the pressure born by the current execution data processing operation of the system, when the system pressure is lower, the memory space is sufficient, the occupancy rate is lower, and a method of merging the metadata in the memory and then landing can be adopted, so that the problems of data splitting and the like possibly caused by the large amount of data needed to be merged by the disk can be avoided while the normal data processing efficiency is not influenced; when the system pressure is large, the memory space is insufficient, in order to not influence the normal data processing operation of the system, a method of first falling the disk and then merging the disk can be adopted, and the data merging process is carried out in the disk, so that the memory space is not occupied, and the normal operation of the data processing operation in the system is effectively ensured.
The metadata disk-dropping device, the equipment, the system and the computer-readable storage medium provided by the application all have the beneficial effects, and are not described again.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a schematic flowchart of a metadata destaging method provided in the present application;
FIG. 2 is a schematic structural diagram of a metadata dropping device provided in the present application;
FIG. 3 is a schematic structural diagram of a metadata drop device provided in the present application;
fig. 4 is a schematic structural diagram of a metadata disk-dropping system provided in the present application.
Detailed Description
The core of the application is to provide a metadata disk-dropping method, which can effectively improve the stability of a storage system and the reliability of metadata thereof on the premise of ensuring the metadata disk-dropping efficiency; at the other core of the present application, a metadata landing device, an apparatus, a system and a computer readable storage medium are provided, which also have the above beneficial effects.
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, fig. 1 is a schematic flow chart of a metadata destaging method provided in the present application, where the metadata destaging method may include:
s100: issuing a pressure acquisition instruction to a pressure acquisition device according to the acquired data tray falling instruction;
specifically, according to the technical scheme provided by the application, the metadata falling method is determined according to different system pressures, so that when the processor acquires the data falling instruction, a pressure acquisition instruction can be further generated, and the pressure acquisition instruction is sent to the corresponding pressure acquisition device so as to acquire the system pressure value.
The data tray-dropping instruction acquisition method is not unique, can be directly issued and acquired by a user based on a corresponding client, and can also be automatically triggered based on some preset trigger conditions, namely when the trigger conditions are met, the data tray-dropping instruction is automatically responded to. Therefore, the present application does not specifically limit the manner of obtaining the same. In addition, for the specific types of the pressure acquisition devices, the pressure acquisition devices may be set differently by users based on actual needs, for example, the pressure acquisition devices are used for acquiring relevant pressure information such as IO operation request pressure, memory occupancy, disk occupancy, and the like, and similarly, the types and the number of the pressure acquisition devices are not specifically limited in this application.
As a preferred embodiment, before issuing the pressure acquisition command to the pressure acquisition device according to the obtained data landing command, the method may further include monitoring the amount of the memory metadata in real time; and when the number reaches a preset threshold value, triggering a data tray falling instruction.
The application provides a method for acquiring a more specific data tray-dropping instruction. Specifically, the memory is also used for realizing data storage, but the storage space is small, and the data loss caused by abnormal power failure is easy to happen, so that the quantity of the metadata in the memory can be monitored and counted in real time for ensuring the data security, and when the quantity reaches a certain quantity, namely the preset threshold value, a data tray-dropping instruction is automatically triggered, and the subsequent data tray-dropping process is further realized.
S200: receiving a system pressure value returned by the pressure acquisition device;
specifically, after the pressure acquisition instruction is issued, the pressure acquisition device can acquire the corresponding system pressure value, the processor acquires the system pressure value, namely the pressure acquisition device feeds the system pressure value back to the processor after acquiring the system pressure value.
S300: judging whether the system pressure value exceeds a preset pressure value or not;
the technical scheme provided by the application is a method for determining metadata tray falling according to different system pressures, so that the system pressure value can be judged to determine the current system pressure, and at the moment, a threshold value, namely the preset pressure value, needs to be preset to realize the judgment of the current system pressure. Specifically, whether the system pressure value acquired by the pressure acquisition device exceeds a preset pressure value or not can be judged, if not, the current pressure of the system can be considered to be small, and if the system pressure value exceeds the preset pressure value, the current pressure of the system can be considered to be large.
The specific value of the preset pressure value may be set according to the actual characteristics of the system, or may be set according to the actual needs of the user, which is not specifically limited in the present application, for example, the specific value may be a middle value, that is, 50%.
S400: if the pressure value does not exceed the preset pressure value, reading the disk metadata, combining the disk metadata and the memory metadata in the memory, and sending the combined metadata to the disk to complete the disk dropping;
specifically, when the system pressure value does not exceed the preset pressure value, that is, the system pressure is small, it indicates that the data processing operation currently performed by the system is small, and the memory space is sufficient, so that the metadata landing can be completed by a method of performing data merging in the memory. Specifically, data can be read from a disk, disk metadata can be obtained and sent to a memory, the memory metadata corresponding to the data landing instruction and the disk metadata are merged in the memory, and merged metadata can be obtained, so that the merged metadata can be sent to the disk, and the data landing can be completed.
The method for merging metadata in the memory may be any one of the prior art, and the present application is not limited specifically.
Preferably, the merging of the disk metadata and the memory metadata in the memory may include creating an intermediate memory block in the memory; and merging the disk metadata and the memory metadata in the intermediate memory block.
The application provides a more specific metadata merging method, in order to avoid mutual influence between a data merging process and other data processing operation processes based on a memory, after it is determined that metadata is to be merged in the memory, an intermediate memory block can be established in the memory, the intermediate memory block is an independent memory space in the memory and is dedicated for performing metadata merging operation, further, for the disk metadata read from a disk, the disk metadata can be directly stored in the intermediate memory block after the disk metadata is read, and meanwhile, the memory metadata required to be landed is stored in the intermediate memory block, so that the metadata can be merged in the intermediate memory block.
Preferably, the sending the merged metadata to the disk to complete the disk crash may further include deleting the intermediate memory block.
This step is intended to implement the delete operation of the intermediate memory block. Specifically, in order to avoid the waste of the memory space, after the metadata is merged and sent to the disk, the intermediate memory block may be deleted, so as to save the memory space and execute other data processing operations.
S500: and if the pressure value exceeds the preset pressure value, sending the memory metadata to the disk, combining the disk metadata and the memory metadata in the disk to obtain combined metadata so as to complete the disk dropping.
Specifically, when the system pressure value exceeds the preset pressure value, that is, the system pressure is higher, it indicates that the data processing operation currently performed by the system is more, and the memory space is insufficient, so that the metadata landing can be completed by a method of first landing and then performing data merging in the disk. Specifically, the metadata corresponding to the data-dropping instruction in the memory, that is, the memory metadata, may be directly sent to the disk, and merged with the disk metadata in the disk to obtain the merged metadata, thereby completing the data-dropping.
Similarly, any one of the prior arts can be adopted for the above method for merging metadata in a memory, and the present application is not limited specifically.
Preferably, the merged metadata and the B + tree are stored in the disk.
Specifically, in order to facilitate the host to quickly complete data location when performing data query, the data after the disk is dropped, that is, the merged metadata is stored in the disk in the form of a B + tree. Specifically, the B + tree is a balanced search tree designed for a disk or other direct access auxiliary device, in the B + tree, all record nodes are stored in leaf nodes of the same layer according to the size sequence of key values, and pointers of the leaf nodes are connected, so that the storage method can effectively improve the data retrieval efficiency.
Of course, the above storage form is only one implementation manner provided by the present application, and is not unique, and may also be a B-tree, a binary tree, a balanced binary tree, or the like, which is not limited in the present application.
The metadata disk-dropping method provided by the application can determine the metadata disk-dropping method according to the pressure born by the current execution data processing operation of the system, when the system pressure is lower, the memory space is sufficient, the occupancy rate is lower, and a method of merging the metadata in the memory and then dropping the disk can be adopted, so that the problems of data splitting and the like possibly caused by the larger data volume needed to be merged by the disk can be avoided while the normal data processing efficiency is not influenced; when the pressure of the system is large, the memory space is insufficient, so that the normal data processing operation of the system is not influenced, a method of firstly falling the disk and then merging the disk can be adopted, the data merging process is carried out in the disk, the memory space does not need to be occupied, the normal operation of the data processing operation in the system is effectively ensured, the data merging process is carried out in the disk, the memory space does not need to be occupied, and the normal operation of the data processing operation in the system is effectively ensured.
To solve the above problem, please refer to fig. 2, fig. 2 is a schematic structural diagram of a metadata tray falling apparatus provided in the present application, where the metadata tray falling apparatus may include:
the instruction issuing module 1 is used for issuing a pressure acquisition instruction to the pressure acquisition device according to the acquired data tray-falling instruction;
the pressure acquisition module 2 is used for receiving a system pressure value returned by the pressure acquisition device;
the pressure judging module 3 is used for judging whether the system pressure value exceeds a preset pressure value or not;
the first disk dropping module 4 is configured to read the disk metadata if the system pressure value does not exceed the preset pressure value, merge the disk metadata and the memory metadata in the memory, and send the merged metadata to the disk to complete disk dropping;
and the second disk dropping module 5 is configured to send the memory metadata to the disk if the system pressure value exceeds the preset pressure value, merge the disk metadata and the memory metadata in the disk, and obtain merged metadata to complete disk dropping.
As a preferred embodiment, the metadata landing device may further include:
the instruction triggering module is used for monitoring the quantity of the memory metadata in real time; and when the number reaches a preset threshold value, triggering a data tray falling instruction.
As a preferred embodiment, the first tray module 4 may include:
an intermediate memory block creating unit, configured to create an intermediate memory block in the memory;
and the metadata merging unit is used for merging the disk metadata and the memory metadata in the intermediate memory block.
As a preferred embodiment, the first landing tray module 4 may further include:
and an intermediate memory block deleting unit configured to delete the intermediate memory block.
For the introduction of the apparatus provided in the present application, please refer to the above method embodiments, which are not described herein again.
To solve the above problem, please refer to fig. 3, fig. 3 is a schematic structural diagram of a metadata tray falling apparatus provided in the present application, where the metadata tray falling apparatus may include:
a memory 11 for storing a computer program;
a processor 12 for implementing the following steps when executing the computer program:
issuing a pressure acquisition instruction to a pressure acquisition device according to the acquired data tray falling instruction; receiving a system pressure value returned by the pressure acquisition device; judging whether the system pressure value exceeds a preset pressure value or not; if the pressure value does not exceed the preset pressure value, reading the disk metadata, combining the disk metadata and the memory metadata in the memory, and sending the combined metadata to the disk to complete the disk dropping; and if the pressure value exceeds the preset pressure value, sending the memory metadata to the disk, combining the disk metadata and the memory metadata in the disk to obtain combined metadata so as to complete the disk dropping.
For the introduction of the device provided in the present application, please refer to the above method embodiment, which is not described herein again.
To solve the above problem, please refer to fig. 4, fig. 4 is a schematic structural diagram of a metadata tray system provided in the present application, where the metadata tray system may include:
the data falling device 10 is configured to, when receiving a data falling instruction, issue a pressure acquisition instruction to the pressure acquisition device, receive a system pressure value returned by the pressure acquisition device 20, and determine whether the system pressure value exceeds a preset pressure value; if not, reading the disk metadata, merging the disk metadata and the memory metadata in the memory 30, and sending the merged metadata to the disk 40 to complete the disk drop; if yes, sending the memory metadata to the disk 40, and merging the disk metadata and the memory metadata in the disk 40 to obtain merged metadata so as to complete the disk drop;
the pressure acquisition device 20 is used for acquiring a system pressure value according to a pressure acquisition instruction;
a memory 30 for storing memory metadata;
and a disk 40 for storing disk metadata and merged metadata.
For the introduction of the system provided by the present application, please refer to the above method embodiment, which is not described herein again.
To solve the above problem, the present application further provides a computer-readable storage medium having a computer program stored thereon, where the computer program when executed by a processor can implement the following steps:
issuing a pressure acquisition instruction to a pressure acquisition device according to the acquired data tray falling instruction; receiving a system pressure value returned by the pressure acquisition device; judging whether the system pressure value exceeds a preset pressure value or not; if the pressure value does not exceed the preset pressure value, reading the disk metadata, combining the disk metadata and the memory metadata in the memory, and sending the combined metadata to the disk to complete the disk dropping; and if the pressure value exceeds the preset pressure value, sending the memory metadata to the disk, combining the disk metadata and the memory metadata in the disk to obtain combined metadata so as to complete the disk dropping.
The computer-readable storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
For the introduction of the computer-readable storage medium provided in the present application, please refer to the above method embodiments, which are not described herein again.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The metadata landing method, apparatus, system, device and computer readable storage medium provided by the present application are described in detail above. The principles and embodiments of the present application are explained herein using specific examples, which are provided only to help understand the method and the core idea of the present application. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and these improvements and modifications also fall into the elements of the protection scope of the claims of the present application.
Claims (10)
1. A metadata destaging method, comprising:
issuing a pressure acquisition instruction to a pressure acquisition device according to the acquired data tray falling instruction;
receiving a system pressure value returned by the pressure acquisition device;
judging whether the system pressure value exceeds a preset pressure value or not;
if the pressure value does not exceed the preset pressure value, reading the disk metadata, merging the disk metadata and the memory metadata in the memory, and sending the merged metadata to the disk to complete the disk drop;
and if the pressure value exceeds the preset pressure value, sending the memory metadata to the disk, and merging the disk metadata and the memory metadata in the disk to obtain the merged metadata so as to complete the disk drop.
2. The metadata landing method according to claim 1, wherein before issuing the pressure acquisition command to the pressure acquisition device according to the acquired data landing command, the method further comprises:
monitoring the quantity of the memory metadata in real time;
and when the number reaches a preset threshold value, triggering the data tray falling instruction.
3. The metadata destaging method according to claim 1, wherein the merging the disk metadata and the memory metadata in the memory comprises:
creating an intermediate memory block in the memory;
and merging the disk metadata and the memory metadata in the intermediate memory block.
4. The metadata destaging method according to claim 3, wherein after sending the merged metadata to the disk to complete the destaging, further comprising:
and deleting the intermediate memory block.
5. The method of any one of claims 1 to 4, wherein the merged metadata is stored in the disk in a form of a B + tree.
6. A metadata dropoff device, comprising:
the command issuing module is used for issuing a pressure acquisition command to the pressure acquisition device according to the acquired data tray-falling command;
the pressure acquisition module is used for receiving a system pressure value returned by the pressure acquisition device;
the pressure judging module is used for judging whether the system pressure value exceeds a preset pressure value or not;
the first disk dropping module is used for reading the disk metadata if the system pressure value does not exceed the preset pressure value, combining the disk metadata and the memory metadata in the memory, and sending the combined metadata to the disk to complete disk dropping;
and the second tray falling module is used for sending the memory metadata to the magnetic disc if the system pressure value exceeds the preset pressure value, merging the magnetic disc metadata and the memory metadata in the magnetic disc to obtain the merged metadata so as to finish the tray falling.
7. The metadata dropoff device of claim 6, further comprising:
the instruction triggering module is used for monitoring the quantity of the memory metadata in real time; and when the number reaches a preset threshold value, triggering the data tray falling instruction.
8. A metadata dropoff device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the metadata landing method according to any one of claims 1 to 5 when executing the computer program.
9. A metadata dropdown system, comprising:
the metadata tray falling device according to claim 8, configured to issue a pressure acquisition command to a pressure acquisition device when receiving a data tray falling command, receive a system pressure value returned by the pressure acquisition device, and determine whether the system pressure value exceeds a preset pressure value; if not, reading the disk metadata, merging the disk metadata and the memory metadata in the memory, and sending the merged metadata to the disk to complete the disk drop; if so, sending the memory metadata to the disk, and merging the disk metadata and the memory metadata in the disk to obtain the merged metadata so as to complete the disk drop;
the pressure acquisition device is used for acquiring the system pressure value according to the pressure acquisition instruction;
the memory is used for storing the memory metadata;
and the disk is used for storing the disk metadata and the merged metadata.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of a metadata landing method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811488967.3A CN109614045B (en) | 2018-12-06 | 2018-12-06 | Metadata dropping method and device and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811488967.3A CN109614045B (en) | 2018-12-06 | 2018-12-06 | Metadata dropping method and device and related equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109614045A CN109614045A (en) | 2019-04-12 |
| CN109614045B true CN109614045B (en) | 2022-04-15 |
Family
ID=66007470
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811488967.3A Active CN109614045B (en) | 2018-12-06 | 2018-12-06 | Metadata dropping method and device and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109614045B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110147204B (en) * | 2019-05-22 | 2020-03-10 | 苏州浪潮智能科技有限公司 | Metadata disk-dropping method, device and system and computer-readable storage medium |
| CN110134340B (en) * | 2019-05-23 | 2020-03-06 | 苏州浪潮智能科技有限公司 | Method, device, equipment and storage medium for updating metadata |
| CN111158594A (en) * | 2019-12-27 | 2020-05-15 | 浪潮电子信息产业股份有限公司 | Metadata disk-dropping method, device, equipment and storage medium |
| CN111984554B (en) * | 2020-07-31 | 2023-03-28 | 新华三技术有限公司成都分公司 | Data processing method and device |
| CN113672523A (en) * | 2021-07-09 | 2021-11-19 | 济南浪潮数据技术有限公司 | Cache pool data flushing method and system, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104809237A (en) * | 2015-05-12 | 2015-07-29 | 百度在线网络技术（北京）有限公司 | LSM-tree (The Log-Structured Merge-Tree) index optimization method and LSM-tree index optimization system |
| CN105183384A (en) * | 2015-09-10 | 2015-12-23 | 浪潮(北京)电子信息产业有限公司 | Direct erasure correction implementation method and device |
| CN105511802A (en) * | 2015-11-24 | 2016-04-20 | 北京达沃时代科技有限公司 | Buffer memory writing method and apparatus and synchronizing method and apparatus for disk cache region |
| CN107608911A (en) * | 2017-09-12 | 2018-01-19 | 郑州云海信息技术有限公司 | One kind is data cached to write with a brush dipped in Chinese ink method, apparatus, equipment and storage medium |
| CN108647151A (en) * | 2018-04-26 | 2018-10-12 | 郑州云海信息技术有限公司 | It is a kind of to dodge system metadata rule method, apparatus, equipment and storage medium entirely |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9201800B2 (en) * | 2013-07-08 | 2015-12-01 | Dell Products L.P. | Restoring temporal locality in global and local deduplication storage systems |
| US9423973B2 (en) * | 2014-11-19 | 2016-08-23 | International Business Machines Corporation | Asynchronous tape backup and restore from tape backup in a disk storage environment |
-
2018
- 2018-12-06 CN CN201811488967.3A patent/CN109614045B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104809237A (en) * | 2015-05-12 | 2015-07-29 | 百度在线网络技术（北京）有限公司 | LSM-tree (The Log-Structured Merge-Tree) index optimization method and LSM-tree index optimization system |
| CN105183384A (en) * | 2015-09-10 | 2015-12-23 | 浪潮(北京)电子信息产业有限公司 | Direct erasure correction implementation method and device |
| CN105511802A (en) * | 2015-11-24 | 2016-04-20 | 北京达沃时代科技有限公司 | Buffer memory writing method and apparatus and synchronizing method and apparatus for disk cache region |
| CN107608911A (en) * | 2017-09-12 | 2018-01-19 | 郑州云海信息技术有限公司 | One kind is data cached to write with a brush dipped in Chinese ink method, apparatus, equipment and storage medium |
| CN108647151A (en) * | 2018-04-26 | 2018-10-12 | 郑州云海信息技术有限公司 | It is a kind of to dodge system metadata rule method, apparatus, equipment and storage medium entirely |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109614045A (en) | 2019-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109614045B (en) | Metadata dropping method and device and related equipment | |
| CN108647151B (en) | Full flash system metadata disk-dropping method, device, equipment and storage medium | |
| CN107643880B (en) | File data migration method and device based on distributed file system | |
| US7567989B2 (en) | Method and system for data processing with data replication for the same | |
| CN102945278B (en) | A kind of method and apparatus of data-base recording redo log | |
| CN114637475B (en) | Distributed storage system control method and device and readable storage medium | |
| CN109683825B (en) | Storage system online data compression method, device and equipment | |
| CN110888837B (en) | Object storage small file merging method and device | |
| CN104516921A (en) | Automatic response method and device | |
| CN107402870B (en) | Method and device for processing log segment in metadata server | |
| CN115543871B (en) | Data storage method and related equipment | |
| US20140337301A1 (en) | Big data extraction system and method | |
| CN109189343B (en) | Metadata disk-dropping method, device, equipment and computer-readable storage medium | |
| CN110019527B (en) | Slave library reading method, related device and equipment | |
| CN103399714B (en) | The maintenance system of storage device configurations information and using method thereof | |
| CN110879764B (en) | Bitmap setting method, device and equipment and readable storage medium | |
| CN117112522A (en) | Concurrent process log management method, device, equipment and storage medium | |
| CN110658993A (en) | Snapshot rollback method, device, equipment and storage medium | |
| CN106997305B (en) | Transaction processing method and device | |
| CN115185458A (en) | Data writing method and device, storage medium and computer equipment | |
| US20120066181A1 (en) | File removal with no available disk blocks in redirect-on-write file systems | |
| CN105488202B (en) | A kind of distributed file system performance bottleneck localization method, device and system | |
| CN112835967A (en) | Data processing method, device, equipment and medium based on distributed storage system | |
| KR100567813B1 (en) | Transaction Analysing System for Tandem system | |
| WO2022239118A1 (en) | History management device, control method, and computer-readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |