CN110147204A - A kind of metadata rule method, apparatus, system and computer readable storage medium - Google Patents

A kind of metadata rule method, apparatus, system and computer readable storage medium Download PDF

Info

Publication number
CN110147204A
CN110147204A CN201910429310.8A CN201910429310A CN110147204A CN 110147204 A CN110147204 A CN 110147204A CN 201910429310 A CN201910429310 A CN 201910429310A CN 110147204 A CN110147204 A CN 110147204A
Authority
CN
China
Prior art keywords
metadata
tree
memory
leaf node
disk
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910429310.8A
Other languages
Chinese (zh)
Other versions
CN110147204B (en
Inventor
刚亚州
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Wave Intelligent Technology Co Ltd
Original Assignee
Suzhou Wave Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Wave Intelligent Technology Co Ltd filed Critical Suzhou Wave Intelligent Technology Co Ltd
Priority to CN201910429310.8A priority Critical patent/CN110147204B/en
Publication of CN110147204A publication Critical patent/CN110147204A/en
Application granted granted Critical
Publication of CN110147204B publication Critical patent/CN110147204B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/901Indexing; Data structures therefor; Storage structures
    • G06F16/9027Trees
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/907Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061Improving I/O performance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0673Single storage device
    • G06F3/0674Disk device
    • G06F3/0676Magnetic disk device
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0673Single storage device
    • G06F3/0679Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Library & Information Science (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

This application discloses a kind of metadata rule methods to determine the continuous overlay area of memory B+ tree Yu disk B+ tree including inquiring the corresponding disk B+ leaf child node of each metadata in memory B+ the first leaf node of tree;Processing is merged to all metadata in same continuous overlay area, obtains new leaf node;If the first leaf node of memory B+ tree is not to terminate leaf node, the metadata of next leaf node is obtained, and returns to the step that continuous overlay area determines, until traversing termination leaf node;The metadata of each new leaf node is stored to disk, metadata rule is completed;The metadata rule method rule efficiency is higher, and large-scale concurrent random access metadata can be made to reach higher efficiency.Disclosed herein as well is a kind of metadata rule device, system and computer readable storage mediums, all have above-mentioned beneficial effect.

Description

A kind of metadata rule method, apparatus, system and computer readable storage medium
Technical field
This application involves technical field of data storage, in particular to a kind of metadata rule method further relates to a kind of first number According to rule device, system and computer readable storage medium.
Background technique
Metadata refers to the data of description data, not only indicates the information such as type, title, the value of data, also provides number According to contextual information, such as the affiliated domain of data, data source.In data-storage system, metadata is the base of information storage Plinth is the minimum unit of data.In the storage of full flash memory, a large amount of and high concurrent data access and inquiry problem are necessarily involved, Effective management metadata, just can increase access efficiency.Therefore, effective management of metadata, rule save efficiency in full flash memory And to avoid writing amplification most important, and large-scale concurrent random access metadata can be made to reach higher efficiency.
Full flash memory metadata generally carrys out tissue with the data structure of B+ tree, including memory B+ tree and two, disk B+ tree Point, it after the metadata of memory B+ tree in write buffer reaches certain threshold value, needs to be written on disk, be closed with disk B+ tree And new B+ tree is obtained, and then complete rule.However, in the prior art, above-mentioned metadata merging process is inserted by B+ tree Enter operation and realizes that traditional B+ tree insertion operation is to be inserted into one by one, i.e., the metadata in memory B+ tree is inserted into disk B one by one In+tree, when need rule metadata quantity it is larger when, this kind of rule method efficiency is more low;Further, when disk B+ tree When middle metadata quantity is larger, insertion process is then related to a large amount of data search and copy, further reduced metadata and falls Disk efficiency.
Therefore, how to effectively improve metadata rule efficiency is those skilled in the art's urgent problem to be solved.
Summary of the invention
The purpose of the application is to provide a kind of metadata rule method, and the metadata rule method rule efficiency is higher, can Large-scale concurrent random access metadata is set to reach higher efficiency;The another object of the application is to provide a kind of metadata rule Device, system and computer readable storage medium, it may have above-mentioned beneficial effect.
In order to solve the above technical problems, this application provides a kind of metadata rule method, the metadata rule method Include:
When receiving rule instruction, the metadata of the first leaf node of memory B+ tree is obtained;
The corresponding disk B+ leaf child node of each metadata is inquired, determines the memory B+ tree and the magnetic The continuous overlay area of PanB+Shu;
The metadata of metadata and the disk B+ tree to the memory B+ tree in same continuous overlay area Processing is merged, new leaf node is obtained;
Judge whether the first leaf node of the memory B+ tree is that memory B+ tree terminates leaf node;
If it is not, then obtaining the metadata of the next leaf node of memory B+ tree, and return to the inquiry and each member The corresponding disk B+ leaf child node of data determines the step of the continuous overlay area of the memory B+ tree and the disk B+ tree Suddenly, until traversing the memory B+ tree terminates leaf node;
The metadata of each new leaf node is stored to disk, metadata rule is completed.
Preferably, before the metadata for obtaining the first leaf node of memory B+ tree, further includes:
Judge in the disk with the presence or absence of the disk B+ tree;
If it does not exist, then the memory B+ tree is stored to the disk, completes metadata rule;
If it exists, then the step of executing the metadata for obtaining the first leaf node of memory B+ tree.
Preferably, the corresponding disk B+ leaf of metadata of the memory B+ tree in the continuous overlay area Node is identical or adjacent.
Preferably, the metadata of the described pair of memory B+ tree in the continuous overlay area and the disk B+ The metadata of tree merges processing, obtains new leaf node, comprising:
By MERGING/SORTING ALGORITHM to the metadata of the memory B+ tree in the continuous overlay area and described The metadata of disk B+ tree merges processing, obtains the new leaf node.
Preferably, before the metadata for obtaining the next leaf node of memory B+ tree, further includes:
By the corresponding disk B+ leaf knot removal of the metadata of the upper leaf node of memory B+ tree.
Preferably, the metadata by each new leaf node is stored to disk, completes metadata rule, packet It includes:
New memory B+ tree is generated by each new leaf node;
The new memory B+ tree is stored to the disk, metadata rule is completed.
Preferably, after the completion metadata rule, further includes:
New memory B+ tree described in memory is deleted, memory release is completed.
In order to solve the above technical problems, present invention also provides a kind of metadata rule device, the metadata rule dress It sets and includes:
Metadata obtains module, for obtaining first number of the first leaf node of memory B+ tree when receiving rule instruction According to;
Continuous overlay area determining module, for looking into the corresponding disk B+ leaf child node of each metadata It askes, determines the continuous overlay area of the memory B+ tree Yu the disk B+ tree;
Metadata merging module, for the memory B+ tree in the same continuous overlay area metadata and institute The metadata for stating disk B+ tree merges processing, obtains new leaf node;
Judgment module is terminated, for judging whether the first leaf node of the memory B+ tree is that memory B+ tree terminates leaf section Point;
Module is looped through, if not being that the memory B+ tree terminates leaf section for the first leaf node of the memory B+ tree Point, then obtain the metadata of the next leaf node of memory B+ tree, and returns to the continuous overlay area determining module, until It traverses the memory B+ tree and terminates leaf node;
Metadata rule module completes metadata for storing the metadata of each new leaf node to disk Rule.
In order to solve the above technical problems, present invention also provides a kind of metadata rule system, metadata rule system System includes:
Memory, for storing computer program;
Processor, the step of any one of the above metadata rule method is realized when for executing the computer program.
In order to solve the above technical problems, the computer can present invention also provides a kind of computer readable storage medium It reads to be stored with computer program on storage medium, the computer program realizes any one of the above member number when being executed by processor The step of according to rule method.
A kind of metadata rule method provided herein, including when receiving rule instruction, obtain memory B+ tree The metadata of first leaf node;The corresponding disk B+ leaf child node of each metadata is inquired, is determined in described Deposit the continuous overlay area of B+ tree Yu the disk B+ tree;To the member of the memory B+ tree in same continuous overlay area Data and the metadata of the disk B+ tree merge processing, obtain new leaf node;Judge the first leaf of the memory B+ tree Whether child node is that memory B+ tree terminates leaf node;If it is not, the metadata of the next leaf node of memory B+ tree is then obtained, And inquiry disk B+ leaf child node corresponding with each metadata is returned, determine the memory B+ tree and the disk The step of continuous overlay area of B+ tree, until traversing the memory B+ tree terminates leaf node;By each new leaf section The metadata of point is stored to disk, completes metadata rule.
As it can be seen that metadata rule method provided herein, based on the characteristic of B+ data tree structure, metadata is all deposited It is stored on the leaf node of B+ tree, and is ordered arrangement, the metadata in each leaf node of memory B+ tree can be seen as a result, Do an ordered sequence, regard the metadata in disk B+ leaf child node as an ordered sequence, at this point, by inquiry and it is true The continuous overlay area of fixed two ordered sequences, to the memory metadata and magnetic disk metadata in same continuous overlay area Processing is merged, to complete the sequence of batch metadata and generate to obtain new leaf node, data is further realized and falls Disk;Compared to the prior art, the problem of can effectively avoid rule inefficiency caused by metadata is inserted into one by one can make extensive Concurrent random access metadata reaches higher efficiency.
A kind of metadata rule device, system and computer readable storage medium provided herein, all has Beneficial effect is stated, details are not described herein.
Detailed description of the invention
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this The embodiment of application for those of ordinary skill in the art without creative efforts, can also basis The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of flow diagram of metadata rule method provided herein;
Fig. 2 is the leaf node comparative diagram of a kind of memory B+ tree and disk B+ tree provided herein;
Fig. 3 is the leaf node comparative diagram of another memory B+ tree and disk B+ tree provided herein;
Fig. 4 is the flow diagram of another metadata rule method provided herein;
Fig. 5 is a kind of structural schematic diagram of metadata rule device provided herein;
Fig. 6 is a kind of structural schematic diagram of metadata rule system provided herein.
Specific embodiment
The core of the application is to provide a kind of metadata rule method, and the metadata rule method rule efficiency is higher, can Large-scale concurrent random access metadata is set to reach higher efficiency;Another core of the application is to provide a kind of metadata rule Device, system and computer readable storage medium, it may have above-mentioned beneficial effect.
To keep the purposes, technical schemes and advantages of the embodiment of the present application clearer, below in conjunction with the embodiment of the present application In attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is Some embodiments of the present application, instead of all the embodiments.Based on the embodiment in the application, those of ordinary skill in the art All other embodiment obtained without making creative work, shall fall in the protection scope of this application.
Referring to FIG. 1, Fig. 1 is a kind of flow diagram of metadata rule method provided herein, the metadata Rule method may include:
S101: when receiving rule instruction, the metadata of the first leaf node of memory B+ tree is obtained;
In general, the quantity of the leaf node in memory B+ tree is not unique, this step aims at memory B+ tree first The acquisition of metadata in leaf node, first leaf node are first leaf node of memory B+ tree.Specifically, first The process of data rule is related to metadata in memory B+ tree and merges with metadata in disk B+ tree, which is caching Memory in execute, therefore, when receive rule instruction when, the first leaf node of memory B+ tree can be read out, acquisition Metadata therein.
Certainly, the characteristic based on B+ data tree structure, the metadata in leaf node is ordered arrangement, therefore, above-mentioned The metadata of memory B+ the first leaf node of tree of acquisition is equivalent to an ordered sequence.
S102: disk B+ leaf child node corresponding to each metadata is inquired, and determines memory B+ tree and disk B+ tree Continuous overlay area;
This step aims at the acquisition of the continuous overlay area of memory B+ tree and disk B+ tree.Due in memory B+ tree Each metadata has its corresponding disk B+ leaf child node, then the metadata in obtaining the first leaf node of memory B+ tree On the basis of, the corresponding disk B+ leaf child node of each metadata in first leaf node is inquired, can be obtained Obtain the continuous overlay area of memory B+ tree and disk B+ tree.
It describes in detail below by citing to above-mentioned continuous overlay area.
Firstly, referring to FIG. 2, Fig. 2 is the leaf node pair of a kind of memory B+ tree and disk B+ tree provided herein According to figure, wherein " node 1 " in memory B+ leaf child node is the first leaf node, including 3 metadata, respectively For key1, key2 and key3;It is determined after being inquired by leaf node, key1 corresponds to first leaf section of disk B+ tree Point, key2 corresponds to second leaf node of disk B+ tree, and key3 corresponds to the 4th leaf node of disk B+ tree, this When, the corresponding leaf node of key3 leaf node corresponding with the first two metadata is discontinuous, then the first leaf of disk B+ tree section Point is a continuous overlay area, the 4th leaf node of disk B+ tree with the region that disk B+ the second leaf node of tree is covered The region covered is a continuous overlay area.
Further, referring to FIG. 3, Fig. 3 is the leaf section of another memory B+ tree and disk B+ tree provided herein Comparative diagram is put, " node 1 " in memory B+ leaf child node is the first leaf node, including 4 metadata, respectively Key a, key b, key c and key d;It is determined after being inquired by leaf node, key a and key b both correspond to disk B+ First leaf node of tree, key c correspond to the third leaf node of disk B+ tree, and key d corresponds to disk B+ tree 4th leaf node, at this point, the corresponding leaf node of key a and key b is continuous, the corresponding leaf node of key c and key d Continuously, and the corresponding leaf node of key b and key c is discontinuous, then the region that the first leaf node of disk B+ tree is covered is The region that one continuous overlay area, the third leaf node of disk B+ tree and the 4th leaf node are covered is a company Continuous overlay area.
It can be seen that the corresponding disk B+ of metadata of the memory B+ tree in continuous overlay area based on above-mentioned elaboration Leaf child node can be identical, be also possible to adjacent.But the implementation is not unique, can also preset Continuous overlay area is same disk B+ leaf child node, the i.e. corresponding disk B+ leaf child node of metadata in memory B+ tree For same leaf node, at this point, the leaf node is continuous overlay area, but this kind of implementation is compared to above-mentioned continuous covering The implementation that the corresponding disk B+ leaf child node of memory B+ tree metadata can be identical or adjacent in region, efficiency It is relatively low, suitable for the situation less to rule metadata.
Therefore, as a kind of preferred embodiment, the application is set at first number of the memory B+ tree in continuous overlay area It is identical or adjacent according to corresponding disk B+ leaf child node.The conjunction of metadata in maximum continuous overlay area is realized as a result, And realize the batch processing of metadata, both effectively ensure that it is lower write amplification, also achieve higher metadata rule effect Rate.
S103: the metadata of metadata and disk B+ tree to the memory B+ tree in same continuous overlay area carries out Merging treatment obtains new leaf node;
The merging treatment that this step aims at metadata and metadata in disk B+ tree in memory B+ tree operates.It is specific and Speech, in same continuous overlay area, the metadata of memory B+ tree is equivalent to an ordered sequence, the metadata phase of disk B+ tree When in an ordered sequence, at this point, the merging process is the ordering by merging process of two ordered sequences.Certainly, for two The ordering by merging process of ordered sequence, can be used any one in prior art, and the application does not limit this.Further, After completing ordering by merging, can be obtained a new complete ordered sequence, the complete ordered sequence constitute one it is new Leaf node.
Since above-mentioned metadata merging process is carried out in the memory of caching, the merging of that identical element data is necessarily involved The acquisition of metadata on disk, that is to say, that the metadata by disk B+ tree is needed to read into memory.By taking Fig. 2 as an example, due to The 4th leaf node that key3 corresponds to disk B+ tree has been inquired, then in data merging process, without to disk B+ Three, the ground leaf node of tree is read out, and avoids reading disk invalid node, while reducing to disk write amplification, is mentioned High metadata combined efficiency.
Preferably, the metadata of above-mentioned metadata and disk B+ tree to the memory B+ tree in the continuous overlay area into Row merging treatment obtains new leaf node, may include by MERGING/SORTING ALGORITHM to the memory B in continuous overlay area + the metadata of tree and the metadata of disk B+ tree merge processing, obtain new leaf node.
Specifically, this application provides a kind of more concrete implementation modes, i.e., for the merging process of above-mentioned metadata It is realized based on MERGING/SORTING ALGORITHM.Merger sequence is built upon a kind of operational effective sort algorithm of merger, specifically will Orderly subsequence merges, and obtains complete ordering sequence, i.e., first makes each subsequence orderly, then makes to have between subsequence section Sequence finally obtains complete orderly sequence.
Merger sequence is a kind of sort algorithm with high efficiency and high stability.For example, have 10 to rule metadata, That is 10 key values may then require to look up 10 positions such as according to traditional insertion method, then do 10 data movements, Efficiency is very low.But if sorting using merger generates new node, do not need to search, a key value only needs to be inserted into once, no It can change again, efficiency is higher.
S104: judge whether the first leaf node of memory B+ tree is that memory B+ tree terminates leaf node;If it is not, then executing S105, if so, executing S106;
Above-mentioned S101~S103 be to operation performed by metadata in memory B+ the first leaf node of tree, to complete it is whole A memory B+ tree merges with disk B+ tree, then needs to execute aforesaid operations to all leaf nodes in memory B+ tree.Therefore, Can determine whether the first leaf node of above-mentioned memory B+ tree is that memory B+ tree terminates leaf node, which is interior Deposit the last one leaf node of B+ tree.
S105: obtaining the metadata of the next leaf node of memory B+ tree, and return to S102, until traversing memory B+ tree eventually Only leaf node;
Specifically, needing to continue under traversal if the first leaf node of above-mentioned memory B+ tree is not to terminate leaf node One leaf node obtains the metadata in the next leaf node of memory B+ tree, and certainly, the acquisition process of the metadata can refer to the The acquisition process of one leaf node metadata, is not repeating herein.As a result, by being held to all leaf nodes in memory B+ tree Above-mentioned S101~S103 the operation of row, can be realized complete the merging of memory B+ tree with disk B+ tree, obtains all new leaves Node.
As a kind of preferred embodiment, before the metadata of the next leaf node of above-mentioned acquisition memory B+ tree, can also wrap It includes: by the corresponding disk B+ leaf knot removal of the metadata of the upper leaf node of memory B+ tree.
Since the merging process of metadata needs the metadata by disk B+ tree to read into memory, however, being based on B+ tree The characteristic of data structure, metadata is ordered into storage wherein, and metadata rule method provided by the present application is with one The leaf node of a memory B+ tree is what unit carried out, it is contemplated that, metadata is corresponding in the upper leaf node of memory B+ tree Disk B+ leaf child node disk B+ leaf child node corresponding with metadata in the next leaf node of memory B+ tree absolutely not It is the same leaf node, and in the next leaf node of memory B+ tree including the corresponding disk B+ leaf child node certainty of metadata It deposits in the upper leaf node of B+ tree after the corresponding disk B+ leaf child node of metadata, therefore, to reduce resources occupation rate, It, can be by each metadata in memory B+ leaf child node after every metadata for completing a memory B+ leaf child node merges Corresponding disk B+ leaf knot removal, to achieve the purpose that releasing memory.
Based on described above, it is contemplated that, when initially carrying out metadata reading to disk B+ tree, without disposable All metadata being queried in disk B+ tree are read into memory, it only need to will be each in memory B+ tree current leaf node The corresponding disk B+ leaf child node of metadata is read into memory, and realizes the swapping in and out of memory, to avoid memory money Source occupies.Further, it for the disk B+ leaf child node read, can also directly be deleted in disk, to reduce The resource occupation of disk;Certainly, which can also hold after all metadata being queried to are read in disk B+ tree Row, the application do not limit this.
S106: the metadata of each new leaf node is stored to disk, completes metadata rule.
This step aims at metadata rule.Specifically, complete memory B+ tree in all leaf nodes metadata Merge with metadata in disk B+ tree, after obtaining all new leaf nodes, can be stored into disk, realizes first number According to rule.Further, for the metadata after rule, further compacting operation can be also executed to it, so that each kv value (member Data) in disk make-up solid matter cloth, to effectively improve disk utilization.
Preferably, the above-mentioned metadata by each new leaf node is stored to disk, is completed metadata rule, be can wrap It includes: new memory B+ tree is generated by each new leaf node;New memory B+ tree is stored to disk, metadata rule is completed.
For the process of above-mentioned metadata rule, this application provides more concrete implementation methods, specifically, caching Memory in, according to the architectural characteristic of memory B+ tree, first can generate new memory B+ tree by all new leaf nodes based on acquisition, have Body realizes process are as follows: generates corresponding non-leaf nodes upwards in turn, until the n omicronn-leaf child node as root node generates, obtains New memory B+ tree, finally again stores the new memory B+ tree into disk, completes metadata rule.
It can also include: to delete new memory in memory after above-mentioned completion metadata rule as a kind of preferred embodiment B+ tree completes memory release.
It, can also further releasing memory after completing metadata rule.Specifically, due to having completed rule, it is interior at this time The new memory B+ tree for depositing middle generation does not use, and is deleted at this time, memory release is realized, so as to first number newly to input According to offer memory space.
Further, it since metadata complete by rule, and is stored in the form of new B+ tree into disk, then, it is former The disk B+ tree of beginning equally no longer has value, and is equally deleted at this time, resource utilization is effectively reduced.
Metadata rule method provided herein, based on the characteristic of B+ data tree structure, metadata is all stored in B On the leaf node of+tree, and it is ordered arrangement, the metadata in each leaf node of memory B+ tree can be regarded as one as a result, A ordered sequence regards the metadata in disk B+ leaf child node as an ordered sequence, at this point, by inquiring and determining two The continuous overlay area of a ordered sequence carries out the memory metadata and magnetic disk metadata that are in same continuous overlay area Merging treatment further realizes data rule to complete the sequence of batch metadata and generate to obtain new leaf node;Phase Compared with the prior art, the problem of can effectively avoid rule inefficiency caused by metadata is inserted into one by one, large-scale concurrent can be made Random access metadata reaches higher efficiency.
On the basis of above-mentioned each embodiment, referring to FIG. 4, Fig. 4 is that another metadata provided herein is fallen The flow diagram of disk method, the metadata rule method may include:
S201: rule instruction is received;
S202: judged in disk according to rule instruction with the presence or absence of disk B+ tree;If so, S203 is executed, if it is not, then holding Row S204;
S203: memory B+ tree is stored to disk, metadata rule is completed;
Specifically, disk B+ tree is not present in itself that is, in disk in the case of disk is empty, at this point, without executing magnetic Memory B+ tree is directly stored to disk, metadata rule can be realized by the union operation of disk metadata and memory metadata.
S204: the metadata of the first leaf node of memory B+ tree is obtained;
S205: disk B+ leaf child node corresponding to each metadata is inquired, and determines memory B+ tree and disk B+ tree Continuous overlay area;
S206: the metadata of metadata and disk B+ tree to the memory B+ tree in same continuous overlay area carries out Merging treatment obtains new leaf node;
S207: judge whether the first leaf node of memory B+ tree is that memory B+ tree terminates leaf node;If it is not, then executing S208, if so, executing S209;
S208: obtaining the metadata of the next leaf node of memory B+ tree, and return to S205, until traversing memory B+ tree eventually Only leaf node;
S209: the metadata of each new leaf node is stored to disk, completes metadata rule.
For above-mentioned S201 and S204~S209 particular content referring to a upper embodiment, the present embodiment is herein not It repeats again.
To solve the above problems, referring to FIG. 5, Fig. 5 is a kind of structure of metadata rule device provided herein Schematic diagram, the metadata rule device can include:
Metadata obtains module 10, for obtaining the member of the first leaf node of memory B+ tree when receiving rule instruction Data;
Continuous overlay area determining module 20, for inquiring the corresponding disk B+ leaf child node of each metadata, Determine the continuous overlay area of memory B+ tree Yu disk B+ tree;
Metadata merging module 30, for the metadata and disk to the memory B+ tree in same continuous overlay area The metadata of B+ tree merges processing, obtains new leaf node;
Judgment module 40 is terminated, for judging whether the first leaf node of memory B+ tree is that memory B+ tree terminates leaf section Point;
Module 50 is looped through, if not being that memory B+ tree terminates leaf node for memory B+ the first leaf node of tree, The metadata of the next leaf node of memory B+ tree is obtained, and returns to continuous overlay area determining module, until traversing memory B+ Tree terminates leaf node;
Metadata rule module 60 is completed metadata and is fallen for storing the metadata of each new leaf node to disk Disk.
As a kind of preferred embodiment, which can also include pre- judgment module, for judging disk In whether there is disk B+ tree;If it does not exist, then memory B+ tree is stored to disk, completes metadata rule;If it exists, then it holds Row obtains the step of metadata of the first leaf node of memory B+ tree.
As a kind of preferred embodiment, above-mentioned metadata merging module 30 can be specifically used for passing through MERGING/SORTING ALGORITHM pair The metadata of memory B+ tree in continuous overlay area and the metadata of disk B+ tree merge processing, obtain new leaf Node.
As a kind of preferred embodiment, which can also include the first memory release module, and being used for will The corresponding disk B+ leaf knot removal of the metadata of the upper leaf node of memory B+ tree.
As a kind of preferred embodiment, above-mentioned metadata rule module 60 can be specifically used for generating by each new leaf node New memory B+ tree;New memory B+ tree is stored to disk, metadata rule is completed.
As a kind of preferred embodiment, which can also include the second memory release module, for deleting Except memory B+ tree new in memory, memory release is completed.
Above method embodiment is please referred to for the introduction of device provided by the present application, this will not be repeated here by the application.
To solve the above problems, referring to FIG. 6, Fig. 6 is a kind of structure of metadata rule system provided herein Schematic diagram, the metadata rule system can include:
Memory 1, for storing computer program;
Processor 2, realizes following steps when for executing computer program:
When receiving rule instruction, the metadata of the first leaf node of memory B+ tree is obtained;It is corresponding to each metadata Disk B+ leaf child node is inquired, and determines the continuous overlay area of memory B+ tree Yu disk B+ tree;To in same continuous The metadata of memory B+ tree in overlay area and the metadata of disk B+ tree merge processing, obtain new leaf node;Sentence Whether disconnected the first leaf node of memory B+ tree is that memory B+ tree terminates leaf node;If it is not, then obtaining the next leaf of memory B+ tree The metadata of node, and inquiry disk B+ leaf child node corresponding with each metadata is returned, determine memory B+ tree and disk B+ The step of continuous overlay area of tree, until traversing memory B+ tree terminates leaf node;By the metadata of each new leaf node It stores to disk, completes metadata rule.
Above method embodiment is please referred to for the introduction of equipment provided by the present application, this will not be repeated here by the application.
To solve the above problems, present invention also provides a kind of computer readable storage medium, the computer-readable storage It is stored with computer program on medium, can be realized when computer program is executed by processor following rapid:
When receiving rule instruction, the metadata of the first leaf node of memory B+ tree is obtained;It is corresponding to each metadata Disk B+ leaf child node is inquired, and determines the continuous overlay area of memory B+ tree Yu disk B+ tree;To in same continuous The metadata of memory B+ tree in overlay area and the metadata of disk B+ tree merge processing, obtain new leaf node;Sentence Whether disconnected the first leaf node of memory B+ tree is that memory B+ tree terminates leaf node;If it is not, then obtaining the next leaf of memory B+ tree The metadata of node, and inquiry disk B+ leaf child node corresponding with each metadata is returned, determine memory B+ tree and disk B+ The step of continuous overlay area of tree, until traversing memory B+ tree terminates leaf node;By the metadata of each new leaf node It stores to disk, completes metadata rule.
The computer readable storage medium may include: USB flash disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic or disk etc. is various to deposit Store up the medium of program code.
Above method embodiment is please referred to for the introduction of computer readable storage medium provided by the present application, the application exists This is not repeated them here.
Each embodiment is described in a progressive manner in specification, the highlights of each of the examples are with other realities The difference of example is applied, the same or similar parts in each embodiment may refer to each other.For device disclosed in embodiment Speech, since it is corresponded to the methods disclosed in the examples, so being described relatively simple, related place is referring to method part illustration ?.
Professional further appreciates that, unit described in conjunction with the examples disclosed in the embodiments of the present disclosure And algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and The interchangeability of software generally describes each exemplary composition and step according to function in the above description.These Function is implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Profession Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered Think beyond scope of the present application.
The step of method described in conjunction with the examples disclosed in this document or algorithm, can directly be held with hardware, processor The combination of capable software module or the two is implemented.Software module can be placed in random access memory (RAM), memory, read-only deposit Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology In any other form of storage medium well known in field.
Above to metadata rule method, apparatus, system and computer readable storage medium provided herein into It has gone and has been discussed in detail.Specific examples are used herein to illustrate the principle and implementation manner of the present application, the above implementation The explanation of example is merely used to help understand the present processes and its core concept.It should be pointed out that for the general of the art For logical technical staff, under the premise of not departing from the application principle, can also to the application, some improvement and modification can also be carried out, this A little improvement and modification also fall into the protection scope element of the claim of this application.

Claims (10)

1. a kind of metadata rule method characterized by comprising
When receiving rule instruction, the metadata of the first leaf node of memory B+ tree is obtained;
The corresponding disk B+ leaf child node of each metadata is inquired, determines the memory B+ tree and the disk B+ The continuous overlay area of tree;
The metadata of metadata and the disk B+ tree to the memory B+ tree in same continuous overlay area carries out Merging treatment obtains new leaf node;
Judge whether the first leaf node of the memory B+ tree is that memory B+ tree terminates leaf node;
If it is not, then obtaining the metadata of the next leaf node of memory B+ tree, and return to the inquiry and each metadata Corresponding disk B+ leaf child node, the step of determining the continuous overlay area of the memory B+ tree and the disk B+ tree, directly Leaf node is terminated to the memory B+ tree is traversed;
The metadata of each new leaf node is stored to disk, metadata rule is completed.
2. metadata rule method as described in claim 1, which is characterized in that acquisition memory B+ first leaf node of tree Metadata before, further includes:
Judge in the disk with the presence or absence of the disk B+ tree;
If it does not exist, then the memory B+ tree is stored to the disk, completes metadata rule;
If it exists, then the step of executing the metadata for obtaining the first leaf node of memory B+ tree.
3. metadata rule method as described in claim 1, which is characterized in that described in the continuous overlay area The corresponding disk B+ leaf child node of the metadata of memory B+ tree is identical or adjacent.
4. the metadata rule method as described in claims 1 to 3 any one, which is characterized in that described pair is in the company The metadata of the memory B+ tree and the metadata of the disk B+ tree in continuous overlay area merge processing, obtain young leaves Child node, comprising:
Metadata and the disk B by MERGING/SORTING ALGORITHM to the memory B+ tree in the continuous overlay area The metadata of+tree merges processing, obtains the new leaf node.
5. metadata rule method as claimed in claim 4, which is characterized in that described to obtain the next leaf of memory B+ tree Before the metadata of node, further includes:
By the corresponding disk B+ leaf knot removal of the metadata of the upper leaf node of memory B+ tree.
6. metadata rule method as claimed in claim 5, which is characterized in that first number by each new leaf node According to storing to disk, metadata rule is completed, comprising:
New memory B+ tree is generated by each new leaf node;
The new memory B+ tree is stored to the disk, metadata rule is completed.
7. metadata rule method as claimed in claim 6, which is characterized in that after the completion metadata rule, also wrap It includes:
New memory B+ tree described in memory is deleted, memory release is completed.
8. a kind of metadata rule device, which is characterized in that further include:
Metadata obtains module, for obtaining the metadata of the first leaf node of memory B+ tree when receiving rule instruction;
Continuous overlay area determining module, for inquiring the corresponding disk B+ leaf child node of each metadata, really The continuous overlay area of fixed the memory B+ tree and the disk B+ tree;
Metadata merging module, for the memory B+ tree in the same continuous overlay area metadata and the magnetic The metadata of PanB+Shu merges processing, obtains new leaf node;
Judgment module is terminated, for judging whether the first leaf node of the memory B+ tree is that memory B+ tree terminates leaf node;
Module is looped through, if not being that the memory B+ tree terminates leaf node for the first leaf node of the memory B+ tree, The metadata of the next leaf node of memory B+ tree is then obtained, and returns to the continuous overlay area determining module, until time It goes through to the memory B+ tree and terminates leaf node;
Metadata rule module completes metadata rule for storing the metadata of each new leaf node to disk.
9. a kind of metadata rule system characterized by comprising
Memory, for storing computer program;
Processor realizes metadata rule as claimed in any one of claims 1 to 7 when for executing the computer program The step of method.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium Program, the computer program realize metadata rule side as claimed in any one of claims 1 to 7 when being executed by processor The step of method.
CN201910429310.8A 2019-05-22 2019-05-22 Metadata disk-dropping method, device and system and computer-readable storage medium Active CN110147204B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910429310.8A CN110147204B (en) 2019-05-22 2019-05-22 Metadata disk-dropping method, device and system and computer-readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910429310.8A CN110147204B (en) 2019-05-22 2019-05-22 Metadata disk-dropping method, device and system and computer-readable storage medium

Publications (2)

Publication Number Publication Date
CN110147204A true CN110147204A (en) 2019-08-20
CN110147204B CN110147204B (en) 2020-03-10

Family

ID=67592733

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910429310.8A Active CN110147204B (en) 2019-05-22 2019-05-22 Metadata disk-dropping method, device and system and computer-readable storage medium

Country Status (1)

Country Link
CN (1) CN110147204B (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110704244A (en) * 2019-09-29 2020-01-17 苏州浪潮智能科技有限公司 Disk offline processing method, device and equipment and readable storage medium
CN110989936A (en) * 2019-12-06 2020-04-10 深信服科技股份有限公司 Storage space allocation method and device and storage medium
CN111309258A (en) * 2020-02-14 2020-06-19 苏州浪潮智能科技有限公司 B + tree access method and device and computer readable storage medium
CN111813603A (en) * 2020-07-16 2020-10-23 济南浪潮数据技术有限公司 Method, device and equipment for backing up metadata of thin volume and readable storage medium
CN113901276A (en) * 2021-09-30 2022-01-07 苏州浪潮智能科技有限公司 Data management method, B + tree loading method, device and electronic equipment
CN116069267A (en) * 2023-04-06 2023-05-05 苏州浪潮智能科技有限公司 Writing cache method, system, equipment and storage medium of RAID card
CN116431080A (en) * 2023-06-09 2023-07-14 苏州浪潮智能科技有限公司 Data disc-dropping method, system, equipment and computer readable storage medium
CN116501267A (en) * 2023-06-27 2023-07-28 苏州浪潮智能科技有限公司 Independent redundant disk array card control method and device

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105487820A (en) * 2015-11-30 2016-04-13 中国科学院信息工程研究所 Time slice rotation mechanism based tree storage structure write amplification optimization method
CN108319625A (en) * 2017-01-17 2018-07-24 广州市动景计算机科技有限公司 Piece file mergence method and apparatus
US20180253255A1 (en) * 2017-03-01 2018-09-06 Tintri Inc. Efficient deduplication for storage systems
CN109033365A (en) * 2018-07-26 2018-12-18 郑州云海信息技术有限公司 A kind of data processing method and relevant device
CN109189343A (en) * 2018-09-14 2019-01-11 郑州云海信息技术有限公司 A kind of metadata rule method, apparatus, equipment and computer readable storage medium
CN109213445A (en) * 2018-08-23 2019-01-15 郑州云海信息技术有限公司 A kind of management method, management system and the relevant apparatus of storage system metadata
CN109388346A (en) * 2018-10-15 2019-02-26 郑州云海信息技术有限公司 A kind of data rule method and relevant apparatus
CN109558084A (en) * 2018-11-29 2019-04-02 文华学院 A kind of data processing method and relevant device
CN109614045A (en) * 2018-12-06 2019-04-12 广东浪潮大数据研究有限公司 A kind of metadata rule method, apparatus and relevant device
US10268403B1 (en) * 2015-10-29 2019-04-23 Pure Storage, Inc. Combining multiple copy operations into a single copy operation

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10268403B1 (en) * 2015-10-29 2019-04-23 Pure Storage, Inc. Combining multiple copy operations into a single copy operation
CN105487820A (en) * 2015-11-30 2016-04-13 中国科学院信息工程研究所 Time slice rotation mechanism based tree storage structure write amplification optimization method
CN108319625A (en) * 2017-01-17 2018-07-24 广州市动景计算机科技有限公司 Piece file mergence method and apparatus
US20180253255A1 (en) * 2017-03-01 2018-09-06 Tintri Inc. Efficient deduplication for storage systems
CN109033365A (en) * 2018-07-26 2018-12-18 郑州云海信息技术有限公司 A kind of data processing method and relevant device
CN109213445A (en) * 2018-08-23 2019-01-15 郑州云海信息技术有限公司 A kind of management method, management system and the relevant apparatus of storage system metadata
CN109189343A (en) * 2018-09-14 2019-01-11 郑州云海信息技术有限公司 A kind of metadata rule method, apparatus, equipment and computer readable storage medium
CN109388346A (en) * 2018-10-15 2019-02-26 郑州云海信息技术有限公司 A kind of data rule method and relevant apparatus
CN109558084A (en) * 2018-11-29 2019-04-02 文华学院 A kind of data processing method and relevant device
CN109614045A (en) * 2018-12-06 2019-04-12 广东浪潮大数据研究有限公司 A kind of metadata rule method, apparatus and relevant device

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110704244A (en) * 2019-09-29 2020-01-17 苏州浪潮智能科技有限公司 Disk offline processing method, device and equipment and readable storage medium
CN110704244B (en) * 2019-09-29 2023-01-10 苏州浪潮智能科技有限公司 Disk offline processing method, device and equipment and readable storage medium
CN110989936A (en) * 2019-12-06 2020-04-10 深信服科技股份有限公司 Storage space allocation method and device and storage medium
CN111309258A (en) * 2020-02-14 2020-06-19 苏州浪潮智能科技有限公司 B + tree access method and device and computer readable storage medium
US11762827B2 (en) 2020-02-14 2023-09-19 Inspur Suzhou Intelligent Technology Co., Ltd. B-plus tree access method and apparatus, and computer-readable storage medium
CN111309258B (en) * 2020-02-14 2021-10-15 苏州浪潮智能科技有限公司 B + tree access method and device and computer readable storage medium
CN111813603B (en) * 2020-07-16 2022-06-10 济南浪潮数据技术有限公司 Method, device and equipment for backing up metadata of thin volume and readable storage medium
CN111813603A (en) * 2020-07-16 2020-10-23 济南浪潮数据技术有限公司 Method, device and equipment for backing up metadata of thin volume and readable storage medium
CN113901276A (en) * 2021-09-30 2022-01-07 苏州浪潮智能科技有限公司 Data management method, B + tree loading method, device and electronic equipment
CN116069267A (en) * 2023-04-06 2023-05-05 苏州浪潮智能科技有限公司 Writing cache method, system, equipment and storage medium of RAID card
CN116431080A (en) * 2023-06-09 2023-07-14 苏州浪潮智能科技有限公司 Data disc-dropping method, system, equipment and computer readable storage medium
CN116431080B (en) * 2023-06-09 2023-08-29 苏州浪潮智能科技有限公司 Data disc-dropping method, system, equipment and computer readable storage medium
CN116501267A (en) * 2023-06-27 2023-07-28 苏州浪潮智能科技有限公司 Independent redundant disk array card control method and device
CN116501267B (en) * 2023-06-27 2023-09-29 苏州浪潮智能科技有限公司 Independent redundant disk array card control method and device

Also Published As

Publication number Publication date
CN110147204B (en) 2020-03-10

Similar Documents

Publication Publication Date Title
CN110147204A (en) A kind of metadata rule method, apparatus, system and computer readable storage medium
US10120869B2 (en) Method and apparatus for fault-tolerant memory management
CN111190904B (en) Method and device for hybrid storage of graph-relational database
US11392314B2 (en) Sequentially writing metadata into a solid state disk by redirect-on-write
WO2021208239A1 (en) Low-latency file system address space management method and system, and medium
CN109359062A (en) A kind of metadata read buffer method, device and equipment
CN109857803A (en) Method of data synchronization, device, equipment, system and computer readable storage medium
CN109471843A (en) A kind of metadata cache method, system and relevant apparatus
CN107766355B (en) Hierarchical data management method, hierarchical data management system and instant messaging system
CN109213445A (en) A kind of management method, management system and the relevant apparatus of storage system metadata
CN104346404B (en) A kind of method, equipment and system for accessing data
CN110795042A (en) Method for writing and flushing metadata of full flash memory storage system and related components
CN110399096A (en) Metadata of distributed type file system caches the method, apparatus and equipment deleted again
CN115576501A (en) Node updating method, system and related device of RAID card
CN109086462A (en) The management method of metadata in a kind of distributed file system
CN104424222A (en) Database indexing method and device
CN110020272A (en) Caching method, device and computer storage medium
KR100907477B1 (en) Apparatus and method for managing index of data stored in flash memory
CN106528703A (en) Deduplication mode switching method and apparatus
CN109753446A (en) Data cached method is brushed under a kind of storage system
CN103714121B (en) The management method and device of a kind of index record
WO2024119797A1 (en) Data processing method and system, device, and storage medium
CN103348653A (en) Capacity expansion method and device and data access method and device
CN105045891A (en) Method and system for improving performance of sequence list, architecture, optimization method and storage apparatus
CN111625198A (en) Metadata caching method and metadata caching device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant