CN110147204A - A kind of metadata rule method, apparatus, system and computer readable storage medium - Google Patents
A kind of metadata rule method, apparatus, system and computer readable storage medium Download PDFInfo
- Publication number
- CN110147204A CN110147204A CN201910429310.8A CN201910429310A CN110147204A CN 110147204 A CN110147204 A CN 110147204A CN 201910429310 A CN201910429310 A CN 201910429310A CN 110147204 A CN110147204 A CN 110147204A
- Authority
- CN
- China
- Prior art keywords
- metadata
- tree
- memory
- leaf node
- disk
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000012545 processing Methods 0.000 claims abstract description 16
- 238000004590 computer program Methods 0.000 claims description 11
- 230000009286 beneficial effect Effects 0.000 abstract description 4
- 230000008569 process Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 11
- 230000003321 amplification Effects 0.000 description 3
- 230000000052 comparative effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000003199 nucleic acid amplification method Methods 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000000151 deposition Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 238000012966 insertion method Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
- G06F16/9027—Trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/907—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Library & Information Science (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application discloses a kind of metadata rule methods to determine the continuous overlay area of memory B+ tree Yu disk B+ tree including inquiring the corresponding disk B+ leaf child node of each metadata in memory B+ the first leaf node of tree;Processing is merged to all metadata in same continuous overlay area, obtains new leaf node;If the first leaf node of memory B+ tree is not to terminate leaf node, the metadata of next leaf node is obtained, and returns to the step that continuous overlay area determines, until traversing termination leaf node;The metadata of each new leaf node is stored to disk, metadata rule is completed;The metadata rule method rule efficiency is higher, and large-scale concurrent random access metadata can be made to reach higher efficiency.Disclosed herein as well is a kind of metadata rule device, system and computer readable storage mediums, all have above-mentioned beneficial effect.
Description
Technical field
This application involves technical field of data storage, in particular to a kind of metadata rule method further relates to a kind of first number
According to rule device, system and computer readable storage medium.
Background technique
Metadata refers to the data of description data, not only indicates the information such as type, title, the value of data, also provides number
According to contextual information, such as the affiliated domain of data, data source.In data-storage system, metadata is the base of information storage
Plinth is the minimum unit of data.In the storage of full flash memory, a large amount of and high concurrent data access and inquiry problem are necessarily involved,
Effective management metadata, just can increase access efficiency.Therefore, effective management of metadata, rule save efficiency in full flash memory
And to avoid writing amplification most important, and large-scale concurrent random access metadata can be made to reach higher efficiency.
Full flash memory metadata generally carrys out tissue with the data structure of B+ tree, including memory B+ tree and two, disk B+ tree
Point, it after the metadata of memory B+ tree in write buffer reaches certain threshold value, needs to be written on disk, be closed with disk B+ tree
And new B+ tree is obtained, and then complete rule.However, in the prior art, above-mentioned metadata merging process is inserted by B+ tree
Enter operation and realizes that traditional B+ tree insertion operation is to be inserted into one by one, i.e., the metadata in memory B+ tree is inserted into disk B one by one
In+tree, when need rule metadata quantity it is larger when, this kind of rule method efficiency is more low;Further, when disk B+ tree
When middle metadata quantity is larger, insertion process is then related to a large amount of data search and copy, further reduced metadata and falls
Disk efficiency.
Therefore, how to effectively improve metadata rule efficiency is those skilled in the art's urgent problem to be solved.
Summary of the invention
The purpose of the application is to provide a kind of metadata rule method, and the metadata rule method rule efficiency is higher, can
Large-scale concurrent random access metadata is set to reach higher efficiency;The another object of the application is to provide a kind of metadata rule
Device, system and computer readable storage medium, it may have above-mentioned beneficial effect.
In order to solve the above technical problems, this application provides a kind of metadata rule method, the metadata rule method
Include:
When receiving rule instruction, the metadata of the first leaf node of memory B+ tree is obtained;
The corresponding disk B+ leaf child node of each metadata is inquired, determines the memory B+ tree and the magnetic
The continuous overlay area of PanB+Shu;
The metadata of metadata and the disk B+ tree to the memory B+ tree in same continuous overlay area
Processing is merged, new leaf node is obtained;
Judge whether the first leaf node of the memory B+ tree is that memory B+ tree terminates leaf node;
If it is not, then obtaining the metadata of the next leaf node of memory B+ tree, and return to the inquiry and each member
The corresponding disk B+ leaf child node of data determines the step of the continuous overlay area of the memory B+ tree and the disk B+ tree
Suddenly, until traversing the memory B+ tree terminates leaf node;
The metadata of each new leaf node is stored to disk, metadata rule is completed.
Preferably, before the metadata for obtaining the first leaf node of memory B+ tree, further includes:
Judge in the disk with the presence or absence of the disk B+ tree;
If it does not exist, then the memory B+ tree is stored to the disk, completes metadata rule;
If it exists, then the step of executing the metadata for obtaining the first leaf node of memory B+ tree.
Preferably, the corresponding disk B+ leaf of metadata of the memory B+ tree in the continuous overlay area
Node is identical or adjacent.
Preferably, the metadata of the described pair of memory B+ tree in the continuous overlay area and the disk B+
The metadata of tree merges processing, obtains new leaf node, comprising:
By MERGING/SORTING ALGORITHM to the metadata of the memory B+ tree in the continuous overlay area and described
The metadata of disk B+ tree merges processing, obtains the new leaf node.
Preferably, before the metadata for obtaining the next leaf node of memory B+ tree, further includes:
By the corresponding disk B+ leaf knot removal of the metadata of the upper leaf node of memory B+ tree.
Preferably, the metadata by each new leaf node is stored to disk, completes metadata rule, packet
It includes:
New memory B+ tree is generated by each new leaf node;
The new memory B+ tree is stored to the disk, metadata rule is completed.
Preferably, after the completion metadata rule, further includes:
New memory B+ tree described in memory is deleted, memory release is completed.
In order to solve the above technical problems, present invention also provides a kind of metadata rule device, the metadata rule dress
It sets and includes:
Metadata obtains module, for obtaining first number of the first leaf node of memory B+ tree when receiving rule instruction
According to;
Continuous overlay area determining module, for looking into the corresponding disk B+ leaf child node of each metadata
It askes, determines the continuous overlay area of the memory B+ tree Yu the disk B+ tree;
Metadata merging module, for the memory B+ tree in the same continuous overlay area metadata and institute
The metadata for stating disk B+ tree merges processing, obtains new leaf node;
Judgment module is terminated, for judging whether the first leaf node of the memory B+ tree is that memory B+ tree terminates leaf section
Point;
Module is looped through, if not being that the memory B+ tree terminates leaf section for the first leaf node of the memory B+ tree
Point, then obtain the metadata of the next leaf node of memory B+ tree, and returns to the continuous overlay area determining module, until
It traverses the memory B+ tree and terminates leaf node;
Metadata rule module completes metadata for storing the metadata of each new leaf node to disk
Rule.
In order to solve the above technical problems, present invention also provides a kind of metadata rule system, metadata rule system
System includes:
Memory, for storing computer program;
Processor, the step of any one of the above metadata rule method is realized when for executing the computer program.
In order to solve the above technical problems, the computer can present invention also provides a kind of computer readable storage medium
It reads to be stored with computer program on storage medium, the computer program realizes any one of the above member number when being executed by processor
The step of according to rule method.
A kind of metadata rule method provided herein, including when receiving rule instruction, obtain memory B+ tree
The metadata of first leaf node;The corresponding disk B+ leaf child node of each metadata is inquired, is determined in described
Deposit the continuous overlay area of B+ tree Yu the disk B+ tree;To the member of the memory B+ tree in same continuous overlay area
Data and the metadata of the disk B+ tree merge processing, obtain new leaf node;Judge the first leaf of the memory B+ tree
Whether child node is that memory B+ tree terminates leaf node;If it is not, the metadata of the next leaf node of memory B+ tree is then obtained,
And inquiry disk B+ leaf child node corresponding with each metadata is returned, determine the memory B+ tree and the disk
The step of continuous overlay area of B+ tree, until traversing the memory B+ tree terminates leaf node;By each new leaf section
The metadata of point is stored to disk, completes metadata rule.
As it can be seen that metadata rule method provided herein, based on the characteristic of B+ data tree structure, metadata is all deposited
It is stored on the leaf node of B+ tree, and is ordered arrangement, the metadata in each leaf node of memory B+ tree can be seen as a result,
Do an ordered sequence, regard the metadata in disk B+ leaf child node as an ordered sequence, at this point, by inquiry and it is true
The continuous overlay area of fixed two ordered sequences, to the memory metadata and magnetic disk metadata in same continuous overlay area
Processing is merged, to complete the sequence of batch metadata and generate to obtain new leaf node, data is further realized and falls
Disk;Compared to the prior art, the problem of can effectively avoid rule inefficiency caused by metadata is inserted into one by one can make extensive
Concurrent random access metadata reaches higher efficiency.
A kind of metadata rule device, system and computer readable storage medium provided herein, all has
Beneficial effect is stated, details are not described herein.
Detailed description of the invention
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of application for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of flow diagram of metadata rule method provided herein;
Fig. 2 is the leaf node comparative diagram of a kind of memory B+ tree and disk B+ tree provided herein;
Fig. 3 is the leaf node comparative diagram of another memory B+ tree and disk B+ tree provided herein;
Fig. 4 is the flow diagram of another metadata rule method provided herein;
Fig. 5 is a kind of structural schematic diagram of metadata rule device provided herein;
Fig. 6 is a kind of structural schematic diagram of metadata rule system provided herein.
Specific embodiment
The core of the application is to provide a kind of metadata rule method, and the metadata rule method rule efficiency is higher, can
Large-scale concurrent random access metadata is set to reach higher efficiency;Another core of the application is to provide a kind of metadata rule
Device, system and computer readable storage medium, it may have above-mentioned beneficial effect.
To keep the purposes, technical schemes and advantages of the embodiment of the present application clearer, below in conjunction with the embodiment of the present application
In attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is
Some embodiments of the present application, instead of all the embodiments.Based on the embodiment in the application, those of ordinary skill in the art
All other embodiment obtained without making creative work, shall fall in the protection scope of this application.
Referring to FIG. 1, Fig. 1 is a kind of flow diagram of metadata rule method provided herein, the metadata
Rule method may include:
S101: when receiving rule instruction, the metadata of the first leaf node of memory B+ tree is obtained;
In general, the quantity of the leaf node in memory B+ tree is not unique, this step aims at memory B+ tree first
The acquisition of metadata in leaf node, first leaf node are first leaf node of memory B+ tree.Specifically, first
The process of data rule is related to metadata in memory B+ tree and merges with metadata in disk B+ tree, which is caching
Memory in execute, therefore, when receive rule instruction when, the first leaf node of memory B+ tree can be read out, acquisition
Metadata therein.
Certainly, the characteristic based on B+ data tree structure, the metadata in leaf node is ordered arrangement, therefore, above-mentioned
The metadata of memory B+ the first leaf node of tree of acquisition is equivalent to an ordered sequence.
S102: disk B+ leaf child node corresponding to each metadata is inquired, and determines memory B+ tree and disk B+ tree
Continuous overlay area;
This step aims at the acquisition of the continuous overlay area of memory B+ tree and disk B+ tree.Due in memory B+ tree
Each metadata has its corresponding disk B+ leaf child node, then the metadata in obtaining the first leaf node of memory B+ tree
On the basis of, the corresponding disk B+ leaf child node of each metadata in first leaf node is inquired, can be obtained
Obtain the continuous overlay area of memory B+ tree and disk B+ tree.
It describes in detail below by citing to above-mentioned continuous overlay area.
Firstly, referring to FIG. 2, Fig. 2 is the leaf node pair of a kind of memory B+ tree and disk B+ tree provided herein
According to figure, wherein " node 1 " in memory B+ leaf child node is the first leaf node, including 3 metadata, respectively
For key1, key2 and key3;It is determined after being inquired by leaf node, key1 corresponds to first leaf section of disk B+ tree
Point, key2 corresponds to second leaf node of disk B+ tree, and key3 corresponds to the 4th leaf node of disk B+ tree, this
When, the corresponding leaf node of key3 leaf node corresponding with the first two metadata is discontinuous, then the first leaf of disk B+ tree section
Point is a continuous overlay area, the 4th leaf node of disk B+ tree with the region that disk B+ the second leaf node of tree is covered
The region covered is a continuous overlay area.
Further, referring to FIG. 3, Fig. 3 is the leaf section of another memory B+ tree and disk B+ tree provided herein
Comparative diagram is put, " node 1 " in memory B+ leaf child node is the first leaf node, including 4 metadata, respectively
Key a, key b, key c and key d;It is determined after being inquired by leaf node, key a and key b both correspond to disk B+
First leaf node of tree, key c correspond to the third leaf node of disk B+ tree, and key d corresponds to disk B+ tree
4th leaf node, at this point, the corresponding leaf node of key a and key b is continuous, the corresponding leaf node of key c and key d
Continuously, and the corresponding leaf node of key b and key c is discontinuous, then the region that the first leaf node of disk B+ tree is covered is
The region that one continuous overlay area, the third leaf node of disk B+ tree and the 4th leaf node are covered is a company
Continuous overlay area.
It can be seen that the corresponding disk B+ of metadata of the memory B+ tree in continuous overlay area based on above-mentioned elaboration
Leaf child node can be identical, be also possible to adjacent.But the implementation is not unique, can also preset
Continuous overlay area is same disk B+ leaf child node, the i.e. corresponding disk B+ leaf child node of metadata in memory B+ tree
For same leaf node, at this point, the leaf node is continuous overlay area, but this kind of implementation is compared to above-mentioned continuous covering
The implementation that the corresponding disk B+ leaf child node of memory B+ tree metadata can be identical or adjacent in region, efficiency
It is relatively low, suitable for the situation less to rule metadata.
Therefore, as a kind of preferred embodiment, the application is set at first number of the memory B+ tree in continuous overlay area
It is identical or adjacent according to corresponding disk B+ leaf child node.The conjunction of metadata in maximum continuous overlay area is realized as a result,
And realize the batch processing of metadata, both effectively ensure that it is lower write amplification, also achieve higher metadata rule effect
Rate.
S103: the metadata of metadata and disk B+ tree to the memory B+ tree in same continuous overlay area carries out
Merging treatment obtains new leaf node;
The merging treatment that this step aims at metadata and metadata in disk B+ tree in memory B+ tree operates.It is specific and
Speech, in same continuous overlay area, the metadata of memory B+ tree is equivalent to an ordered sequence, the metadata phase of disk B+ tree
When in an ordered sequence, at this point, the merging process is the ordering by merging process of two ordered sequences.Certainly, for two
The ordering by merging process of ordered sequence, can be used any one in prior art, and the application does not limit this.Further,
After completing ordering by merging, can be obtained a new complete ordered sequence, the complete ordered sequence constitute one it is new
Leaf node.
Since above-mentioned metadata merging process is carried out in the memory of caching, the merging of that identical element data is necessarily involved
The acquisition of metadata on disk, that is to say, that the metadata by disk B+ tree is needed to read into memory.By taking Fig. 2 as an example, due to
The 4th leaf node that key3 corresponds to disk B+ tree has been inquired, then in data merging process, without to disk B+
Three, the ground leaf node of tree is read out, and avoids reading disk invalid node, while reducing to disk write amplification, is mentioned
High metadata combined efficiency.
Preferably, the metadata of above-mentioned metadata and disk B+ tree to the memory B+ tree in the continuous overlay area into
Row merging treatment obtains new leaf node, may include by MERGING/SORTING ALGORITHM to the memory B in continuous overlay area
+ the metadata of tree and the metadata of disk B+ tree merge processing, obtain new leaf node.
Specifically, this application provides a kind of more concrete implementation modes, i.e., for the merging process of above-mentioned metadata
It is realized based on MERGING/SORTING ALGORITHM.Merger sequence is built upon a kind of operational effective sort algorithm of merger, specifically will
Orderly subsequence merges, and obtains complete ordering sequence, i.e., first makes each subsequence orderly, then makes to have between subsequence section
Sequence finally obtains complete orderly sequence.
Merger sequence is a kind of sort algorithm with high efficiency and high stability.For example, have 10 to rule metadata,
That is 10 key values may then require to look up 10 positions such as according to traditional insertion method, then do 10 data movements,
Efficiency is very low.But if sorting using merger generates new node, do not need to search, a key value only needs to be inserted into once, no
It can change again, efficiency is higher.
S104: judge whether the first leaf node of memory B+ tree is that memory B+ tree terminates leaf node;If it is not, then executing
S105, if so, executing S106;
Above-mentioned S101~S103 be to operation performed by metadata in memory B+ the first leaf node of tree, to complete it is whole
A memory B+ tree merges with disk B+ tree, then needs to execute aforesaid operations to all leaf nodes in memory B+ tree.Therefore,
Can determine whether the first leaf node of above-mentioned memory B+ tree is that memory B+ tree terminates leaf node, which is interior
Deposit the last one leaf node of B+ tree.
S105: obtaining the metadata of the next leaf node of memory B+ tree, and return to S102, until traversing memory B+ tree eventually
Only leaf node;
Specifically, needing to continue under traversal if the first leaf node of above-mentioned memory B+ tree is not to terminate leaf node
One leaf node obtains the metadata in the next leaf node of memory B+ tree, and certainly, the acquisition process of the metadata can refer to the
The acquisition process of one leaf node metadata, is not repeating herein.As a result, by being held to all leaf nodes in memory B+ tree
Above-mentioned S101~S103 the operation of row, can be realized complete the merging of memory B+ tree with disk B+ tree, obtains all new leaves
Node.
As a kind of preferred embodiment, before the metadata of the next leaf node of above-mentioned acquisition memory B+ tree, can also wrap
It includes: by the corresponding disk B+ leaf knot removal of the metadata of the upper leaf node of memory B+ tree.
Since the merging process of metadata needs the metadata by disk B+ tree to read into memory, however, being based on B+ tree
The characteristic of data structure, metadata is ordered into storage wherein, and metadata rule method provided by the present application is with one
The leaf node of a memory B+ tree is what unit carried out, it is contemplated that, metadata is corresponding in the upper leaf node of memory B+ tree
Disk B+ leaf child node disk B+ leaf child node corresponding with metadata in the next leaf node of memory B+ tree absolutely not
It is the same leaf node, and in the next leaf node of memory B+ tree including the corresponding disk B+ leaf child node certainty of metadata
It deposits in the upper leaf node of B+ tree after the corresponding disk B+ leaf child node of metadata, therefore, to reduce resources occupation rate,
It, can be by each metadata in memory B+ leaf child node after every metadata for completing a memory B+ leaf child node merges
Corresponding disk B+ leaf knot removal, to achieve the purpose that releasing memory.
Based on described above, it is contemplated that, when initially carrying out metadata reading to disk B+ tree, without disposable
All metadata being queried in disk B+ tree are read into memory, it only need to will be each in memory B+ tree current leaf node
The corresponding disk B+ leaf child node of metadata is read into memory, and realizes the swapping in and out of memory, to avoid memory money
Source occupies.Further, it for the disk B+ leaf child node read, can also directly be deleted in disk, to reduce
The resource occupation of disk;Certainly, which can also hold after all metadata being queried to are read in disk B+ tree
Row, the application do not limit this.
S106: the metadata of each new leaf node is stored to disk, completes metadata rule.
This step aims at metadata rule.Specifically, complete memory B+ tree in all leaf nodes metadata
Merge with metadata in disk B+ tree, after obtaining all new leaf nodes, can be stored into disk, realizes first number
According to rule.Further, for the metadata after rule, further compacting operation can be also executed to it, so that each kv value (member
Data) in disk make-up solid matter cloth, to effectively improve disk utilization.
Preferably, the above-mentioned metadata by each new leaf node is stored to disk, is completed metadata rule, be can wrap
It includes: new memory B+ tree is generated by each new leaf node;New memory B+ tree is stored to disk, metadata rule is completed.
For the process of above-mentioned metadata rule, this application provides more concrete implementation methods, specifically, caching
Memory in, according to the architectural characteristic of memory B+ tree, first can generate new memory B+ tree by all new leaf nodes based on acquisition, have
Body realizes process are as follows: generates corresponding non-leaf nodes upwards in turn, until the n omicronn-leaf child node as root node generates, obtains
New memory B+ tree, finally again stores the new memory B+ tree into disk, completes metadata rule.
It can also include: to delete new memory in memory after above-mentioned completion metadata rule as a kind of preferred embodiment
B+ tree completes memory release.
It, can also further releasing memory after completing metadata rule.Specifically, due to having completed rule, it is interior at this time
The new memory B+ tree for depositing middle generation does not use, and is deleted at this time, memory release is realized, so as to first number newly to input
According to offer memory space.
Further, it since metadata complete by rule, and is stored in the form of new B+ tree into disk, then, it is former
The disk B+ tree of beginning equally no longer has value, and is equally deleted at this time, resource utilization is effectively reduced.
Metadata rule method provided herein, based on the characteristic of B+ data tree structure, metadata is all stored in B
On the leaf node of+tree, and it is ordered arrangement, the metadata in each leaf node of memory B+ tree can be regarded as one as a result,
A ordered sequence regards the metadata in disk B+ leaf child node as an ordered sequence, at this point, by inquiring and determining two
The continuous overlay area of a ordered sequence carries out the memory metadata and magnetic disk metadata that are in same continuous overlay area
Merging treatment further realizes data rule to complete the sequence of batch metadata and generate to obtain new leaf node;Phase
Compared with the prior art, the problem of can effectively avoid rule inefficiency caused by metadata is inserted into one by one, large-scale concurrent can be made
Random access metadata reaches higher efficiency.
On the basis of above-mentioned each embodiment, referring to FIG. 4, Fig. 4 is that another metadata provided herein is fallen
The flow diagram of disk method, the metadata rule method may include:
S201: rule instruction is received;
S202: judged in disk according to rule instruction with the presence or absence of disk B+ tree;If so, S203 is executed, if it is not, then holding
Row S204;
S203: memory B+ tree is stored to disk, metadata rule is completed;
Specifically, disk B+ tree is not present in itself that is, in disk in the case of disk is empty, at this point, without executing magnetic
Memory B+ tree is directly stored to disk, metadata rule can be realized by the union operation of disk metadata and memory metadata.
S204: the metadata of the first leaf node of memory B+ tree is obtained;
S205: disk B+ leaf child node corresponding to each metadata is inquired, and determines memory B+ tree and disk B+ tree
Continuous overlay area;
S206: the metadata of metadata and disk B+ tree to the memory B+ tree in same continuous overlay area carries out
Merging treatment obtains new leaf node;
S207: judge whether the first leaf node of memory B+ tree is that memory B+ tree terminates leaf node;If it is not, then executing
S208, if so, executing S209;
S208: obtaining the metadata of the next leaf node of memory B+ tree, and return to S205, until traversing memory B+ tree eventually
Only leaf node;
S209: the metadata of each new leaf node is stored to disk, completes metadata rule.
For above-mentioned S201 and S204~S209 particular content referring to a upper embodiment, the present embodiment is herein not
It repeats again.
To solve the above problems, referring to FIG. 5, Fig. 5 is a kind of structure of metadata rule device provided herein
Schematic diagram, the metadata rule device can include:
Metadata obtains module 10, for obtaining the member of the first leaf node of memory B+ tree when receiving rule instruction
Data;
Continuous overlay area determining module 20, for inquiring the corresponding disk B+ leaf child node of each metadata,
Determine the continuous overlay area of memory B+ tree Yu disk B+ tree;
Metadata merging module 30, for the metadata and disk to the memory B+ tree in same continuous overlay area
The metadata of B+ tree merges processing, obtains new leaf node;
Judgment module 40 is terminated, for judging whether the first leaf node of memory B+ tree is that memory B+ tree terminates leaf section
Point;
Module 50 is looped through, if not being that memory B+ tree terminates leaf node for memory B+ the first leaf node of tree,
The metadata of the next leaf node of memory B+ tree is obtained, and returns to continuous overlay area determining module, until traversing memory B+
Tree terminates leaf node;
Metadata rule module 60 is completed metadata and is fallen for storing the metadata of each new leaf node to disk
Disk.
As a kind of preferred embodiment, which can also include pre- judgment module, for judging disk
In whether there is disk B+ tree;If it does not exist, then memory B+ tree is stored to disk, completes metadata rule;If it exists, then it holds
Row obtains the step of metadata of the first leaf node of memory B+ tree.
As a kind of preferred embodiment, above-mentioned metadata merging module 30 can be specifically used for passing through MERGING/SORTING ALGORITHM pair
The metadata of memory B+ tree in continuous overlay area and the metadata of disk B+ tree merge processing, obtain new leaf
Node.
As a kind of preferred embodiment, which can also include the first memory release module, and being used for will
The corresponding disk B+ leaf knot removal of the metadata of the upper leaf node of memory B+ tree.
As a kind of preferred embodiment, above-mentioned metadata rule module 60 can be specifically used for generating by each new leaf node
New memory B+ tree;New memory B+ tree is stored to disk, metadata rule is completed.
As a kind of preferred embodiment, which can also include the second memory release module, for deleting
Except memory B+ tree new in memory, memory release is completed.
Above method embodiment is please referred to for the introduction of device provided by the present application, this will not be repeated here by the application.
To solve the above problems, referring to FIG. 6, Fig. 6 is a kind of structure of metadata rule system provided herein
Schematic diagram, the metadata rule system can include:
Memory 1, for storing computer program;
Processor 2, realizes following steps when for executing computer program:
When receiving rule instruction, the metadata of the first leaf node of memory B+ tree is obtained;It is corresponding to each metadata
Disk B+ leaf child node is inquired, and determines the continuous overlay area of memory B+ tree Yu disk B+ tree;To in same continuous
The metadata of memory B+ tree in overlay area and the metadata of disk B+ tree merge processing, obtain new leaf node;Sentence
Whether disconnected the first leaf node of memory B+ tree is that memory B+ tree terminates leaf node;If it is not, then obtaining the next leaf of memory B+ tree
The metadata of node, and inquiry disk B+ leaf child node corresponding with each metadata is returned, determine memory B+ tree and disk B+
The step of continuous overlay area of tree, until traversing memory B+ tree terminates leaf node;By the metadata of each new leaf node
It stores to disk, completes metadata rule.
Above method embodiment is please referred to for the introduction of equipment provided by the present application, this will not be repeated here by the application.
To solve the above problems, present invention also provides a kind of computer readable storage medium, the computer-readable storage
It is stored with computer program on medium, can be realized when computer program is executed by processor following rapid:
When receiving rule instruction, the metadata of the first leaf node of memory B+ tree is obtained;It is corresponding to each metadata
Disk B+ leaf child node is inquired, and determines the continuous overlay area of memory B+ tree Yu disk B+ tree;To in same continuous
The metadata of memory B+ tree in overlay area and the metadata of disk B+ tree merge processing, obtain new leaf node;Sentence
Whether disconnected the first leaf node of memory B+ tree is that memory B+ tree terminates leaf node;If it is not, then obtaining the next leaf of memory B+ tree
The metadata of node, and inquiry disk B+ leaf child node corresponding with each metadata is returned, determine memory B+ tree and disk B+
The step of continuous overlay area of tree, until traversing memory B+ tree terminates leaf node;By the metadata of each new leaf node
It stores to disk, completes metadata rule.
The computer readable storage medium may include: USB flash disk, mobile hard disk, read-only memory (Read-Only
Memory, ROM), random access memory (Random Access Memory, RAM), magnetic or disk etc. is various to deposit
Store up the medium of program code.
Above method embodiment is please referred to for the introduction of computer readable storage medium provided by the present application, the application exists
This is not repeated them here.
Each embodiment is described in a progressive manner in specification, the highlights of each of the examples are with other realities
The difference of example is applied, the same or similar parts in each embodiment may refer to each other.For device disclosed in embodiment
Speech, since it is corresponded to the methods disclosed in the examples, so being described relatively simple, related place is referring to method part illustration
?.
Professional further appreciates that, unit described in conjunction with the examples disclosed in the embodiments of the present disclosure
And algorithm steps, can be realized with electronic hardware, computer software, or a combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software generally describes each exemplary composition and step according to function in the above description.These
Function is implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Profession
Technical staff can use different methods to achieve the described function each specific application, but this realization is not answered
Think beyond scope of the present application.
The step of method described in conjunction with the examples disclosed in this document or algorithm, can directly be held with hardware, processor
The combination of capable software module or the two is implemented.Software module can be placed in random access memory (RAM), memory, read-only deposit
Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology
In any other form of storage medium well known in field.
Above to metadata rule method, apparatus, system and computer readable storage medium provided herein into
It has gone and has been discussed in detail.Specific examples are used herein to illustrate the principle and implementation manner of the present application, the above implementation
The explanation of example is merely used to help understand the present processes and its core concept.It should be pointed out that for the general of the art
For logical technical staff, under the premise of not departing from the application principle, can also to the application, some improvement and modification can also be carried out, this
A little improvement and modification also fall into the protection scope element of the claim of this application.
Claims (10)
1. a kind of metadata rule method characterized by comprising
When receiving rule instruction, the metadata of the first leaf node of memory B+ tree is obtained;
The corresponding disk B+ leaf child node of each metadata is inquired, determines the memory B+ tree and the disk B+
The continuous overlay area of tree;
The metadata of metadata and the disk B+ tree to the memory B+ tree in same continuous overlay area carries out
Merging treatment obtains new leaf node;
Judge whether the first leaf node of the memory B+ tree is that memory B+ tree terminates leaf node;
If it is not, then obtaining the metadata of the next leaf node of memory B+ tree, and return to the inquiry and each metadata
Corresponding disk B+ leaf child node, the step of determining the continuous overlay area of the memory B+ tree and the disk B+ tree, directly
Leaf node is terminated to the memory B+ tree is traversed;
The metadata of each new leaf node is stored to disk, metadata rule is completed.
2. metadata rule method as described in claim 1, which is characterized in that acquisition memory B+ first leaf node of tree
Metadata before, further includes:
Judge in the disk with the presence or absence of the disk B+ tree;
If it does not exist, then the memory B+ tree is stored to the disk, completes metadata rule;
If it exists, then the step of executing the metadata for obtaining the first leaf node of memory B+ tree.
3. metadata rule method as described in claim 1, which is characterized in that described in the continuous overlay area
The corresponding disk B+ leaf child node of the metadata of memory B+ tree is identical or adjacent.
4. the metadata rule method as described in claims 1 to 3 any one, which is characterized in that described pair is in the company
The metadata of the memory B+ tree and the metadata of the disk B+ tree in continuous overlay area merge processing, obtain young leaves
Child node, comprising:
Metadata and the disk B by MERGING/SORTING ALGORITHM to the memory B+ tree in the continuous overlay area
The metadata of+tree merges processing, obtains the new leaf node.
5. metadata rule method as claimed in claim 4, which is characterized in that described to obtain the next leaf of memory B+ tree
Before the metadata of node, further includes:
By the corresponding disk B+ leaf knot removal of the metadata of the upper leaf node of memory B+ tree.
6. metadata rule method as claimed in claim 5, which is characterized in that first number by each new leaf node
According to storing to disk, metadata rule is completed, comprising:
New memory B+ tree is generated by each new leaf node;
The new memory B+ tree is stored to the disk, metadata rule is completed.
7. metadata rule method as claimed in claim 6, which is characterized in that after the completion metadata rule, also wrap
It includes:
New memory B+ tree described in memory is deleted, memory release is completed.
8. a kind of metadata rule device, which is characterized in that further include:
Metadata obtains module, for obtaining the metadata of the first leaf node of memory B+ tree when receiving rule instruction;
Continuous overlay area determining module, for inquiring the corresponding disk B+ leaf child node of each metadata, really
The continuous overlay area of fixed the memory B+ tree and the disk B+ tree;
Metadata merging module, for the memory B+ tree in the same continuous overlay area metadata and the magnetic
The metadata of PanB+Shu merges processing, obtains new leaf node;
Judgment module is terminated, for judging whether the first leaf node of the memory B+ tree is that memory B+ tree terminates leaf node;
Module is looped through, if not being that the memory B+ tree terminates leaf node for the first leaf node of the memory B+ tree,
The metadata of the next leaf node of memory B+ tree is then obtained, and returns to the continuous overlay area determining module, until time
It goes through to the memory B+ tree and terminates leaf node;
Metadata rule module completes metadata rule for storing the metadata of each new leaf node to disk.
9. a kind of metadata rule system characterized by comprising
Memory, for storing computer program;
Processor realizes metadata rule as claimed in any one of claims 1 to 7 when for executing the computer program
The step of method.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program, the computer program realize metadata rule side as claimed in any one of claims 1 to 7 when being executed by processor
The step of method.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910429310.8A CN110147204B (en) | 2019-05-22 | 2019-05-22 | Metadata disk-dropping method, device and system and computer-readable storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910429310.8A CN110147204B (en) | 2019-05-22 | 2019-05-22 | Metadata disk-dropping method, device and system and computer-readable storage medium |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110147204A true CN110147204A (en) | 2019-08-20 |
CN110147204B CN110147204B (en) | 2020-03-10 |
Family
ID=67592733
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910429310.8A Active CN110147204B (en) | 2019-05-22 | 2019-05-22 | Metadata disk-dropping method, device and system and computer-readable storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110147204B (en) |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110704244A (en) * | 2019-09-29 | 2020-01-17 | 苏州浪潮智能科技有限公司 | Disk offline processing method, device and equipment and readable storage medium |
CN110989936A (en) * | 2019-12-06 | 2020-04-10 | 深信服科技股份有限公司 | Storage space allocation method and device and storage medium |
CN111309258A (en) * | 2020-02-14 | 2020-06-19 | 苏州浪潮智能科技有限公司 | B + tree access method and device and computer readable storage medium |
CN111813603A (en) * | 2020-07-16 | 2020-10-23 | 济南浪潮数据技术有限公司 | Method, device and equipment for backing up metadata of thin volume and readable storage medium |
CN113901276A (en) * | 2021-09-30 | 2022-01-07 | 苏州浪潮智能科技有限公司 | Data management method, B + tree loading method, device and electronic equipment |
CN116069267A (en) * | 2023-04-06 | 2023-05-05 | 苏州浪潮智能科技有限公司 | Writing cache method, system, equipment and storage medium of RAID card |
CN116431080A (en) * | 2023-06-09 | 2023-07-14 | 苏州浪潮智能科技有限公司 | Data disc-dropping method, system, equipment and computer readable storage medium |
CN116501267A (en) * | 2023-06-27 | 2023-07-28 | 苏州浪潮智能科技有限公司 | Independent redundant disk array card control method and device |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105487820A (en) * | 2015-11-30 | 2016-04-13 | 中国科学院信息工程研究所 | Time slice rotation mechanism based tree storage structure write amplification optimization method |
CN108319625A (en) * | 2017-01-17 | 2018-07-24 | 广州市动景计算机科技有限公司 | Piece file mergence method and apparatus |
US20180253255A1 (en) * | 2017-03-01 | 2018-09-06 | Tintri Inc. | Efficient deduplication for storage systems |
CN109033365A (en) * | 2018-07-26 | 2018-12-18 | 郑州云海信息技术有限公司 | A kind of data processing method and relevant device |
CN109189343A (en) * | 2018-09-14 | 2019-01-11 | 郑州云海信息技术有限公司 | A kind of metadata rule method, apparatus, equipment and computer readable storage medium |
CN109213445A (en) * | 2018-08-23 | 2019-01-15 | 郑州云海信息技术有限公司 | A kind of management method, management system and the relevant apparatus of storage system metadata |
CN109388346A (en) * | 2018-10-15 | 2019-02-26 | 郑州云海信息技术有限公司 | A kind of data rule method and relevant apparatus |
CN109558084A (en) * | 2018-11-29 | 2019-04-02 | 文华学院 | A kind of data processing method and relevant device |
CN109614045A (en) * | 2018-12-06 | 2019-04-12 | 广东浪潮大数据研究有限公司 | A kind of metadata rule method, apparatus and relevant device |
US10268403B1 (en) * | 2015-10-29 | 2019-04-23 | Pure Storage, Inc. | Combining multiple copy operations into a single copy operation |
-
2019
- 2019-05-22 CN CN201910429310.8A patent/CN110147204B/en active Active
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10268403B1 (en) * | 2015-10-29 | 2019-04-23 | Pure Storage, Inc. | Combining multiple copy operations into a single copy operation |
CN105487820A (en) * | 2015-11-30 | 2016-04-13 | 中国科学院信息工程研究所 | Time slice rotation mechanism based tree storage structure write amplification optimization method |
CN108319625A (en) * | 2017-01-17 | 2018-07-24 | 广州市动景计算机科技有限公司 | Piece file mergence method and apparatus |
US20180253255A1 (en) * | 2017-03-01 | 2018-09-06 | Tintri Inc. | Efficient deduplication for storage systems |
CN109033365A (en) * | 2018-07-26 | 2018-12-18 | 郑州云海信息技术有限公司 | A kind of data processing method and relevant device |
CN109213445A (en) * | 2018-08-23 | 2019-01-15 | 郑州云海信息技术有限公司 | A kind of management method, management system and the relevant apparatus of storage system metadata |
CN109189343A (en) * | 2018-09-14 | 2019-01-11 | 郑州云海信息技术有限公司 | A kind of metadata rule method, apparatus, equipment and computer readable storage medium |
CN109388346A (en) * | 2018-10-15 | 2019-02-26 | 郑州云海信息技术有限公司 | A kind of data rule method and relevant apparatus |
CN109558084A (en) * | 2018-11-29 | 2019-04-02 | 文华学院 | A kind of data processing method and relevant device |
CN109614045A (en) * | 2018-12-06 | 2019-04-12 | 广东浪潮大数据研究有限公司 | A kind of metadata rule method, apparatus and relevant device |
Cited By (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110704244A (en) * | 2019-09-29 | 2020-01-17 | 苏州浪潮智能科技有限公司 | Disk offline processing method, device and equipment and readable storage medium |
CN110704244B (en) * | 2019-09-29 | 2023-01-10 | 苏州浪潮智能科技有限公司 | Disk offline processing method, device and equipment and readable storage medium |
CN110989936A (en) * | 2019-12-06 | 2020-04-10 | 深信服科技股份有限公司 | Storage space allocation method and device and storage medium |
CN111309258A (en) * | 2020-02-14 | 2020-06-19 | 苏州浪潮智能科技有限公司 | B + tree access method and device and computer readable storage medium |
US11762827B2 (en) | 2020-02-14 | 2023-09-19 | Inspur Suzhou Intelligent Technology Co., Ltd. | B-plus tree access method and apparatus, and computer-readable storage medium |
CN111309258B (en) * | 2020-02-14 | 2021-10-15 | 苏州浪潮智能科技有限公司 | B + tree access method and device and computer readable storage medium |
CN111813603B (en) * | 2020-07-16 | 2022-06-10 | 济南浪潮数据技术有限公司 | Method, device and equipment for backing up metadata of thin volume and readable storage medium |
CN111813603A (en) * | 2020-07-16 | 2020-10-23 | 济南浪潮数据技术有限公司 | Method, device and equipment for backing up metadata of thin volume and readable storage medium |
CN113901276A (en) * | 2021-09-30 | 2022-01-07 | 苏州浪潮智能科技有限公司 | Data management method, B + tree loading method, device and electronic equipment |
CN116069267A (en) * | 2023-04-06 | 2023-05-05 | 苏州浪潮智能科技有限公司 | Writing cache method, system, equipment and storage medium of RAID card |
CN116431080A (en) * | 2023-06-09 | 2023-07-14 | 苏州浪潮智能科技有限公司 | Data disc-dropping method, system, equipment and computer readable storage medium |
CN116431080B (en) * | 2023-06-09 | 2023-08-29 | 苏州浪潮智能科技有限公司 | Data disc-dropping method, system, equipment and computer readable storage medium |
CN116501267A (en) * | 2023-06-27 | 2023-07-28 | 苏州浪潮智能科技有限公司 | Independent redundant disk array card control method and device |
CN116501267B (en) * | 2023-06-27 | 2023-09-29 | 苏州浪潮智能科技有限公司 | Independent redundant disk array card control method and device |
Also Published As
Publication number | Publication date |
---|---|
CN110147204B (en) | 2020-03-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110147204A (en) | A kind of metadata rule method, apparatus, system and computer readable storage medium | |
US10120869B2 (en) | Method and apparatus for fault-tolerant memory management | |
CN111190904B (en) | Method and device for hybrid storage of graph-relational database | |
US11392314B2 (en) | Sequentially writing metadata into a solid state disk by redirect-on-write | |
WO2021208239A1 (en) | Low-latency file system address space management method and system, and medium | |
CN109359062A (en) | A kind of metadata read buffer method, device and equipment | |
CN109857803A (en) | Method of data synchronization, device, equipment, system and computer readable storage medium | |
CN109471843A (en) | A kind of metadata cache method, system and relevant apparatus | |
CN107766355B (en) | Hierarchical data management method, hierarchical data management system and instant messaging system | |
CN109213445A (en) | A kind of management method, management system and the relevant apparatus of storage system metadata | |
CN104346404B (en) | A kind of method, equipment and system for accessing data | |
CN110795042A (en) | Method for writing and flushing metadata of full flash memory storage system and related components | |
CN110399096A (en) | Metadata of distributed type file system caches the method, apparatus and equipment deleted again | |
CN115576501A (en) | Node updating method, system and related device of RAID card | |
CN109086462A (en) | The management method of metadata in a kind of distributed file system | |
CN104424222A (en) | Database indexing method and device | |
CN110020272A (en) | Caching method, device and computer storage medium | |
KR100907477B1 (en) | Apparatus and method for managing index of data stored in flash memory | |
CN106528703A (en) | Deduplication mode switching method and apparatus | |
CN109753446A (en) | Data cached method is brushed under a kind of storage system | |
CN103714121B (en) | The management method and device of a kind of index record | |
WO2024119797A1 (en) | Data processing method and system, device, and storage medium | |
CN103348653A (en) | Capacity expansion method and device and data access method and device | |
CN105045891A (en) | Method and system for improving performance of sequence list, architecture, optimization method and storage apparatus | |
CN111625198A (en) | Metadata caching method and metadata caching device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |