CN106777062A - A kind of method and device for managing metadata - Google Patents
A kind of method and device for managing metadata Download PDFInfo
- Publication number
- CN106777062A CN106777062A CN201611139129.6A CN201611139129A CN106777062A CN 106777062 A CN106777062 A CN 106777062A CN 201611139129 A CN201611139129 A CN 201611139129A CN 106777062 A CN106777062 A CN 106777062A
- Authority
- CN
- China
- Prior art keywords
- packet
- virtual directory
- metadata
- virtual
- currently
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/188—Virtual file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/119—Details of migration of file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
- G06F16/164—File meta data generation
Abstract
The invention provides a kind of method and device for managing metadata, the method includes:For corresponding at least two packet of catalogue, the corresponding mapping range of each packet is determined;The corresponding metadata of each file is stored into packet corresponding with the mapping range that it matches respectively;When the memory capacity for monitoring catalogue is not located in preset threshold range, the virtual directory of the first quantity is set up, and according to packet migration rules, corresponding migration process is carried out for the whole packets that there is currently.Based on packet to store magnanimity metadata respectively, therefore when the memory capacity of catalogue transfinites, can be by way of increasing virtual directory and migrating packet, with by the grouped storage metadata of virtual directory, such that it is able to solve the problems, such as that catalogue memory capacity transfinites.Therefore, this programme can improve the recall precision of metadata.
Description
Technical field
The present invention relates to field of computer technology, more particularly to a kind of method and device for managing metadata.
Background technology
As the development in big data epoch, non-structured data are in explosive growth, data yield increasingly explodes.Example
Such as, large-scale sensor can produce the data slot of magnanimity, so that the huge small data file of quantity of formation.Wherein it is possible to adopt
The mode of data is preserved to store the mass data of generation with file, in order to data sharing and management.
At present, for the corresponding metadata of file, the metadata of magnanimity can be stored in the single mesh of file system
In record.
But, with being constantly stored in for each metadata, the quantity of metadata is easily excessive in single catalogue.Therefore work as and require to look up
When file destination is retrieved with carrying out respective meta-data, existing metadata management mode can reduce the recall precision of metadata.
The content of the invention
The invention provides a kind of method and device for managing metadata, it is possible to increase the recall precision of metadata.
In order to achieve the above object, the present invention is achieved through the following technical solutions:
On the one hand, the invention provides a kind of method for managing metadata, including:
S1:For corresponding at least two packet of catalogue, each corresponding mapping range of the packet is determined;
S2:It is performed both by for the corresponding metadata of each file:The metadata is stored to the mapping matched with it
In the corresponding packet of scope;
S3:When the memory capacity for monitoring the catalogue is not located in the range of first threshold set in advance, S4 is performed;
S4:The virtual directory of the first quantity is set up, and according to predetermined packet migration rules, for what be there is currently
All packet, carries out corresponding migration process.
Further, the title of this document is included in the corresponding metadata of each file;
The S2, including:It is performed both by for the corresponding metadata of each file:According to including file name
Claim, calculate the corresponding cryptographic Hash of the title;It is determined that the target mapping range matched with the cryptographic Hash, wherein, the Kazakhstan
Uncommon value is located in the target mapping range;The metadata is stored into the corresponding packet of the target mapping range.
Further, the memory capacity coincidence formula one of the catalogue, wherein,
The formula one includes:
Wherein, X is the memory capacity of the catalogue, and n is the number of at least two packet, NiIt is described at least two
The number of the metadata stored in i-th packet in packet.
Further, first quantity is by the migration process, so that any virtual directory that there is currently
Memory capacity is respectively positioned on the minimum number in the range of the first threshold.
Further, also include after S4:Monitoring that it is described that the memory capacity that there is any virtual directory is not located at
When in the range of first threshold, S4 is performed.
Further, the packet migration rules, including:Each order of packets arrangement in the whole packets that there is currently,
And in the whole virtual directories that there is currently each virtual directory order arrange when, according to formula two determine each virtual directory
The number of corresponding packet;
When the number of the whole virtual directories that there is currently is first quantity, according to each the virtual mesh determined
The number of corresponding packet is recorded, described at least two packets are migrated into each virtual directory that there is currently respectively, its
In, for any two virtual directory, when second virtual directory that is located therein of putting in order of the first virtual directory therein
Put in order after when, the putting in order of each packet in first virtual directory is respectively positioned on each in second virtual directory
Packet put in order after;
When the number of the whole virtual directories that there is currently not is first quantity, the arrangement based on each virtual directory is suitable
Sequence, is performed both by for each virtual directory successively:The number for calculating the current corresponding packet of the virtual directory is subtracted and determined
The corresponding packet of the virtual directory number difference;When it is positive number to judge the difference, the virtual directory pair is determined
The packet to be migrated answered, wherein, in the current corresponding all packets of the virtual directory, the position that puts in order of the packet to be migrated
After other packets, and the number of the packet to be migrated is equal to the difference;The packet to be migrated is migrated to order
It is arranged in the next virtual directory after the virtual directory;
The formula two includes:
Wherein, X is the number of the whole packets that there is currently;N is the number of the whole virtual directories that there is currently;x1For
In the whole virtual directories that there is currently, the number of the corresponding packet of any virtual directory in addition to last position virtual directory, x2For
The number of the corresponding packet of the last position virtual directory.
Further, also include after S4:Monitoring the last position virtual directory and virtual directory adjacent thereto
Packet it is unbalanced degree be located at Second Threshold set in advance in the range of when, set up correspond to the last position virtual directory extremely
A few packet, so that the unbalanced degree of the packet is located in the range of the Second Threshold;For the whole packet that there is currently,
Redefine each and be grouped corresponding mapping range;It is performed both by for the corresponding metadata of each file:Judge
Whether packet that the metadata is currently located and packet corresponding with the mapping range that it matches are identical, if it is not, by this yuan of number
According to migrating into packet corresponding with the mapping range that it matches.
Further, the number of at least one packet of the foundation is to meet described so that the unbalanced degree of packet
Minimum number in the range of the Second Threshold.
Further, the unbalanced degree coincidence formula three of the packet, wherein,
The formula three includes:
Wherein, Y is the unbalanced degree of packet, XnIt is the number of the corresponding packet of the last position virtual directory, XmIt is and institute
State the number of the adjacent corresponding packet of virtual directory of last position virtual directory.
On the other hand, the invention provides a kind of device for managing metadata, including:
Determining unit, for for corresponding at least two packet of catalogue, determining each corresponding mapping of the packet
Scope;
Map unit, for being performed both by for the corresponding metadata of each file:By the metadata store to its phase
In the corresponding packet of mapping range of matching, and trigger the first monitoring unit;
First monitoring unit, for not being located at the first threshold set in advance in the memory capacity for monitoring the catalogue
When in the range of value, processing unit is triggered;
The processing unit, the virtual directory for setting up the first quantity, and according to predetermined packet migration rules,
For the whole packet that there is currently, corresponding migration process is carried out.
Further, the title of this document is included in the corresponding metadata of each file;
The map unit, specifically for being performed both by for the corresponding metadata of each file:According to including
The title of file, calculates the corresponding cryptographic Hash of the title;It is determined that the target mapping range matched with the cryptographic Hash, its
In, the cryptographic Hash is located in the target mapping range;The metadata is stored to corresponding point of the target mapping range
In group.
Further, the memory capacity coincidence formula one of the catalogue, wherein,
The formula one includes:
Wherein, X is the memory capacity of the catalogue, and n is the number of at least two packet, NiIt is described at least two
The number of the metadata stored in i-th packet in packet.
Further, first quantity is by the migration process, so that any virtual directory that there is currently
Memory capacity is respectively positioned on the minimum number in the range of the first threshold.
Further, first monitoring unit, is additionally operable in the memory capacity for monitoring to have any virtual directory not
When in the range of the first threshold, the processing unit is triggered.
Further, the packet migration rules, including:Each order of packets arrangement in the whole packets that there is currently,
And in the whole virtual directories that there is currently each virtual directory order arrange when, according to formula two determine each virtual directory
The number of corresponding packet;
When the number of the whole virtual directories that there is currently is first quantity, according to each the virtual mesh determined
The number of corresponding packet is recorded, described at least two packets are migrated into each virtual directory that there is currently respectively, its
In, for any two virtual directory, when second virtual directory that is located therein of putting in order of the first virtual directory therein
Put in order after when, the putting in order of each packet in first virtual directory is respectively positioned on each in second virtual directory
Packet put in order after;
When the number of the whole virtual directories that there is currently not is first quantity, the arrangement based on each virtual directory is suitable
Sequence, is performed both by for each virtual directory successively:The number for calculating the current corresponding packet of the virtual directory is subtracted and determined
The corresponding packet of the virtual directory number difference;When it is positive number to judge the difference, the virtual directory pair is determined
The packet to be migrated answered, wherein, in the current corresponding all packets of the virtual directory, the position that puts in order of the packet to be migrated
After other packets, and the number of the packet to be migrated is equal to the difference;The packet to be migrated is migrated to order
It is arranged in the next virtual directory after the virtual directory;
The formula two includes:
Wherein, X is the number of the whole packets that there is currently;N is the number of the whole virtual directories that there is currently;x1For
In the whole virtual directories that there is currently, the number of the corresponding packet of any virtual directory in addition to last position virtual directory, x2For
The number of the corresponding packet of the last position virtual directory.
Further, the device of the management metadata also includes:Unit is set up in second monitoring unit, packet;
Second monitoring unit, for monitor the last position virtual directory and virtual directory adjacent thereto point
The unbalanced degree of group is not located at when in the range of Second Threshold set in advance, is triggered the packet and is set up unit;
Unit is set up in the packet, for setting up at least one packet corresponding to the last position virtual directory, so that institute
State the unbalanced degree of packet to be located in the range of the Second Threshold, and trigger the determining unit;
The determining unit, be additionally operable to receive it is described packet set up the trigger signal that unit is sent when, for current
Whole packets of presence, redefine each and are grouped corresponding mapping range, and trigger the map unit;
The map unit, is additionally operable to when the trigger signal that the determining unit is sent is received, for each text
The corresponding metadata of part is performed both by:Judge packet that the metadata is currently located and corresponding with the mapping range that it matches point
Whether group is identical, if it is not, by the metadata migration to packet corresponding with the mapping range that it matches.
Further, the number that at least one packet of unit foundation is set up in the packet is to meet described so that described
Minimum number of the unbalanced degree of packet in the range of the Second Threshold.
Further, the unbalanced degree coincidence formula three of the packet, wherein,
The formula three includes:
Wherein, Y is the unbalanced degree of packet, XnIt is the number of the corresponding packet of the last position virtual directory, XmIt is and institute
State the number of the adjacent corresponding packet of virtual directory of last position virtual directory.
The invention provides a kind of method and device for managing metadata, it is grouped for catalogue corresponding at least two, really
Determine the corresponding mapping range of each packet;The corresponding metadata of each file is stored to the mapping range matched with it respectively
In corresponding packet;When the memory capacity for monitoring catalogue is not located in preset threshold range, the virtual of the first quantity is set up
Catalogue, and according to packet migration rules, corresponding migration process is carried out for the whole packets that there is currently.Based on packet distinguishing
Storage magnanimity metadata, therefore when the memory capacity of catalogue transfinites, can by way of increasing virtual directory and migrating packet,
With by the grouped storage metadata of virtual directory, such that it is able to solve the problems, such as that catalogue memory capacity transfinites.Therefore, the present invention
The recall precision of metadata can be improved.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
The accompanying drawing to be used needed for having technology description is briefly described, it should be apparent that, drawings in the following description are the present invention
Some embodiments, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis
These accompanying drawings obtain other accompanying drawings.
Fig. 1 is a kind of flow chart of the method for management metadata that one embodiment of the invention is provided;
Fig. 2 is a kind of schematic diagram of the framework of management metadata that one embodiment of the invention is provided;
Fig. 3 is the flow chart of the method for another management metadata that one embodiment of the invention is provided;
Fig. 4 is a kind of schematic diagram of the device of management metadata that one embodiment of the invention is provided;
Fig. 5 is the schematic diagram of the device of another management metadata that one embodiment of the invention is provided.
Specific embodiment
To make the purpose, technical scheme and advantage of the embodiment of the present invention clearer, below in conjunction with the embodiment of the present invention
In accompanying drawing, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is
A part of embodiment of the present invention, rather than whole embodiments, based on the embodiment in the present invention, those of ordinary skill in the art
The every other embodiment obtained on the premise of creative work is not made, belongs to the scope of protection of the invention.
As shown in figure 1, the embodiment of the invention provides a kind of method for managing metadata, may comprise steps of:
Step 101:For corresponding at least two packet of catalogue, each corresponding mapping range of the packet is determined.
Step 102:It is performed both by for the corresponding metadata of each file:The metadata is stored to it and is matched
In the corresponding packet of mapping range.
Step 103:When the memory capacity for monitoring the catalogue is not located in the range of first threshold set in advance, hold
Row step 104.
Step 104:The virtual directory of the first quantity is set up, and according to predetermined packet migration rules, for current
Whole packets of presence, carry out corresponding migration process.
A kind of method for managing metadata is the embodiment of the invention provides, is grouped for catalogue corresponding at least two, really
Determine the corresponding mapping range of each packet;The corresponding metadata of each file is stored to the mapping range matched with it respectively
In corresponding packet;When the memory capacity for monitoring catalogue is not located in preset threshold range, the virtual of the first quantity is set up
Catalogue, and according to packet migration rules, corresponding migration process is carried out for the whole packets that there is currently.Based on packet distinguishing
Storage magnanimity metadata, therefore when the memory capacity of catalogue transfinites, can by way of increasing virtual directory and migrating packet,
With by the grouped storage metadata of virtual directory, such that it is able to solve the problems, such as that catalogue memory capacity transfinites.Therefore, the present invention
Embodiment can improve the recall precision of metadata.
In detail, for each file in file system, two parts, the respectively corresponding unit of file be may each comprise
The content of data and file.Wherein, in the embodiment of the present invention, the metadata of each file can be carried out by catalogue to unify pipe
Reason.
In detail, for the catalogue of file system, the catalogue to that should have multiple packets, and can respectively be grouped corresponding mapping
Scope can be stored in each metadata correspondence mappings to respective packets and is managed with difference such that it is able to make.
In detail, the corresponding metadata of file can include title, time, authority, extended attribute, the storage location of file
Deng.
Therefore, in an embodiment of the invention, in order to illustrate a kind of possibility reality that metadata is mapped to respective packets
Existing mode, so, the title of this document is included in the corresponding metadata of each file;
The step 102, including:It is performed both by for the corresponding metadata of each file:According to including file
Title, calculate the corresponding cryptographic Hash of the title;It is determined that the target mapping range matched with the cryptographic Hash, wherein, institute
Cryptographic Hash is stated to be located in the target mapping range;The metadata is stored into the corresponding packet of the target mapping range.
As an example it is assumed that the catalogue of file system is current to that should have 10 packets, and this 10 are grouped corresponding mapping
Scope be followed successively by [0,10], (10,20] ..., (90,100].Metadata corresponding for any file, according to including
This document title, its corresponding cryptographic Hash can be calculated according to the title.Assuming that for a certain metadata, the Kazakhstan for calculating
Uncommon value is 17, due to the 17 corresponding mapping ranges of second packet that can be fallen into above-mentioned 10 packets:(10,20], explanation
The metadata and mapping range:(10,20] match, therefore deposited in the metadata being mapped into this second packet
Storage.In this way, based on same realization principle, each metadata can be respectively mapped to the mapping range pair that matches with it
In the packet answered.
With the continuous generation of metadata, each metadata is constantly stored in respective packets, therefore in the corresponding each packet of catalogue
The quantity of the metadata for being stored is continuously increased.When incrementss reach to a certain degree, the memory capacity of catalogue can be caused to surpass
Limit.When the memory capacity of catalogue transfinites, the execution of the operations such as retrieval, the cache management of metadata is not conducive to, therefore can be by increasing
Plus the implementation of virtual directory shares the storage pressure of catalogue.
In detail, by increasing virtual directory, the corresponding all packets of catalogue can be migrated to newly-increased virtual directory
In, to alleviate the storage pressure of catalogue.Wherein, when the memory capacity of catalogue transfinites, if only increase a virtual directory, and will
The corresponding all packets of catalogue are migrated into the virtual directory, and the memory capacity for also resulting in the virtual directory transfinites.Cause
This, when the memory capacity of catalogue transfinites, that is, when needing to set up virtual directory first, acquiescence needs to increase by least two virtual mesh
Record.Therefore, the first quantity in above-mentioned steps 104 can be at least two.
For example, catalogue is current to that should have 10 packets, if the memory capacity of catalogue transfinites, can increase two virtual directories newly,
And migrate into two virtual directories this 10 packets respectively, so that each virtual directory both corresponds to 5 packets.Due to inciting somebody to action
Corresponding 10 packets of catalogue are moved out, and the storage pressure of catalogue can be made to alleviate significantly, so as to solve what its memory capacity transfinited
Problem.Meanwhile, the memory capacity of each virtual directory does not transfinite currently.
Accordingly, when setting up of any virtual directory is completed, the corresponding metadata of the virtual directory can be being generated.The present invention
In one embodiment, it is preferable that the corresponding metadata of the virtual directory of generation can be stored into catalogue, with by catalogue pair
Each virtual directory is managed collectively.Wherein, each virtual directory of foundation can be as the subdirectory of catalogue, with by mesh
Record is managed.
Based on the above, in an embodiment of the invention, it is preferable that first quantity is by the migration
Reason, so that the memory capacity of any virtual directory that there is currently is respectively positioned on the minimum number in the range of the first threshold.
For example, being constantly stored in metadata, it is assumed that the current memory capacity of catalogue has reached 120,000, and sets
Fixed first threshold scope is:≤ 10 ten thousand, illustrate that the current memory capacity of catalogue transfinites, therefore at least need to increase by two virtually
Catalogue.For example, increase by two virtual directories and perform packet migration with divide equally packet after, the memory capacity of each newly-increased virtual directory
60,000 are can be about, both memory capacity does not transfinite.
Under normal circumstances, the cloud data writing rate in practical application, when the memory capacity of catalogue transfinites, that is, works as head
Secondary when setting up virtual directory, the minimum number of virtual directory to be increased is usually 2.
Certainly, for some limit or special circumstances, it is assumed that the metadata amount of being stored at a certain moment is excessive, such as work as catalogue
Preceding memory capacity has reached 220,000, each new if equally increasing by two virtual directories and performing packet migration to divide packet equally
The memory capacity for increasing virtual directory can be about 110,000, and both memory capacity transfinites, or at least have one virtually
Catalogue transfinites.Therefore, the minimum number of virtual directory to be increased should be 3.
Therefore, based on the above as can be seen that when virtual directory is set up first, the minimum number of above-mentioned first quantity
It is 2.
Generally speaking, when the memory capacity that there is any virtual directory transfinites, the number of newly-established virtual directory should
After ensure that execution packet migration operation, the memory capacity of each virtual directory does not transfinite in file system.
In an embodiment of the invention, the memory capacity meets following formula (1);
Wherein, X is the memory capacity of the catalogue, and n is the number of at least two packet, NiIt is described at least two
The number of the metadata stored in i-th packet in packet.
As can be seen that the memory capacity of catalogue can be, the metadata of storage is individual in corresponding each packet of catalogue
It is several plus and.
Accordingly, based on same realization principle, same to above-mentioned formula (1), in an embodiment of the invention, for any
Virtual directory, the memory capacity of the virtual directory can be, the metadata stored in corresponding each packet of the virtual directory
Number plus and.
After newly-built virtual directory, magnanimity metadata can be stored by the way that virtual directory is grouped.Certainly, with
The continuous of newly-increased metadata is stored in, and the memory capacity of each virtual directory equally occurs overrun condition.
As shown in the above, when the memory capacity of catalogue transfinites, can be by increasing virtual directory and being grouped
Migrate to solve the memory capacity overrun issues of catalogue.Similarly, when the memory capacity that there is any virtual directory transfinites, equally
Can be by increasing virtual directory and carrying out packet migration to solve memory capacity overrun issues.Wherein, for depositing each time
Storage capacity overrun issues, can be with newly-built at least one virtual directory.In this way, when it is non-set up virtual directory first when, above-mentioned
The minimum number of one quantity is 1.
Therefore, in an embodiment of the invention, further include after step 104:Monitoring there is any void
The memory capacity for intending catalogue is not located at when in the range of the first threshold, performs step 104.
In detail, the memory capacity simply by the presence of any virtual directory transfinites, you can perform step 104, with newly-increased virtual
Catalogue simultaneously carries out packet migration, so that the memory capacity of each virtual directory does not transfinite.
In detail, for any virtual directory, by performing step 104, part therein packet can be moved out, to subtract
The number of few its corresponding packet, therefore its memory capacity can be reduced, so as to solve the problems, such as that virtual directory memory capacity transfinites.
In an embodiment of the invention, in order to illustrate a kind of possible packet migration implementation, so, the packet
Migration rules, including:Each order of packets arrangement in the whole packets that there is currently, and in the whole virtual directories that there is currently
Each virtual directory order arrange when, the number of the corresponding packet of each virtual directory is determined according to following formula (2);
When the number of the whole virtual directories that there is currently is first quantity, according to each the virtual mesh determined
The number of corresponding packet is recorded, described at least two packets are migrated into each virtual directory that there is currently respectively, its
In, for any two virtual directory, when second virtual directory that is located therein of putting in order of the first virtual directory therein
Put in order after when, the putting in order of each packet in first virtual directory is respectively positioned on each in second virtual directory
Packet put in order after;
When the number of the whole virtual directories that there is currently not is first quantity, the arrangement based on each virtual directory is suitable
Sequence, is performed both by for each virtual directory successively:The number for calculating the current corresponding packet of the virtual directory is subtracted and determined
The corresponding packet of the virtual directory number difference;When it is positive number to judge the difference, the virtual directory pair is determined
The packet to be migrated answered, wherein, in the current corresponding all packets of the virtual directory, the position that puts in order of the packet to be migrated
After other packets, and the number of the packet to be migrated is equal to the difference;The packet to be migrated is migrated to order
It is arranged in the next virtual directory after the virtual directory;
Wherein, X is the number of the whole packets that there is currently;N is the number of the whole virtual directories that there is currently;x1For
In the whole virtual directories that there is currently, the number of the corresponding packet of any virtual directory in addition to last position virtual directory, x2For
The number of the corresponding packet of the last position virtual directory.
In an embodiment of the invention, Fig. 2 can be expressed as a kind of framework for managing metadata.The framework can be represented
It is to be currently included 20 packets, virtual directory 1 and virtual directories 2 under the catalogue of file system, and catalogue.Wherein, virtual mesh
Record 1 corresponds to packet 1~packet 10, and virtual directory 2 corresponds to packet 11~packet 20.Additionally, for each packet, wherein
Be stored with the metadata (not shown in fig. 2) matched with it.
In detail, for above-mentioned packet migration rules, Fig. 2 is refer to, for example:Assuming that being deposited in the catalogue of file system
In 20 packets, 1~packet 20 is respectively grouped, and this 20 packets initially both correspond to catalogue.When the memory capacity of catalogue
When transfiniting, it is generally the case that 2 virtual directories can be set up:Virtual directory 1 and virtual directory 2.In detail, for foundation
Each virtual directory, is managed collectively by catalogue.
In fig. 2, above-mentioned 20 order of packets arrangement, virtual directory 1 is the first virtual directory, and virtual directory 2 is order
The second virtual directory of arrangement.According to above-mentioned formula (2), grouping strategy can be:19 packets of the correspondence of virtual directory 1 and void
Intend catalogue 2 correspondence 1 packet, virtual directory 1 correspondence 18 packet and virtual directory 2 correspondence 2 packet ..., virtual directory 1
10 packets of 10 packets of correspondence and the correspondence of virtual directory 2.
Wherein, because the metadata quantity stored in each packet is generally more or less the same, therefore no matter any of the above-described packet is used
Strategy, after can ensureing packet migration, the memory capacity of virtual directory 1 and virtual directory 2 does not transfinite.But, when both are right
When the packet number answered has big difference, both unbalanced degree of packet are higher, can influence the metadata management of whole file system
Stability.
It is therefore preferred that can as much as possible choose both minimum grouping strategies of corresponding packet number difference, i.e., it is empty
Intend catalogue 1 and virtual directory 2 and correspond to 10 packets.Certainly, based on different practical application requests, other can equally be used
Grouping strategy, so that after packet migration, the memory capacity of each virtual directory does not transfinite.
10 grouping strategies of packet are corresponded to for virtual directory 1 and virtual directory 2,1~packet 10 can will be grouped,
Migrated from catalogue into virtual directory 1, and 11~packet 20 will be grouped, migrated from catalogue into virtual directory 2.Migration
Situation afterwards can correspond to Fig. 2.
In this way, there is currently two virtual directories, i.e. virtual directory 1 and virtual directory 2 in file system.With first number
According to constantly be stored in, when the memory capacity that there is any virtual directory transfinites, can again perform step 104.In this way, can be with
A virtual directory is set up again:Virtual directory 3, virtual directory 3 is that the 3rd be sequentially arranged in after virtual directory 2 is virtual
Catalogue.According to above-mentioned formula (2), grouping strategy can be:Tactic three virtual directories are corresponding to be grouped number successively
It is 9,9 and 2, or is followed successively by 8,8 and 4, or is followed successively by 7,7 and 6.
In the embodiment of the present invention, it is contemplated that improve the unbalanced degree of packet between different virtual directories as far as possible, it may be preferred to
The corresponding number that is grouped of tactic three virtual directories is followed successively by 7,7 and 6.
In this way, putting in order based on each virtual directory, firstly for the first virtual directory:Virtual directory 1, it is current right
That answers is grouped into packet 1~packet 10, and number is 10, and its corresponding packet number determined is 7, and both are at difference
3, therefore there are 3 packets to be migrated during virtual directory 1 can be determined:Packet 8~packet 10, therefore can be to be migrated by this 3
Packet is migrated to second virtual directory from virtual directory 1:In virtual directory 2.
Then, for second virtual directory:Virtual directory 2, due to having completed moving to packet 8~packet 10 at present
Move, thus virtual directory 2 it is current it is corresponding be grouped into packet 8~packet 20, number is 13, and its corresponding packet determined
Number is 7, and both differences are 6, therefore there are 6 packets to be migrated in can determining virtual directory 2:Packet 15~packet
20, therefore this 6 packets to be migrated can be migrated to the 3rd virtual directory from virtual directory 2:In virtual directory 3.
Subsequently, for the 3rd virtual directory:Virtual directory 3, due to having completed moving to packet 15~packet 20 at present
Move, thus virtual directory 3 it is current it is corresponding be grouped into packet 15~packet 20, number is 6, and its corresponding packet determined
Number is 6, and both differences are 0, therefore does not exist packet to be migrated in can determining virtual directory 3, therefore can terminate this
Packet migration flow.
It is same as above, with being constantly stored in for metadata, when the memory capacity that there is any virtual directory in file system
When transfiniting, step 104 can be again performed.In this way, a virtual directory can be set up:Virtual directory 4.Then, complete to divide
After the execution of group migration, virtual directory 1 can correspond to packet 1~packet 5, and virtual directory 2 can correspond to packet 6~packet
10, virtual directory 3 can correspond to packet 11~packet 15, and virtual directory 4 can correspond to packet 16~packet 20.
So circulation, for the whole virtual directories that there is currently, as long as monitoring there is the storage of any virtual directory
Capacity transfinites, and can repeat step 104, is not transfinited with the memory capacity for ensureing any virtual directory, so as to ensure
The smooth execution of the operations such as retrieval, the cache management of metadata.
Based on the above, after packet migration is completed, the packet of last position virtual directory and virtual directory adjacent thereto
Number may be different.For example, when there is currently three virtual directories, the corresponding packet number of three virtual directories can be successively
It is 7,7 and 6.Due to when the unbalanced degree of the packet between two virtual directories of presence is higher, whole file system can be influenceed
The metadata management stability of system, therefore can be adjusted with to the unbalanced degree of packet higher by way of adding packet.
Therefore, in an embodiment of the invention, in order to illustrate a kind of possibility implementation for adjusting the unbalanced degree of packet,
So, further include after step 104:Monitor the last position virtual directory and virtual directory adjacent thereto point
The unbalanced degree of group is not located at when in the range of Second Threshold set in advance, sets up at least corresponding to the last position virtual directory
Individual packet, so that the unbalanced degree of the packet is located in the range of the Second Threshold;It is grouped for the whole that there is currently, again
Determine that each is grouped corresponding mapping range;It is performed both by for the corresponding metadata of each file:Judge this yuan
Whether the packet and packet corresponding with the mapping range that it matches that data are currently located are identical, if it is not, the metadata is moved
In moving to packet corresponding with the mapping range that it matches.
Based on the above, for example, when there is currently three virtual directories, the corresponding packet of three virtual directories
Number can be followed successively by 7,7 and 6.The first two virtual group both corresponds to 7 packets, therefore the unbalanced degree of its packet is closed
Lattice, and latter two virtual group there may be packet it is unbalanced.In this way, a packet can be increased newly, and the newly-increased packet is right
Ying Yumo virtual directory, so that last position virtual directory is same to that should have 7 packets, to adjust its dividing with adjacent virtual catalogue
The unbalanced degree of group.
For newly-established packet, it is thus necessary to determine that its corresponding mapping range, therefore need currently to be deposited in adjustment file system
Each packet mapping range.For example, there is currently 20 packets, 1~packet 20, and this 20 points are respectively grouped
The corresponding mapping range of group be followed successively by [0,10], (10,20] ..., (190,200].After a newly-increased packet, there is currently
21 packets, therefore [0,200] this total mapping range can be redistributed.
For example, this 21 corresponding mapping ranges of packet can be followed successively by [0,10], (10,20] ..., (100,110],
(110,119], (119,128] ..., (191,200].As can be seen that the mapping range of packet 1~packet 11 does not change, and
Packet 12 to packet 20 mapping range change, such as be grouped 13 mapping range by (120,130] be changed to (119,
128], and a newly-increased order of packets is used as packet 21, can to should have mapping range (191,200].
By taking above-mentioned packet 13 as an example, it is assumed that for a certain metadata, cryptographic Hash can be calculated for 129,129 can fall into
(120,130] in, therefore the metadata can be mapped to and be grouped in 13.But, when packet 13 mapping range be changed into (119,
128] still it is 129 because the cryptographic Hash for calculating is constant after, therefore can be seen that 129 can fall into corresponding to packet 14
Mapping range (128,137] in, therefore can determine that the metadata corresponds to packet 14.In this way, because the metadata is currently located at
In packet 13, and determine be grouped into packet 14, both are inconsistent, therefore the metadata 13 can be migrated to packet from packet
In 14.
Based on same realization principle, corresponding migration operation can be carried out to each metadata to be migrated.In this way, new
The packet of foundation:Can also be stored with a number of metadata in packet 21, such that it is able to make the difference in current all packets
The metadata quantity stored in packet is more or less the same, so that the memory capacity of each virtual directory can not only be alleviated, while can
Reduce the unbalanced degree of packet between different virtual directories.
Certainly, for each new metadata, can be according to the current mapping range of each packet, by each new metadata
Map to during correspondence is grouped and stored.
In an embodiment of the invention, the number of at least one packet of the foundation is to meet described so that described
Minimum number of the unbalanced degree of packet in the range of the Second Threshold.
As an example it is assumed that virtual directory corresponding packet number in last position is 8, virtual directory adjacent thereto is corresponding
Packet number is 10, and both unbalanced degree of packet are 20%, are unsatisfactory for the Second Threshold scope of setting:≤ 10%, therefore can
To increase by 1 packet or 2 packets, accordingly, so that virtual directory corresponding packet number in last position is 9 or 10, so that
The unbalanced degree of packet for making both is 10% or 0%, to meet Second Threshold scope:≤ 10%.In this way, due to setting up 1 or 2
It is individual, therefore minimum value can be taken, that is, set up 1 packet.
In an embodiment of the invention, in order to illustrate a kind of possibility implementation for calculating the unbalanced degree of packet, so,
The unbalanced degree of packet meets following formula (3);
Wherein, Y is the unbalanced degree of packet, XnIt is the number of the corresponding packet of the last position virtual directory, XmIt is and institute
State the number of the adjacent corresponding packet of virtual directory of last position virtual directory.
In detail, it is grouped that unbalanced degree is lower, illustrates the number of the corresponding packet of two virtual directories closer to more beneficial
In the metadata management stability for ensureing whole file system.
In sum, the embodiment of the present invention proposes a kind of method for managing metadata, by real-time dynamic newly-built void
Intend catalogue, by the corresponding different grouping of each virtual directory, to be managed to magnanimity metadata, it is to avoid all metadata are equal
All the time it is stored under catalogue, such that it is able to improve the recall precision to magnanimity metadata.The memory capacity of each virtual directory transfinites
When, can be solved by the implementation of newly-built virtual directory and migration packet, and be grouped between different virtual directories uneven
During weighing apparatus, can by it is newly-built packet and migration metadata implementation be solved so that realize whole file system according to
Practical application carries out dynamic expansion.
As shown in figure 3, the method that one embodiment of the invention provides another management metadata, specifically includes following step
Suddenly:
Step 301:For corresponding 20 packets of catalogue, each corresponding mapping range of packet is determined.
In detail, tactic 20 packets are there is currently in file system, and both corresponds to catalogue, be respectively grouped
1~packet 20, and this 20 corresponding mapping ranges of packet be followed successively by [0,10], (10,20] ..., (190,200].
Step 302:It is performed both by for the corresponding metadata of each file:According to including file title, meter
Calculate the corresponding cryptographic Hash of title;It is determined that the target mapping range matched with cryptographic Hash, wherein, the cryptographic Hash maps positioned at target
In the range of;The metadata is stored into the corresponding packet of target mapping range.
In detail, the title of this document is included in the corresponding metadata of each file.For example, the corresponding unit of file
Data can be including the title of file, time, authority, extended attribute, storage location etc..
For example, it is assumed that for a certain metadata, the cryptographic Hash for calculating is 17, above-mentioned 20 packets can be fallen into due to 17
In second packet:Be grouped 2 corresponding mapping ranges (10,20], illustrate the metadata and mapping range (10,20] phase
Match somebody with somebody, therefore stored in the metadata being mapped into packet 2.
Step 303:When the memory capacity for monitoring catalogue is not located in the range of first threshold set in advance, step is performed
Rapid 304.
In detail, memory capacity can be calculated and obtained by above-mentioned formula (1).
With being constantly stored in for metadata, it is assumed that be computed, the memory capacity of catalogue is 110,000, not positioned at first threshold model
Enclose:In≤10 ten thousand.
Step 304:The virtual directory of the first quantity is set up, and according to predetermined packet migration rules, for current
Whole packets of presence, carry out corresponding migration process.
In detail, the first quantity can be by migration process, so that the storage of any virtual directory that there is currently is held
Amount is respectively positioned on the minimum number in the range of the first threshold.
Based on the above, it can be seen that, the minimum number of first quantity is 2.Therefore, it can set up 2 virtually
Catalogue:Virtual directory 1 and virtual directory 2.The order of virtual directory 2 is arranged in after virtual directory 1.Therefore virtual directory 1 is first place
Virtual directory, virtual directory 2 is second virtual directory.If a newly-built virtual directory, the newly-built virtual directory again
Order is used as the 3rd virtual directory.So circulation.
In detail, after completing to set up each virtual directory, the corresponding metadata of the virtual directory can be generated.Its
In, the corresponding metadata of each virtual directory of generation can be stored into catalogue, be managed collectively with by catalogue.
In detail, the packet migration rules can be:Each order of packets arrangement in the whole packets that there is currently, and work as
When each virtual directory order in whole virtual directories of preceding presence is arranged, each virtual mesh is determined according to above-mentioned formula (2)
Record the number of corresponding packet;
When the number of the whole virtual directories that there is currently is first quantity, according to each the virtual mesh determined
The number of corresponding packet is recorded, described at least two packets are migrated into each virtual directory that there is currently respectively, its
In, for any two virtual directory, when second virtual directory that is located therein of putting in order of the first virtual directory therein
Put in order after when, the putting in order of each packet in first virtual directory is respectively positioned on each in second virtual directory
Packet put in order after;
When the number of the whole virtual directories that there is currently not is first quantity, the arrangement based on each virtual directory is suitable
Sequence, is performed both by for each virtual directory successively:The number for calculating the current corresponding packet of the virtual directory is subtracted and determined
The corresponding packet of the virtual directory number difference;When it is positive number to judge the difference, the virtual directory pair is determined
The packet to be migrated answered, wherein, in the current corresponding all packets of the virtual directory, the position that puts in order of the packet to be migrated
After other packets, and the number of the packet to be migrated is equal to the difference;The packet to be migrated is migrated to order
It is arranged in the next virtual directory after the virtual directory.
In embodiments of the present invention, two virtual directories be presently, there are:Virtual directory 1 and virtual directory 2.There is currently
The number of whole virtual directories is 2, equal to the first quantity:2, that is, illustrate that the whole virtual directories that there is currently are virtual mesh
The foundation first of record, therefore according to above-mentioned packet migration rules, packet 1~packet 10 can be migrated to virtual directory from catalogue
In 1, packet 11~packet 20 is migrated into virtual directory 2 from catalogue.
Step 305:When monitoring that the memory capacity that there is any virtual directory is not located in the range of first threshold, perform
Step 304, and perform step 306.
Accordingly, based on same realization principle, same to above-mentioned formula (1), in an embodiment of the invention, for any
Virtual directory, the memory capacity of the virtual directory can be, the metadata stored in corresponding each packet of the virtual directory
Number plus and.
For the whole virtual directories that be there is currently in file system, if the memory capacity that there is any virtual directory surpasses
Limit, can be held by performing step 304, newly-built virtual directory and to carry out respective packets migration with the storage for solving to occur
Amount overrun issues.
If for example, detect virtual directory 1 transfinite, therefore can be with a newly-built virtual directory:Virtual directory 3.Now, currently
The number of whole virtual directories of presence is 3, is not equal to the first quantity:1, that is, illustrate the whole virtual directories that there is currently
It is not the foundation first of virtual directory.
In this way, according to above-mentioned packet migration rules, packet 8~packet 10 can be migrated to virtual mesh from virtual directory 1
In record 2, packet 15~packet 20 is migrated into virtual directory 3 from virtual directory 2.Now, these three virtual directories are corresponding
Packet number is respectively 7,7 and 6.
Similarly, how many virtual directories no matter are there is currently, as long as monitoring there is the memory capacity of any virtual directory
Transfinite, can be by repeating step 304, newly-built virtual directory and to carry out respective packets migration, to solve what is occurred
Memory capacity overrun issues.
Step 306:It is not located in the unbalanced degree of packet for monitoring last position virtual directory and virtual directory adjacent thereto
When in the range of Second Threshold set in advance, at least one packet for corresponding to last position virtual directory is set up, so that packet is uneven
Weighing apparatus degree is located in the range of Second Threshold.
In detail, being grouped unbalanced degree can be calculated and be obtained by above-mentioned formula (3).
In detail, the number of at least one packet of foundation can be to meet so that being grouped unbalanced degree is located at the second threshold
Minimum number in the range of value.
Assuming that three virtual directories are there is currently, and the corresponding packet number of these three virtual directories is respectively 7,7
With 6.It is computed, the unbalanced degree of packet of virtual directory 2 and virtual directory 3 is 14.2%, not positioned at Second Threshold scope:≤
In 10%, therefore can be with a newly-built packet corresponding to virtual directory 3:Packet 21.
Step 307:For the whole packet that there is currently, redefine each and be grouped corresponding mapping model
Enclose.
After a newly-increased packet, 21 packets are there is currently, therefore this always maps model can to redistribute [0,200]
Enclose.For example, this 21 corresponding mapping ranges of packet can be followed successively by [0,10], (10,20] ..., (100,110], (110,
119], (119,128] ..., (191,200].
Step 308:It is performed both by for the corresponding metadata of each file:Judge packet that the metadata is currently located and
Whether packet corresponding with the mapping range that it matches is identical, if so, terminate current process, otherwise, by the metadata migration
Into packet corresponding with the mapping range that it matches.
To be grouped as a example by 13, it is assumed that for a certain metadata, it is 129 that can calculate cryptographic Hash, illustrates that it there is currently
In packet 13.But, when packet 13 mapping range be changed into (119,128] after, because the cryptographic Hash for calculating is constant, still
Be 129, thus can be seen that 129 can fall into corresponding to packet 14 mapping range (128,137] in.In this way, can be by this yuan
Data are migrated into packet 14 from packet 13.
Based on same realization principle, corresponding migration operation can be carried out to each metadata to be migrated.
Certainly, for each new metadata, can be according to the current mapping range of each packet, by each new metadata
Map to during correspondence is grouped and stored.
As shown in figure 4, one embodiment of the invention provides a kind of device for managing metadata, including:
Determining unit 401, for for corresponding at least two packet of catalogue, determining that each described packet corresponding is reflected
Penetrate scope;
Map unit 402, for being performed both by for the corresponding metadata of each file:By the metadata store to its
In the corresponding packet of mapping range for matching, and trigger the first monitoring unit 403;
First monitoring unit 403, for monitoring that the memory capacity of the catalogue is not located at set in advance the
When in one threshold range, processing unit 404 is triggered;
The processing unit 404, the virtual directory for setting up the first quantity, and according to predetermined packet migration rule
Then, for the whole packet that there is currently, corresponding migration process is carried out.
In an embodiment of the invention, the title of this document is included in the corresponding metadata of each file;
The map unit 402, specifically for being performed both by for the corresponding metadata of each file:According to including
File title, calculate the corresponding cryptographic Hash of the title;It is determined that the target mapping range matched with the cryptographic Hash, its
In, the cryptographic Hash is located in the target mapping range;The metadata is stored to corresponding point of the target mapping range
In group.
In an embodiment of the invention, the memory capacity of the catalogue meets above-mentioned formula (1).
In an embodiment of the invention, first quantity is by the migration process, so that what be there is currently appoints
The memory capacity of one virtual directory is respectively positioned on the minimum number in the range of the first threshold.
In an embodiment of the invention, first monitoring unit 403, is additionally operable to monitoring there is any virtual mesh
The memory capacity of record is not located at when in the range of the first threshold, triggers the processing unit 404.
In an embodiment of the invention, the packet migration rules, including:Each point in the whole packets that there is currently
Group order arrange, and in the whole virtual directories that there is currently each virtual directory order arrange when, according to above-mentioned formula (2) really
The number of the corresponding packet of fixed each virtual directory;
When the number of the whole virtual directories that there is currently is first quantity, according to each the virtual mesh determined
The number of corresponding packet is recorded, described at least two packets are migrated into each virtual directory that there is currently respectively, its
In, for any two virtual directory, when second virtual directory that is located therein of putting in order of the first virtual directory therein
Put in order after when, the putting in order of each packet in first virtual directory is respectively positioned on each in second virtual directory
Packet put in order after;
When the number of the whole virtual directories that there is currently not is first quantity, the arrangement based on each virtual directory is suitable
Sequence, is performed both by for each virtual directory successively:The number for calculating the current corresponding packet of the virtual directory is subtracted and determined
The corresponding packet of the virtual directory number difference;When it is positive number to judge the difference, the virtual directory pair is determined
The packet to be migrated answered, wherein, in the current corresponding all packets of the virtual directory, the position that puts in order of the packet to be migrated
After other packets, and the number of the packet to be migrated is equal to the difference;The packet to be migrated is migrated to order
It is arranged in the next virtual directory after the virtual directory.
In an embodiment of the invention, Fig. 5 is refer to, the device of the management metadata can also include:Second monitoring
Unit 502 is set up in unit 501, packet;
Second monitoring unit 501, for monitoring the last position virtual directory and virtual directory adjacent thereto
Packet it is unbalanced degree be located at Second Threshold set in advance in the range of when, trigger it is described packet set up unit 502;
Unit 502 is set up in the packet, for setting up at least one packet corresponding to the last position virtual directory, so that
The unbalanced degree of packet is located in the range of the Second Threshold, and triggers the determining unit 401;
The determining unit 401, be additionally operable to receive it is described packet set up the trigger signal that unit 502 is sent when, pin
Whole packet to there is currently, redefines each and is grouped corresponding mapping range, and it is single to trigger the mapping
Unit 402;
The map unit 402, is additionally operable to when the trigger signal that the determining unit 401 is sent is received, for every
The corresponding metadata of one file is performed both by:Judge the packet that the metadata is currently located and the mapping range pair matched with it
Whether the packet answered is identical, if it is not, by the metadata migration to packet corresponding with the mapping range that it matches.
In an embodiment of the invention, the number that at least one packet of the foundation of unit 502 is set up in the packet is to accord with
Close described so that minimum number of the unbalanced degree of packet in the range of the Second Threshold.
In an embodiment of the invention, the unbalanced degree of the packet meets above-mentioned formula (3).
The contents such as the information exchange between each unit, implementation procedure in said apparatus, due to implementing with the inventive method
Example is based on same design, and particular content can be found in the narration in the inventive method embodiment, and here is omitted.
In sum, each embodiment of the invention at least has the advantages that:
1st, in the embodiment of the present invention, for corresponding at least two packet of catalogue, the corresponding mapping range of each packet is determined;
The corresponding metadata of each file is stored into packet corresponding with the mapping range that it matches respectively;Monitoring mesh
The memory capacity of record is not located at when in preset threshold range, sets up the virtual directory of the first quantity, and according to packet migration rules,
Corresponding migration process is carried out for the whole packets that there is currently.Based on packet to store magnanimity metadata respectively, therefore in catalogue
Memory capacity when transfiniting, can be deposited with by the way that virtual directory is grouped by way of increasing virtual directory and migrating packet
Storage metadata, such that it is able to solve the problems, such as that catalogue memory capacity transfinites.Therefore, the embodiment of the present invention can improve metadata
Recall precision.
2nd, in the embodiment of the present invention, it is proposed that a kind of method for managing metadata, by real-time dynamic newly-built virtual mesh
Record, by the corresponding different grouping of each virtual directory, to be managed to magnanimity metadata, it is to avoid by all metadata all the time
It is stored under catalogue, such that it is able to improve the recall precision to magnanimity metadata.When the memory capacity of each virtual directory transfinites, can
When being solved with the implementation by newly-built virtual directory and migration packet, and being grouped unbalanced between different virtual directories,
Can be solved by the implementation of newly-built packet and migration metadata, so as to realize that whole file system is answered according to actual
With carrying out dynamic expansion.
It should be noted that herein, such as first and second etc relational terms are used merely to an entity
Or operation makes a distinction with another entity or operation, and not necessarily require or imply these entities or exist between operating
Any this actual relation or order.And, term " including ", "comprising" or its any other variant be intended to it is non-
It is exclusive to include, so that process, method, article or equipment including a series of key elements not only include those key elements,
But also other key elements including being not expressly set out, or also include by this process, method, article or equipment are solid
Some key elements.In the absence of more restrictions, the key element limited by sentence " including ", does not arrange
Except also there is other identical factor in the process including the key element, method, article or equipment.
One of ordinary skill in the art will appreciate that:Realizing all or part of step of above method embodiment can pass through
Programmed instruction related hardware is completed, and foregoing program can be stored in the storage medium of embodied on computer readable, the program
Upon execution, the step of including above method embodiment is performed;And foregoing storage medium includes:ROM, RAM, magnetic disc or light
Disk etc. is various can be with the medium of store program codes.
It is last it should be noted that:Presently preferred embodiments of the present invention is the foregoing is only, skill of the invention is merely to illustrate
Art scheme, is not intended to limit the scope of the present invention.All any modifications made within the spirit and principles in the present invention,
Equivalent, improvement etc., are all contained in protection scope of the present invention.
Claims (10)
1. it is a kind of manage metadata method, it is characterised in that including:
S1:For corresponding at least two packet of catalogue, each corresponding mapping range of the packet is determined;
S2:It is performed both by for the corresponding metadata of each file:The metadata is stored to the mapping range matched with it
In corresponding packet;
S3:When the memory capacity for monitoring the catalogue is not located in the range of first threshold set in advance, S4 is performed;
S4:The virtual directory of the first quantity is set up, and according to predetermined packet migration rules, for the whole that there is currently
Packet, carries out corresponding migration process.
2. method according to claim 1, it is characterised in that
The title of this document is included in the corresponding metadata of each file;
The S2, including:It is performed both by for the corresponding metadata of each file:According to including file title, meter
Calculate the corresponding cryptographic Hash of the title;It is determined that the target mapping range matched with the cryptographic Hash, wherein, the cryptographic Hash position
In in the target mapping range;The metadata is stored into the corresponding packet of the target mapping range;
And/or,
The memory capacity coincidence formula one of the catalogue, wherein,
The formula one includes:
Wherein, X is the memory capacity of the catalogue, and n is the number of at least two packet, NiIt is described at least two packets
In i-th packet in store metadata number;
And/or,
First quantity is by the migration process, so that the memory capacity of any virtual directory that there is currently is respectively positioned on
Minimum number in the range of the first threshold;
And/or,
Further included after S4:In the memory capacity for monitoring to have any virtual directory not positioned at the first threshold model
When enclosing interior, S4 is performed.
3. method according to claim 1 and 2, it is characterised in that
The packet migration rules, including:Each order of packets arrangement in the whole packets that there is currently, and there is currently it is complete
When each virtual directory order in portion's virtual directory is arranged, the individual of the corresponding packet of each virtual directory is determined according to formula two
Number;
When the number of the whole virtual directories that there is currently is first quantity, according to each virtual directory pair determined
The number of the packet answered, described at least two packets are migrated into each virtual directory that there is currently respectively, wherein, it is right
In any two virtual directory, when the arrangement of second virtual directory being located therein that puts in order of the first virtual directory therein
When after order, each packet that the putting in order of each packet in first virtual directory is respectively positioned in second virtual directory
After putting in order;
When the number of the whole virtual directories that there is currently not is first quantity, based on putting in order for each virtual directory,
It is performed both by for each virtual directory successively:Calculate the current corresponding packet of the virtual directory number subtract determine should
The difference of the number of the corresponding packet of virtual directory;When it is positive number to judge the difference, determine that the virtual directory is corresponding
Packet to be migrated, wherein, in the current corresponding all packets of the virtual directory, the packet to be migrated puts in order positioned at it
After he is grouped, and the number of the packet to be migrated is equal to the difference;The packet to be migrated is migrated to order and is arranged
In next virtual directory after the virtual directory;
The formula two includes:
Wherein, X is the number of the whole packets that there is currently;N is the number of the whole virtual directories that there is currently;x1For current
In whole virtual directories of presence, the number of the corresponding packet of any virtual directory in addition to last position virtual directory, x2For described
The number of the corresponding packet of last position virtual directory.
4. method according to claim 3, it is characterised in that
Further included after S4:It is uneven in the packet for monitoring the last position virtual directory and virtual directory adjacent thereto
Weighing apparatus degree is not located at when in the range of Second Threshold set in advance, sets up at least one point corresponding to the last position virtual directory
Group, so that the unbalanced degree of the packet is located in the range of the Second Threshold;For the whole packet that there is currently, redefine
Each is grouped corresponding mapping range;It is performed both by for the corresponding metadata of each file:Judge the metadata
Whether the packet and packet corresponding with the mapping range that it matches being currently located are identical, if it is not, by the metadata migration extremely
In packet corresponding with the mapping range that it matches.
5. method according to claim 4, it is characterised in that
The number of at least one packet of the foundation is to meet described so that the unbalanced degree of packet is located at second threshold
Minimum number in the range of value;
And/or,
The unbalanced degree coincidence formula three of packet, wherein,
The formula three includes:
Wherein, Y is the unbalanced degree of packet, XnIt is the number of the corresponding packet of the last position virtual directory, XmIt is and the end
The number of the adjacent corresponding packet of virtual directory of position virtual directory.
6. it is a kind of manage metadata device, it is characterised in that including:
Determining unit, for for corresponding at least two packet of catalogue, determining each corresponding mapping range of the packet;
Map unit, for being performed both by for the corresponding metadata of each file:The metadata is stored to it and is matched
The corresponding packet of mapping range in, and trigger the first monitoring unit;
First monitoring unit, for not being located at first threshold model set in advance in the memory capacity for monitoring the catalogue
When enclosing interior, processing unit is triggered;
The processing unit, the virtual directory for setting up the first quantity, and according to predetermined packet migration rules, for
The whole packets that there is currently, carry out corresponding migration process.
7. it is according to claim 6 management metadata device, it is characterised in that
The title of this document is included in the corresponding metadata of each file;
The map unit, specifically for being performed both by for the corresponding metadata of each file:According to including file
Title, calculate the corresponding cryptographic Hash of the title;It is determined that the target mapping range matched with the cryptographic Hash, wherein, institute
Cryptographic Hash is stated to be located in the target mapping range;The metadata is stored into the corresponding packet of the target mapping range;
And/or,
The memory capacity coincidence formula one of the catalogue, wherein,
The formula one includes:
Wherein, X is the memory capacity of the catalogue, and n is the number of at least two packet, NiIt is described at least two packets
In i-th packet in store metadata number;
And/or,
First quantity is by the migration process, so that the memory capacity of any virtual directory that there is currently is respectively positioned on
Minimum number in the range of the first threshold;
And/or,
First monitoring unit, is additionally operable in the memory capacity for monitoring to have any virtual directory not positioned at first threshold
When in the range of value, the processing unit is triggered.
8. the device of the management metadata according to claim 6 or 7, it is characterised in that
The packet migration rules, including:Each order of packets arrangement in the whole packets that there is currently, and there is currently it is complete
When each virtual directory order in portion's virtual directory is arranged, the individual of the corresponding packet of each virtual directory is determined according to formula two
Number;
When the number of the whole virtual directories that there is currently is first quantity, according to each virtual directory pair determined
The number of the packet answered, described at least two packets are migrated into each virtual directory that there is currently respectively, wherein, it is right
In any two virtual directory, when the arrangement of second virtual directory being located therein that puts in order of the first virtual directory therein
When after order, each packet that the putting in order of each packet in first virtual directory is respectively positioned in second virtual directory
After putting in order;
When the number of the whole virtual directories that there is currently not is first quantity, based on putting in order for each virtual directory,
It is performed both by for each virtual directory successively:Calculate the current corresponding packet of the virtual directory number subtract determine should
The difference of the number of the corresponding packet of virtual directory;When it is positive number to judge the difference, determine that the virtual directory is corresponding
Packet to be migrated, wherein, in the current corresponding all packets of the virtual directory, the packet to be migrated puts in order positioned at it
After he is grouped, and the number of the packet to be migrated is equal to the difference;The packet to be migrated is migrated to order and is arranged
In next virtual directory after the virtual directory;
The formula two includes:
Wherein, X is the number of the whole packets that there is currently;N is the number of the whole virtual directories that there is currently;x1For current
In whole virtual directories of presence, the number of the corresponding packet of any virtual directory in addition to last position virtual directory, x2For described
The number of the corresponding packet of last position virtual directory.
9. it is according to claim 8 management metadata device, it is characterised in that
Also include:Unit is set up in second monitoring unit, packet;
Second monitoring unit, in the packet for monitoring the last position virtual directory and virtual directory adjacent thereto not
Equilibrium degree is not located at when in the range of Second Threshold set in advance, is triggered the packet and is set up unit;
Unit is set up in the packet, for setting up at least one packet corresponding to the last position virtual directory, so that described point
The unbalanced degree of group is located in the range of the Second Threshold, and triggers the determining unit;
The determining unit, is additionally operable to when receiving the packet and setting up the trigger signal that unit is sent, for there is currently
Whole packets, redefine each and be grouped corresponding mapping range, and trigger the map unit;
The map unit, is additionally operable to when the trigger signal that the determining unit is sent is received, for each file pair
The metadata answered is performed both by:Judging packet that the metadata is currently located and packet corresponding with the mapping range that it matches is
It is no identical, if it is not, by the metadata migration to packet corresponding with the mapping range that it matches.
10. it is according to claim 9 management metadata device, it is characterised in that
The number that at least one packet of unit foundation is set up in the packet is to meet described so that the unbalanced degree position of packet
Minimum number in the range of the Second Threshold;
And/or,
The unbalanced degree coincidence formula three of packet, wherein,
The formula three includes:
Wherein, Y is the unbalanced degree of packet, XnIt is the number of the corresponding packet of the last position virtual directory, XmIt is and the end
The number of the adjacent corresponding packet of virtual directory of position virtual directory.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201611139129.6A CN106777062B (en) | 2016-12-12 | 2016-12-12 | Method and device for managing metadata |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201611139129.6A CN106777062B (en) | 2016-12-12 | 2016-12-12 | Method and device for managing metadata |
Publications (2)
Publication Number | Publication Date |
---|---|
CN106777062A true CN106777062A (en) | 2017-05-31 |
CN106777062B CN106777062B (en) | 2020-03-10 |
Family
ID=58879884
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201611139129.6A Active CN106777062B (en) | 2016-12-12 | 2016-12-12 | Method and device for managing metadata |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN106777062B (en) |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102497428A (en) * | 2011-12-13 | 2012-06-13 | 方正国际软件有限公司 | Remote storage system and method for remote storage thereof |
CN103688257A (en) * | 2012-11-27 | 2014-03-26 | 华为技术有限公司 | Method and device for managing metadata |
CN103916459A (en) * | 2014-03-04 | 2014-07-09 | 南京邮电大学 | Big data filing and storing system |
CN103916467A (en) * | 2014-03-25 | 2014-07-09 | 中国科学院计算技术研究所 | Load transfer method and system in metadata cluster |
CN106021462A (en) * | 2016-05-17 | 2016-10-12 | 深圳市中博科创信息技术有限公司 | File storage method of cluster file system and cluster file system |
-
2016
- 2016-12-12 CN CN201611139129.6A patent/CN106777062B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102497428A (en) * | 2011-12-13 | 2012-06-13 | 方正国际软件有限公司 | Remote storage system and method for remote storage thereof |
CN103688257A (en) * | 2012-11-27 | 2014-03-26 | 华为技术有限公司 | Method and device for managing metadata |
CN103916459A (en) * | 2014-03-04 | 2014-07-09 | 南京邮电大学 | Big data filing and storing system |
CN103916467A (en) * | 2014-03-25 | 2014-07-09 | 中国科学院计算技术研究所 | Load transfer method and system in metadata cluster |
CN106021462A (en) * | 2016-05-17 | 2016-10-12 | 深圳市中博科创信息技术有限公司 | File storage method of cluster file system and cluster file system |
Also Published As
Publication number | Publication date |
---|---|
CN106777062B (en) | 2020-03-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10176113B2 (en) | Scalable indexing | |
CA2131968C (en) | Computer method and system for b-tree optimization | |
CN107423422B (en) | Spatial data distributed storage and search method and system based on grid | |
US9336152B1 (en) | Method and system for determining FIFO cache size | |
CN103581331B (en) | The online moving method of virtual machine and system | |
CN103995855B (en) | The method and apparatus of data storage | |
CN106612619B (en) | Hierarchical stor, using the computer and correction of hierachical storing device to the computer of the access count of file | |
CN103140840B (en) | The method and device of data management | |
CN106911743B (en) | Small documents write polymerization, read polymerization and system and client | |
CN101419616A (en) | Data synchronization method and apparatus | |
CN107273042A (en) | Deduplication DRAM system algorithm framework | |
CN103701469B (en) | A kind of compression and storage method of large-scale graph data | |
CN107315814A (en) | Data consistency checking method and system after a kind of KDB database data migrations | |
CN107958079A (en) | Aggregate file delet method, system, device and readable storage medium storing program for executing | |
CN107870970A (en) | A kind of data store query method and system | |
CN104346384A (en) | Method and device for processing small files | |
CN110597835B (en) | Transaction data deleting method and device based on blockchain | |
CN104021223B (en) | The access method and device of a kind of Cluster Database measuring point | |
CN107391045A (en) | A kind of directory space quota method and system | |
CN102722450B (en) | Storage method for redundancy deletion block device based on location-sensitive hash | |
CN113867627B (en) | Storage system performance optimization method and system | |
US9275091B2 (en) | Database management device and database management method | |
CN106777062A (en) | A kind of method and device for managing metadata | |
CN106933974A (en) | The structure and querying method and device of a kind of video file index information | |
CN107273443A (en) | A kind of hybrid index method based on big data model metadata |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |