CN106021466A - Data storage method and apparatus - Google Patents
Data storage method and apparatus Download PDFInfo
- Publication number
- CN106021466A CN106021466A CN201610327515.1A CN201610327515A CN106021466A CN 106021466 A CN106021466 A CN 106021466A CN 201610327515 A CN201610327515 A CN 201610327515A CN 106021466 A CN106021466 A CN 106021466A
- Authority
- CN
- China
- Prior art keywords
- catalogue
- idle
- nodes
- locations
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
Abstract
Embodiments of the invention relate to a data storage method and apparatus, which is used for solving the problem of low efficiency due to the fact that an available directory can be found only through multi-time recursive search in a mode of managing the directory in a directory tree form during data storage in an existing distributed file system. The method comprises the steps of determining at least one idle directory in all directories corresponding to a bitmap according to attribute information of a directory stored in each position node of the bitmap and corresponding to the position node, wherein each directory corresponds to one position node in the bitmap; and storing data required to be stored currently in a storage position corresponding to the determined idle directory. According to the data storage method and apparatus, the idle directory capable of storing the data can be searched for according to the attribute information of the directory stored in each position node of the bitmap and corresponding to the position node, and the available directory can be found without the multi-time recursive search, so that compared with the directory tree management mode, the query efficiency is improved and the data storage speed is increased.
Description
Technical field
The present invention relates to data processing field, particularly to a kind of method and device storing data.
Background technology
Along with the development of computer technology, in distributed file system, the catalogue quantity of storage generally increases comparatively fast.
When certain distributed file system exists substantial amounts of catalogue, how to store and manage these directory informations and become
It it is a big problem.The file of User space is split by prior art and is stored in different data for block one by one
In node, these blocks are exactly a file for back end.Back end management for convenience this
A little files need to deposit in file still in idle catalogue, it is to avoid a catalogue is excessive, causes at ff,
Opening procedure is the most time-consuming.Meanwhile, need regular to deposit in by layer in order to avoid depositing catalogue
In various level catalogue, if data is the data directory for specifying, then data catalogue can be deposited
The file of certain quantity and subdirectory, can deposit again in these subdirectories certain data volume file and
Subdirectory, the like.
For realizing above-mentioned target, common mode be use directory tree form in internal memory in each catalogue
Number of files, the relative path etc. of catalogue that can deposit are managed, and the problem that this way to manage is brought is meeting
Cause a subtree degree of depth the deepest, and the process searching available directories needs just may be used by repeatedly recursive lookup
Realize, inefficient, affect the process of normal data storage and query.
In sum, when current distributed file system carries out data storage, the form pipe of directory tree is used
The mode of reason catalogue, needs just to find available directories by repeatedly recursive lookup, and inefficient, impact is just
The process of normal data storage and query.
Summary of the invention
The present invention provides a kind of method and device storing data, in order to solve in existing distributed file system
When carrying out data storage, use the mode of the format management catalogue of directory tree, need by repeatedly recursive lookup
Just can find available directories, inefficient, the problem that affects the process of normal data storage and query.
Based on the problems referred to above, a kind of method storing data that the embodiment of the present invention provides, the method includes:
According to the attribute information of the catalogue corresponding with this nodes of locations of storage in each nodes of locations of bitmap,
Determine the idle catalogue of at least one in all catalogues corresponding with described bitmap, the most each catalogue correspondence position
A nodes of locations in figure, described idle catalogue is data-storable catalogue on corresponding storage position；
The data being currently needed for storage are stored the storage position that the idle catalogue determined is corresponding.
The method of the storage data that the embodiment of the present invention provides, is to store in units of file, with catalogue
The distributed memory system being managed for unit, when storing data, can be according to each position of bitmap
In node, the attribute information of the catalogue corresponding with this nodes of locations of storage, searches the data-storable free time
Catalogue, just can find available directories, compared to the manager of directory tree without by repeatedly recursive lookup
Formula, improves search efficiency and the speed of data storage.
It is also preferred that the left the idle catalogue of at least one determined in all catalogues corresponding with described bitmap described, bag
Include:
Determining the nodes of locations that in bitmap, call number is pointed to, the nodes of locations that wherein said call number is pointed to is upper
The nodes of locations corresponding to catalogue that once the storage position of storage data is corresponding；
If data can be stored on the storage position that the catalogue of the nodes of locations that described call number is pointed to is corresponding, will
This catalogue is as idle catalogue.
It is also preferred that the left the idle catalogue of at least one determined in all catalogues corresponding with described bitmap described, bag
Include:
If data can not be stored on the storage position that the catalogue of the nodes of locations that described call number is pointed to is corresponding,
All catalogues the level at the catalogue place of the nodes of locations pointed to from described call number determine at least one
Individual idle catalogue.
It is also preferred that the left the idle catalogue of at least one determined in all catalogues corresponding with described bitmap described, bag
Include:
If can not in all catalogues in the level at the catalogue place of the nodes of locations that described call number is pointed to
Storage data, start catalogue level one by one from root and carry out traversal and search, determine in all catalogues at least
One idle catalogue.
It is also preferred that the left judge whether catalogue is idle catalogue according to following manner:
For a catalogue, according to the file in the attribute information of storage in the nodes of locations that this catalogue is corresponding
Number information, determines in the storage position that this catalogue is corresponding, whether the file number of storage reaches predetermined threshold value；
If described file number is not reaching to predetermined threshold value, using this catalogue as idle catalogue.
It is also preferred that the left determine, according to following manner, the storage position that idle catalogue is corresponding:
For an idle catalogue determined, according to the simplification routing information that this free time catalogue is corresponding, Yi Jijian
Change the corresponding relation between routing information and complete path information, determine the fullpath that this free time catalogue is corresponding
Information；
According to the complete path information determined, determine the storage position that this free time catalogue is corresponding.
It is also preferred that the left before the idle catalogue determined in the catalogue corresponding with described bitmap, also include:
Content according to default directory attribute information and the storage state of current directory, determine catalogue
The attribute information that each catalogue in each level is corresponding；
According to continuously arranged form, respectively by attribute information corresponding for each catalogue in described each level
Store nodes of locations corresponding in bitmap one by one.
It is also preferred that the left described, the data being currently needed for storage are stored the storage position that the idle catalogue determined is corresponding
Before putting, also include:
If the catalogue corresponding with described bitmap does not has idle catalogue, create more under the subdirectory of lowest hierarchical level
The catalogue of low-level, and using the catalogue of the lower level level of establishment as idle catalogue.
A kind of device storing data that the embodiment of the present invention provides, this device includes:
Determine module, for according to the mesh corresponding with this nodes of locations of storage in each nodes of locations of bitmap
The attribute information of record, determines the idle catalogue of at least one in all catalogues corresponding with described bitmap, wherein
A nodes of locations in each catalogue correspondence bitmap, described idle catalogue is can on corresponding storage position
The catalogue of storage data；
Memory module, for storing, by the data being currently needed for storage, the storage that the idle catalogue determined is corresponding
Position.
It is also preferred that the left described determine module specifically for:
Determining the nodes of locations that in bitmap, call number is pointed to, the nodes of locations that wherein said call number is pointed to is upper
The nodes of locations corresponding to catalogue that once the storage position of storage data is corresponding；If described call number is pointed to
Data can be stored, using this catalogue as idle catalogue on the storage position that the catalogue of nodes of locations is corresponding.
It is also preferred that the left described determine module specifically for:
If data can not be stored on the storage position that the catalogue of the nodes of locations that described call number is pointed to is corresponding,
All catalogues the level at the catalogue place of the nodes of locations pointed to from described call number determine at least one
Individual idle catalogue.
It is also preferred that the left described determine module specifically for:
If can not in all catalogues in the level at the catalogue place of the nodes of locations that described call number is pointed to
Storage data, start catalogue level one by one from root and carry out traversal and search, determine in all catalogues at least
One idle catalogue.
Determine according to following manner, module judges whether catalogue is idle catalogue it is also preferred that the left described:
For a catalogue, according to the file in the attribute information of storage in the nodes of locations that this catalogue is corresponding
Number information, determines in the storage position that this catalogue is corresponding, whether the file number of storage reaches predetermined threshold value；If
Described file number is not reaching to predetermined threshold value, using this catalogue as idle catalogue.
It is also preferred that the left described memory module determines, according to following manner, the storage position that idle catalogue is corresponding:
For an idle catalogue determined, according to the simplification routing information that this free time catalogue is corresponding, Yi Jijian
Change the corresponding relation between routing information and complete path information, determine the fullpath that this free time catalogue is corresponding
Information；According to the complete path information determined, determine the storage position that this free time catalogue is corresponding.
It is also preferred that the left this device also includes:
Set up bitmap module, before the idle catalogue in determining the catalogue corresponding with described bitmap, root
According to content and the storage state of current directory of default directory attribute information, determine each layer of catalogue
The attribute information that each catalogue in Ji is corresponding；According to continuously arranged form, respectively by described each level
In attribute information corresponding to each catalogue store nodes of locations corresponding in bitmap one by one.
It is also preferred that the left this device also includes:
Create directory module, for the data being currently needed for storage being stored the idle catalogue determined described
Before corresponding storage position, if the catalogue corresponding with described bitmap there is no idle catalogue, in lowest hierarchical level
Subdirectory under create the catalogue of lower level level, and using the catalogue of lower level level that creates as idle catalogue.
Accompanying drawing explanation
Fig. 1 is a kind of method flow schematic diagram storing data of the embodiment of the present invention；
The structural representation of a kind of bitmap that Fig. 2 provides for the embodiment of the present invention；
The overall flow schematic diagram of the data storage that Fig. 3 provides for the embodiment of the present invention；
The structural representation of a kind of device storing data that Fig. 4 provides for the embodiment of the present invention.
Detailed description of the invention
The embodiment of the present invention is according to the mesh corresponding with this nodes of locations of storage in each nodes of locations of bitmap
The attribute information of record, determines the idle catalogue of at least one in all catalogues corresponding with bitmap, the most each
A nodes of locations in catalogue correspondence bitmap, idle catalogue is can to store data on corresponding storage position
Catalogue；The data being currently needed for storage are stored the storage position that the idle catalogue determined is corresponding.Due to
Can according to the attribute information of the catalogue corresponding with this nodes of locations of storage in each nodes of locations of bitmap,
Search data-storable idle catalogue, just can find available directories without by repeatedly recursive lookup,
Compared to the way to manage of directory tree, improve search efficiency and the speed of data storage.
Wherein, the embodiment of the present invention can be applied in distributed file system, it is also possible to being applied to other needs
To be stored in the system of file data by administrative directory, such as fast photographic system etc..
Below in conjunction with Figure of description, the embodiment of the present invention is described in further detail.
As it is shown in figure 1, a kind of method flow schematic diagram storing data of the embodiment of the present invention, the method includes:
Step 101, according to the catalogue corresponding with this nodes of locations of storage in each nodes of locations of bitmap
Attribute information, determines the idle catalogue of at least one in all catalogues corresponding with bitmap, the most each catalogue
A nodes of locations in corresponding bitmap, idle catalogue is data-storable mesh on corresponding storage position
Record；
The data being currently needed for storage are stored the storage position that the idle catalogue determined is corresponding by step 102.
At present, when distributed file system carries out data storage, the format management catalogue of directory tree is used
Mode, needs just to find available directories by repeatedly recursive lookup, inefficient, affects normal data
Storage and the process of inquiry.And the method for the storage data that the embodiment of the present invention provides, it is in units of file
Store, the distributed memory system being managed in units of catalogue, when storing data, Ke Yigen
According to the attribute information of the catalogue corresponding with this nodes of locations of storage in each nodes of locations of bitmap, search energy
Enough store the idle catalogue of data, just can find available directories without by repeatedly recursive lookup, compare
In the way to manage of directory tree, improve search efficiency and the speed of data storage.
In the specific implementation, the embodiment of the present invention passes through bitmap management catalogue, comprises some positions in its Bitmap
Putting a nodes of locations in node, and each catalogue correspondence bitmap, in each nodes of locations, storage has and is somebody's turn to do
The attribute information of the catalogue that nodes of locations is corresponding；Determine need store data time, according to each position of bitmap
Put the attribute information of the catalogue of storage in node, determine at least one in all catalogues corresponding with this bitmap
Idle catalogue, so as to the data being currently needed for storage are stored the storage that the idle catalogue determined is corresponding
On position.
An above-mentioned bitmap catalogue corresponding with each nodes of locations in catalogue one_to_one corresponding, i.e. bitmap, makes
Before the idle catalogue of bitmap inquiry, in addition it is also necessary to build bitmap according to catalogue, it is also preferred that the left before step 101,
Also include: according to content and the storage state of current directory of default directory attribute information, determine mesh
Attribute information corresponding to each catalogue in each level of record；According to continuously arranged form, respectively will be every
Attribute information corresponding to each catalogue in individual level stores nodes of locations corresponding in bitmap one by one.
In the specific implementation, user can the attribute of config directory as required, than catalogic level number,
The file number of the general act number of storage, each subdirectory storage and the path rule letter of each catalogue
Breath etc., in addition it is also necessary to judge that current directory has existed some catalogues used, if had,
The service condition of the catalogue then these used is converted to attribute information, i.e. according to user configured catalogue
Attribute (content of default directory attribute information) and the storage state of current directory, determine catalogue
Each level in attribute information corresponding to each catalogue.
Wherein, the form of the existence of bitmap can be one section of continuous print internal memory in internal memory, by the layer of catalogue
Level deposits the attribute information of the catalogue of correspondence the most in order.I.e. according to continuously arranged form, respectively by each
Attribute information corresponding to each catalogue in level stores nodes of locations corresponding in bitmap one by one.Lower mask
Body introduces the form of bitmap.
As in figure 2 it is shown, the structural representation of a kind of bitmap provided for the embodiment of the present invention, by diagram
Data structure is indicated, the degree of depth be 1 catalogue represent ground floor catalogue Id0 (i.e. root), the degree of depth is
The nodes of locations of 1 only occupies a node in bitmap, and wherein root is the same with the subdirectory character under it,
File can be deposited and have subdirectory.As a example by Fig. 2, each catalogue can have 3 subdirectories, that
The degree of depth be 1 ground floor catalogue can have 3 subdirectories, these three subdirectory (Id1, Id2 and Id3)
Be exactly the degree of depth be 2, then third layer subdirectory just has 9 subdirectories (Id4, Id5, Id6 to Id13),
Represent that the degree of depth is 3.Owing to embodiment of the present invention Bitmap is the mode that a continuous print sorts by level, number
It is believed that being stored in internal memory of breath is continuous print, can check that current directory is deposited only by inquiry bitmap
File number whether reach the upper limit, if the upper limit of being not reaching to, then complete search, using this catalogue as energy
Enough idle catalogues used, and the number of files in accumulative current directory, which enhance the speed of lookup.
In bitmap, the attribute information of the catalogue of storage can configure as required, for example, it is possible to include working as
Whether front catalogue is whether root node isTop, degree of depth deep of current directory, current directory have subdirectory
File number information Num of hasChildren, each subdirectory storage and the path rule of each catalogue
Information key etc., the embodiment of the present invention simply lists the information of several relatively common directory attribute, at this
The not concrete data structure of the attribute information of the catalogue of storage in defined position node, user can be according to reality
Border needs to configure.
In the specific implementation, each different attribute information has different effects, such as file number information
Num can be used to judge the most whether current directory can store data, it is also preferred that the left judge according to following manner
Whether catalogue is idle catalogue: for a catalogue, according to the genus of storage in the nodes of locations that this catalogue is corresponding
Property information in file number information, determine that in the storage position that this catalogue is corresponding, the file number of storage is whether
Reach predetermined threshold value；If file number is not reaching to predetermined threshold value, using this catalogue as idle catalogue.
For any one catalogue in all catalogues, either root or subdirectory, can basis
In corresponding nodes of locations, the file number information in the attribute information of storage, determines the storage that this catalogue is corresponding
In position, the file number of storage has reached the default upper limit (i.e. predetermined threshold value), if being not reaching to
The upper limit, then this catalogue is idle catalogue, if reaching the upper limit, then it represents that this catalogue can not store data, no again
Can be as idle catalogue.It is to say, embodiments of the invention can directly determine each mesh by bitmap
In record, the file number of storage, actual need not go the catalogue checking in file system to house number of files, subtract
Lack the expense that system is called, improve the efficiency of inquiry.
In the specific implementation, in order to save the storage of pathname, the module of the idle catalogue of application in file system
Only need to store an absolute path representing root, the path of follow-up subdirectory is all according to configurable name rule
Then generate.Such as, the root/data of storage data, internal subdirectory naming rule is
Sub0...sub2, equally exists again the subdirectory of sub0...sub2 in each subdirectory./ data is in bitmap
Being expressed as position is 0, and the degree of depth is the root node of 1, this only one of which；/ data/sub0 represents in bitmap
Being 1 for position, the degree of depth is the node of 2；It is 2 that/data/sub1 is then expressed as position in bitmap, and the degree of depth is
The node of 2；It is 4 that/data/sub0/sub0 is then expressed as position in bitmap, and the degree of depth is the node of 3, other
The like.
Further, say, that using the catalogue specified as root, root prefix is fixed, with root
Catalogue starts, and retrains and can deposit the file that subdirectory number can be deposited with each catalogue in each catalogue
Number, subdirectory name prefix fixes, and suffix starts to be incremented to (maximum number-1), also from 0 successively
That is subdirectory name can be such as subdirectory prefix name 0, subdirectory prefix name 1, by that analogy.Again from catalogue
Degree of depth angle is seen, root is first order catalogue, has and only one of which；Second level catalogue is under root
Subdirectory, number is 3 specified；The subdirectory of the second level catalogue under third level catalogue root, number
For 3*3.
Wherein, bitmap size has been fixed when what the level of corresponding catalogue is, such as, catalogue is three grades,
So in bitmap, root is deposited in first position, and the degree of depth is 1, if having subdirectory, oneself is specific item
Record then also has subdirectory numbering, contains how many files；End position is deposited immediately the first order in the second level
Starting storage, depositing subdirectory number is 3, represents subdirectory 0 to the subdirectory 2 under root respectively；
The third level is deposited end position immediately the second level and starts storage, deposits 3*3 subdirectory, the of the third level
One 3 expression are that first subdirectory of the second level houses 3 subdirectories for 0 time, and other is by that analogy.
In order to decrease the expense of internal memory, can be by the way of mapping, by complete path information (as above-mentioned
The naming rule in path) it is mapped as simplifying routing information (such as integer form), and then need not store complete
Pathname, by corresponding relation between the two, i.e. may know that the complete path information of catalogue.It is also preferred that the left
The storage position that idle catalogue is corresponding is determined: for an idle catalogue determined according to following manner, according to
The simplification routing information that this free time catalogue is corresponding, and simplify between routing information and complete path information right
Should be related to, determine the complete path information that this free time catalogue is corresponding；According to the complete path information determined, really
The storage position that this free time catalogue fixed is corresponding.
Such as, the nodes of locations of each bitmap stores two kinds of routing informations, i.e. simplifies routing information keyId
With complete path information layer [n], wherein layer [n] represents the complete routing information that each catalogue is corresponding,
KeyId is the form of shaping, and the effect of keyId is to facilitate record in order to layer [n] is converted to shaping value,
As a example by/data/sub0/sub1, then have recorded in layer 0,0, and these values of 1 ,-1}, wherein-1
Represent end value.
In order to make catalogue use equilibrium, the embodiment of the present invention introduces call number, can realize according to call number
Storage position Coutinuous store with last time storage data.It is also preferred that the left determine all catalogues corresponding with bitmap
In at least one idle catalogue, comprise determining that the nodes of locations that in bitmap, call number is pointed to, wherein index
Number nodes of locations pointed to is the joint of the position corresponding to catalogue that the storage position of last storage data is corresponding
Point；If data can be stored, by this mesh on the storage position that the catalogue of the nodes of locations that call number is pointed to is corresponding
Record is as idle catalogue.
In the specific implementation, when storing data and needing the idle catalogue of application, according to the call number record of record
Which nodes of locations being being currently used in bitmap, it is judged that whether current location node is an idle catalogue,
If it is, the value of file number information num in the attribute information of catalogue is added 1, and return a use
In the simplification routing information keyId of this free time catalogue of expression, by corresponding relation between the two, by this letter
Change routing information keyId and be converted to complete path information layer [n], to the idle catalogue of application in file system
Module uses.
If data can not be stored again, then on the storage position that the catalogue of the nodes of locations that call number is pointed to is corresponding
In order to ensure the equilibrium that catalogue uses, the degree of depth making again whole directory tree will not be too low, it is also preferred that the left determine
The idle catalogue of at least one in all catalogues corresponding with bitmap, including: if the position joint that call number is pointed to
Data can not be stored, from the catalogue of the nodes of locations that call number is pointed on the storage position corresponding to catalogue of point
All catalogues in the level at place determine at least one idle catalogue.
In the specific implementation, if the catalogue of the nodes of locations of call number sensing is not an idle catalogue, then
Judge the most available free catalogue in this level that nodes of locations that current cable quotation marks point to is corresponding, it is ensured that first will
All catalogues in current level make to be finished, and just can use the catalogue of next level, if had, and will be currently
The attribute information of the catalogue found in level is updated, and returns the simplification path letter of this free time catalogue
Breath keyId.
If all catalogues in the level at the catalogue place of the nodes of locations that call number is pointed to all can not be deposited again
Storage data, then in order to ensure the equilibrium that catalogue uses, the degree of depth making again whole directory tree will not be too low, relatively
Good, determine the idle catalogue of at least one in all catalogues corresponding with bitmap, including: if call number refers to
To nodes of locations catalogue place level in all catalogues in can not store data, open from root
The catalogue level one by one that begins carries out traversal and searches, and determines the idle catalogue of at least one in all catalogues.
In the specific implementation, if the level at the catalogue place of the nodes of locations of call number sensing does not find
One idle catalogue, then begin look for from root, because before current level, may have file
The level that the degree of depth is shallower is deleted, releases space so that those catalogues become again idle catalogue, if
Have, the attribute information of the catalogue found in current level is updated, and return this free time catalogue
Simplification routing information keyId.
If it addition, all catalogues in the level at the catalogue place of the nodes of locations of call number sensing all can not
Store data again, then in order to ensure the equilibrium that catalogue uses, the degree of depth making again whole directory tree will not be too low,
It is also preferred that the left at least one the idle catalogue determined in all catalogues corresponding with bitmap, including: if call number
All catalogues in the level at the catalogue place of the nodes of locations pointed to can not store data, determine peso
Idle catalogue in the level that the rank of level at the catalogue place of the nodes of locations that quotation marks point to is high.Such as,
Call number points to the 3rd level of catalogue, and whole 3rd level does not all have idle catalogue, then can directly from
Root starts down to search, it is determined whether available free catalogue.
In the specific implementation, if the level at the catalogue place of the nodes of locations of call number sensing does not find
One idle catalogue, then can be high from the rank of the level at the catalogue place of the nodes of locations pointed to than call number
Level in idle catalogue because before current level, may have file at the shallower layer of the degree of depth
Delete in Ji, release space so that those catalogues become again idle catalogue, if had, will be at current layer
The attribute information of the catalogue found in Ji is updated, and returns the simplification routing information of this free time catalogue
keyId.Such as, call number points to the 3rd level of catalogue, and whole 3rd level does not all have idle catalogue,
Then can continue to search for the most available free catalogue in the 2nd level higher than the rank of the 3rd level and root.
If said method does not the most find idle catalogue, illustrate in existing root and subdirectory the most
It is filled with, can further create subdirectory, determine it is also preferred that the left the data being currently needed for storage are stored
Storage position corresponding to idle catalogue before, also include: if the catalogue corresponding with bitmap does not has idle mesh
Record, creates the catalogue of lower level level, and the catalogue of the lower level level by establishment under the subdirectory of lowest hierarchical level
As idle catalogue.
In the specific implementation, if not finding idle catalogue in the catalogue corresponding with bitmap, and this bitmap
Use full, then returned and search idle catalogue failure, and under the subdirectory of lowest hierarchical level, create lower level level
Catalogue, as created subdirectory sub0..sub2 under/data/sub0, and update/data/sub0 in
HasChildren is 1, choose simultaneously/data/sub0/sub0 as an available idle catalogue, by this road
The value of file number information num of the nodes of locations that footpath is corresponding in bitmap adds 1, and returns this free time
The simplification routing information keyId of catalogue.
By the way of above-mentioned storage data, because depositing continuously between the level up and down of catalogue, then
Catalogue in use, the most preferentially uses the catalogue level on upper strata so that the subdirectory of the superiors is preferentially made
With.Only when currently pressing level order traversal bitmap, find that all catalogues are used up, then after finding
One subdirectory that can be created.The most both can ensure that the equilibrium that catalogue uses, make again whole catalogue
The degree of depth step-down of tree, decreases the occupancy of internal memory simultaneously.
Be introduced to more clearly embodiment of the present invention offer be stored the method for data, below with
Illustrate as a example by not finding idle catalogue in the catalogue corresponding with bitmap.As it is shown on figure 3, be this
The overall flow schematic diagram of the data storage that bright embodiment provides, this overall flow includes:
Step 301, file system, after write file, needs the idle catalogue of application；
Step 302, it is judged that whether the catalogue that call number is pointed to is idle catalogue, the most then perform step 307,
Otherwise, step 303 is performed；
Step 303, it is judged that whether have in all catalogues in the current level at the catalogue place that call number is pointed to
Idle catalogue, if having, then performs step 307, otherwise, performs step 304；
Step 304, searches no available free catalogue from root, if having, then and execution step 307, otherwise,
Perform step 305；
Step 305, it is judged that whether bitmap with full, the most then perform step 306, otherwise, perform step
307；
Step 306, returns and searches idle catalogue failure, create lower level level under the subdirectory of lowest hierarchical level
Catalogue, and perform step 307；
The attribute information of step 307, more new directory, and return the simplification routing information of this catalogue.
Based on same inventive concept, the embodiment of the present invention additionally provides a kind of device storing data, due to
The method that the device of Fig. 4 is corresponding is a kind of method storing data of the embodiment of the present invention, and therefore the present invention implements
The enforcement of example device may refer to the enforcement of method, repeats no more in place of repetition.
As shown in Figure 4, the structural representation of a kind of device storing data that the embodiment of the present invention provides, should
Device comprises determining that module 401 and memory module 402；
Determine module 401, for corresponding with this nodes of locations according to what each nodes of locations of bitmap stored
The attribute information of catalogue, determine the idle catalogue of at least one in all catalogues corresponding with bitmap, wherein
A nodes of locations in each catalogue correspondence bitmap, idle catalogue is can to store on corresponding storage position
The catalogue of data；
Memory module 402, corresponding for the data being currently needed for storage are stored the idle catalogue determined
Storage position.
It is also preferred that the left determine module 401 specifically for:
Determine that the nodes of locations that in bitmap, call number is pointed to, the nodes of locations that wherein call number is pointed to are last
Store the nodes of locations corresponding to catalogue that the storage position of data is corresponding；If the nodes of locations that call number is pointed to
Storage position corresponding to catalogue on can store data, using this catalogue as idle catalogue.
It is also preferred that the left determine module 401 specifically for:
If data can not be stored, from rope on the storage position that the catalogue of the nodes of locations that call number is pointed to is corresponding
All catalogues in the level at the catalogue place of the nodes of locations that quotation marks point to determine at least one idle mesh
Record.
It is also preferred that the left determine module 401 specifically for:
If all catalogues in the level at the catalogue place of the nodes of locations that call number is pointed to can not store
Data, start catalogue level one by one from root and carry out traversal lookup, determine at least one in all catalogues
Idle catalogue.
It is also preferred that the left determine according to following manner, module 401 judges whether catalogue is idle catalogue:
For a catalogue, according to the file in the attribute information of storage in the nodes of locations that this catalogue is corresponding
Number information, determines in the storage position that this catalogue is corresponding, whether the file number of storage reaches predetermined threshold value；If
File number is not reaching to predetermined threshold value, using this catalogue as idle catalogue.
It is also preferred that the left memory module 402 determines, according to following manner, the storage position that idle catalogue is corresponding:
For an idle catalogue determined, according to the simplification routing information that this free time catalogue is corresponding, Yi Jijian
Change the corresponding relation between routing information and complete path information, determine the fullpath that this free time catalogue is corresponding
Information；According to the complete path information determined, determine the storage position that this free time catalogue is corresponding.
It is also preferred that the left this device also includes:
Set up bitmap module 403, before the idle catalogue in determining the catalogue corresponding with bitmap, root
According to content and the storage state of current directory of default directory attribute information, determine each layer of catalogue
The attribute information that each catalogue in Ji is corresponding；According to continuously arranged form, respectively by each level
Attribute information corresponding to each catalogue stores nodes of locations corresponding in bitmap one by one.
It is also preferred that the left this device also includes:
Create directory module 404, for the data being currently needed for storage are being stored the idle catalogue determined
Before corresponding storage position, if the catalogue corresponding with bitmap there is no idle catalogue, at the son of lowest hierarchical level
The catalogue of lower level level is created under catalogue, and using the catalogue of the lower level level of establishment as idle catalogue.
It can be seen from the above: the method for the storage data that the embodiment of the present invention provides, is to be with file
Unit stores, the distributed memory system being managed in units of catalogue, when storing data, and can
According to the attribute information of the catalogue corresponding with this nodes of locations of storage in each nodes of locations of bitmap, to look into
Look for data-storable idle catalogue, just can find available directories without by repeatedly recursive lookup,
Compared to the way to manage of directory tree, improve search efficiency and the speed of data storage.
Those skilled in the art are it should be appreciated that embodiments of the invention can be provided as method, system or meter
Calculation machine program product.Therefore, the present invention can use complete hardware embodiment, complete software implementation or knot
The form of the embodiment in terms of conjunction software and hardware.And, the present invention can use and wherein wrap one or more
Computer-usable storage medium containing computer usable program code (include but not limited to disk memory,
CD-ROM, optical memory etc.) form of the upper computer program implemented.
The present invention is with reference to method, equipment (system) and computer program product according to embodiments of the present invention
The flow chart of product and/or block diagram describe.It should be understood that can by computer program instructions flowchart and
/ or block diagram in each flow process and/or flow process in square frame and flow chart and/or block diagram and/
Or the combination of square frame.These computer program instructions can be provided to general purpose computer, special-purpose computer, embedding
The processor of formula datatron or other programmable data processing device is to produce a machine so that by calculating
The instruction that the processor of machine or other programmable data processing device performs produces for realizing at flow chart one
The device of the function specified in individual flow process or multiple flow process and/or one square frame of block diagram or multiple square frame.
These computer program instructions may be alternatively stored in and computer or the process of other programmable datas can be guided to set
In the standby computer-readable memory worked in a specific way so that be stored in this computer-readable memory
Instruction produce and include the manufacture of command device, this command device realizes in one flow process or multiple of flow chart
The function specified in flow process and/or one square frame of block diagram or multiple square frame.
These computer program instructions also can be loaded in computer or other programmable data processing device, makes
Sequence of operations step must be performed to produce computer implemented place on computer or other programmable devices
Reason, thus the instruction performed on computer or other programmable devices provides for realizing flow chart one
The step of the function specified in flow process or multiple flow process and/or one square frame of block diagram or multiple square frame.
Although preferred embodiments of the present invention have been described, but those skilled in the art once know base
This creativeness concept, then can make other change and amendment to these embodiments.So, appended right is wanted
Ask and be intended to be construed to include preferred embodiment and fall into all changes and the amendment of the scope of the invention.
Obviously, those skilled in the art can carry out various change and modification without deviating from the present invention's to the present invention
Spirit and scope.So, if the present invention these amendment and modification belong to the claims in the present invention and etc.
Within the scope of technology, then the present invention is also intended to comprise these change and modification.
Claims (16)
1. the method storing data, it is characterised in that the method includes:
According to the attribute information of the catalogue corresponding with this nodes of locations of storage in each nodes of locations of bitmap,
Determine the idle catalogue of at least one in all catalogues corresponding with described bitmap, the most each catalogue correspondence position
A nodes of locations in figure, described idle catalogue is data-storable catalogue on corresponding storage position；
The data being currently needed for storage are stored the storage position that the idle catalogue determined is corresponding.
2. the method for claim 1, it is characterised in that described determine the institute corresponding with described bitmap
There is the idle catalogue of at least one in catalogue, including:
Determining the nodes of locations that in bitmap, call number is pointed to, the nodes of locations that wherein said call number is pointed to is upper
The nodes of locations corresponding to catalogue that once the storage position of storage data is corresponding；
If data can be stored on the storage position that the catalogue of the nodes of locations that described call number is pointed to is corresponding, will
This catalogue is as idle catalogue.
3. method as claimed in claim 2, it is characterised in that described determine the institute corresponding with described bitmap
There is the idle catalogue of at least one in catalogue, including:
If data can not be stored on the storage position that the catalogue of the nodes of locations that described call number is pointed to is corresponding,
All catalogues the level at the catalogue place of the nodes of locations pointed to from described call number determine at least one
Idle catalogue.
4. method as claimed in claim 3, it is characterised in that described determine the institute corresponding with described bitmap
There is the idle catalogue of at least one in catalogue, including:
If all catalogues in the level at the catalogue place of the nodes of locations that described call number is pointed to can not be deposited
Storage data, start catalogue level one by one from root and carry out traversal lookup, determine at least in all catalogues
Individual idle catalogue.
5. the method as described in claim 1-4 is arbitrary, it is characterised in that judge catalogue according to following manner
Whether it is idle catalogue:
For a catalogue, according to the file in the attribute information of storage in the nodes of locations that this catalogue is corresponding
Number information, determines in the storage position that this catalogue is corresponding, whether the file number of storage reaches predetermined threshold value；
If described file number is not reaching to predetermined threshold value, using this catalogue as idle catalogue.
6. the method as described in claim 1-4 is arbitrary, it is characterised in that determine the free time according to following manner
The storage position that catalogue is corresponding:
For an idle catalogue determined, according to the simplification routing information that this free time catalogue is corresponding, Yi Jijian
Change the corresponding relation between routing information and complete path information, determine the fullpath that this free time catalogue is corresponding
Information；
According to the complete path information determined, determine the storage position that this free time catalogue is corresponding.
7. the method as described in claim 1-4 is arbitrary, it is characterised in that determine corresponding with described bitmap
Before idle catalogue in catalogue, also include:
Content according to default directory attribute information and the storage state of current directory, determine catalogue
The attribute information that each catalogue in each level is corresponding；
According to continuously arranged form, respectively by attribute information corresponding for each catalogue in described each level
Store nodes of locations corresponding in bitmap one by one.
8. the method as described in claim 1-4 is arbitrary, it is characterised in that described will be currently needed for storage
Before data store the storage position that the idle catalogue determined is corresponding, also include:
If the catalogue corresponding with described bitmap does not has idle catalogue, create more under the subdirectory of lowest hierarchical level
The catalogue of low-level, and using the catalogue of the lower level level of establishment as idle catalogue.
9. the device storing data, it is characterised in that this device includes:
Determine module, for according to the mesh corresponding with this nodes of locations of storage in each nodes of locations of bitmap
The attribute information of record, determines the idle catalogue of at least one in all catalogues corresponding with described bitmap, wherein
A nodes of locations in each catalogue correspondence bitmap, described idle catalogue is can on corresponding storage position
The catalogue of storage data；
Memory module, for storing, by the data being currently needed for storage, the storage that the idle catalogue determined is corresponding
Position.
10. device as claimed in claim 9, it is characterised in that described determine module specifically for:
Determining the nodes of locations that in bitmap, call number is pointed to, the nodes of locations that wherein said call number is pointed to is upper
The nodes of locations corresponding to catalogue that once the storage position of storage data is corresponding；If described call number is pointed to
Data can be stored, using this catalogue as idle catalogue on the storage position that the catalogue of nodes of locations is corresponding.
11. devices as claimed in claim 10, it is characterised in that described determine module specifically for:
If data can not be stored on the storage position that the catalogue of the nodes of locations that described call number is pointed to is corresponding,
All catalogues the level at the catalogue place of the nodes of locations pointed to from described call number determine at least one
Idle catalogue.
12. devices as claimed in claim 11, it is characterised in that described determine module specifically for:
If all catalogues in the level at the catalogue place of the nodes of locations that described call number is pointed to can not be deposited
Storage data, start catalogue level one by one from root and carry out traversal lookup, determine at least in all catalogues
Individual idle catalogue.
13. devices as described in claim 9-12 is arbitrary, it is characterised in that described determine module according under
Row mode judges whether catalogue is idle catalogue:
For a catalogue, according to the file in the attribute information of storage in the nodes of locations that this catalogue is corresponding
Number information, determines in the storage position that this catalogue is corresponding, whether the file number of storage reaches predetermined threshold value；If
Described file number is not reaching to predetermined threshold value, using this catalogue as idle catalogue.
14. devices as described in claim 9-12 is arbitrary, it is characterised in that described memory module according under
Row mode determines the storage position that idle catalogue is corresponding:
For an idle catalogue determined, according to the simplification routing information that this free time catalogue is corresponding, Yi Jijian
Change the corresponding relation between routing information and complete path information, determine the fullpath that this free time catalogue is corresponding
Information；According to the complete path information determined, determine the storage position that this free time catalogue is corresponding.
15. devices as described in claim 9-12 is arbitrary, it is characterised in that this device also includes:
Set up bitmap module, before the idle catalogue in determining the catalogue corresponding with described bitmap, root
According to content and the storage state of current directory of default directory attribute information, determine each layer of catalogue
The attribute information that each catalogue in Ji is corresponding；According to continuously arranged form, respectively by described each level
In attribute information corresponding to each catalogue store nodes of locations corresponding in bitmap one by one.
16. devices as described in claim 9-12 is arbitrary, it is characterised in that this device also includes:
Create directory module, for the data being currently needed for storage being stored the idle catalogue determined described
Before corresponding storage position, if the catalogue corresponding with described bitmap there is no idle catalogue, in lowest hierarchical level
Subdirectory under create the catalogue of lower level level, and using the catalogue of lower level level that creates as idle catalogue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610327515.1A CN106021466A (en) | 2016-05-17 | 2016-05-17 | Data storage method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610327515.1A CN106021466A (en) | 2016-05-17 | 2016-05-17 | Data storage method and apparatus |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106021466A true CN106021466A (en) | 2016-10-12 |
Family
ID=57097396
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610327515.1A Pending CN106021466A (en) | 2016-05-17 | 2016-05-17 | Data storage method and apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106021466A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107122136A (en) * | 2017-04-25 | 2017-09-01 | 浙江宇视科技有限公司 | A kind of capacity acquiring method and device |
| CN110442570A (en) * | 2019-06-06 | 2019-11-12 | 北京左江科技股份有限公司 | A kind of BitMap high speed fuzzy search method |
| CN110928498A (en) * | 2019-11-15 | 2020-03-27 | 浙江大华技术股份有限公司 | Directory traversal method, device, equipment and storage medium |
| CN111190549A (en) * | 2019-12-30 | 2020-05-22 | 浪潮电子信息产业股份有限公司 | Method, device, equipment and medium for acquiring available capacity of shared volume |
| CN111400259A (en) * | 2020-03-24 | 2020-07-10 | 中孚信息股份有限公司 | Directory content traversal method |
| CN112817538A (en) * | 2021-02-22 | 2021-05-18 | 腾讯科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1661568A (en) * | 2004-02-24 | 2005-08-31 | 中国科学院声学研究所 | File system in device of recording and playing back sounds and images under embedded type environment |
| CN103067170A (en) * | 2012-12-14 | 2013-04-24 | 深圳国微技术有限公司 | Encrypting file system, encrypting method and deciphering method based on EXT2 file system |
| CN103559260A (en) * | 2013-11-05 | 2014-02-05 | 无锡云动科技发展有限公司 | Method for dynamically adjusting I node list on the basis of RAMDISK file system |
| CN103714013A (en) * | 2013-12-31 | 2014-04-09 | 华为技术有限公司 | Method and device for allocating storage space of file system |
| CN105607960A (en) * | 2015-10-26 | 2016-05-25 | 成都华为技术有限公司 | Repairing method and device of file system directory tree |
-
2016
- 2016-05-17 CN CN201610327515.1A patent/CN106021466A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1661568A (en) * | 2004-02-24 | 2005-08-31 | 中国科学院声学研究所 | File system in device of recording and playing back sounds and images under embedded type environment |
| CN103067170A (en) * | 2012-12-14 | 2013-04-24 | 深圳国微技术有限公司 | Encrypting file system, encrypting method and deciphering method based on EXT2 file system |
| CN103559260A (en) * | 2013-11-05 | 2014-02-05 | 无锡云动科技发展有限公司 | Method for dynamically adjusting I node list on the basis of RAMDISK file system |
| CN103714013A (en) * | 2013-12-31 | 2014-04-09 | 华为技术有限公司 | Method and device for allocating storage space of file system |
| CN105607960A (en) * | 2015-10-26 | 2016-05-25 | 成都华为技术有限公司 | Repairing method and device of file system directory tree |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107122136A (en) * | 2017-04-25 | 2017-09-01 | 浙江宇视科技有限公司 | A kind of capacity acquiring method and device |
| CN110442570A (en) * | 2019-06-06 | 2019-11-12 | 北京左江科技股份有限公司 | A kind of BitMap high speed fuzzy search method |
| CN110442570B (en) * | 2019-06-06 | 2021-08-17 | 北京左江科技股份有限公司 | BitMap high-speed fuzzy search method |
| CN110928498A (en) * | 2019-11-15 | 2020-03-27 | 浙江大华技术股份有限公司 | Directory traversal method, device, equipment and storage medium |
| CN110928498B (en) * | 2019-11-15 | 2023-11-10 | 浙江大华技术股份有限公司 | Directory traversal method, device, equipment and storage medium |
| CN111190549A (en) * | 2019-12-30 | 2020-05-22 | 浪潮电子信息产业股份有限公司 | Method, device, equipment and medium for acquiring available capacity of shared volume |
| CN111400259A (en) * | 2020-03-24 | 2020-07-10 | 中孚信息股份有限公司 | Directory content traversal method |
| CN111400259B (en) * | 2020-03-24 | 2023-04-21 | 中孚信息股份有限公司 | Method for traversing directory contents |
| CN112817538A (en) * | 2021-02-22 | 2021-05-18 | 腾讯科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
| CN112817538B (en) * | 2021-02-22 | 2022-08-30 | 腾讯科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106021466A (en) | Data storage method and apparatus | |
| US7849112B2 (en) | Using a file handle for associating the file with a tree quota in a file server | |
| US5935210A (en) | Mapping the structure of a collection of computer resources | |
| CN101673307B (en) | Space data index method and system | |
| JP5106045B2 (en) | Search engine linkage file sharing system | |
| CN106294352B (en) | A kind of document handling method, device and file system | |
| CN109471838B (en) | Directory document operation method and device, electronic equipment and readable storage medium | |
| CN106663056A (en) | Metadata index search in file system | |
| CN106021266A (en) | Fast multi-tier indexing supporting dynamic update | |
| CN102262650B (en) | The data base of link | |
| CN103870588B (en) | A kind of method and device used in data base | |
| US7769719B2 (en) | File system dump/restore by node numbering | |
| CN110321325A (en) | File inode lookup method, terminal, server, system and storage medium | |
| US10949385B2 (en) | Hybrid metadata and folder based file access | |
| CN105843867A (en) | Metadata model-based search method and search device based on metadata model | |
| EP2131293A1 (en) | Method for mapping an X500 data model onto a relational database | |
| CN113312392A (en) | Lightweight rule engine processing method and device | |
| CN107239568B (en) | Distributed index implementation method and device | |
| US8965906B1 (en) | Geographic data structuring and analysis | |
| CN115705313A (en) | Data processing method, device, equipment and computer readable storage medium | |
| WO2021082581A1 (en) | File system | |
| CN107818113B (en) | Method and device for determining file access position | |
| US20110078193A1 (en) | Query expansion through searching content identifiers | |
| CN105025013A (en) | A dynamic IP coupling model based on a priority Trie tree | |
| WO2023201002A1 (en) | Implementing graph search with in-structure metadata of a graph-organized file system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20161012 |