CN109144425A - A kind of metadata storing method, device, equipment and computer readable storage medium - Google Patents
A kind of metadata storing method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN109144425A CN109144425A CN201811043806.3A CN201811043806A CN109144425A CN 109144425 A CN109144425 A CN 109144425A CN 201811043806 A CN201811043806 A CN 201811043806A CN 109144425 A CN109144425 A CN 109144425A
- Authority
- CN
- China
- Prior art keywords
- metadata
- dirty
- meta data
- list
- pointer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0643—Management of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of metadata storing methods based on distributed cluster storage system, comprising: when first object file is by modification or creation, the dirty metadata pointer corresponding with first object file stored in directory listing is stored in dirty list of meta data；When executing the operation of dirty metadata rule, only scan the dirty metadata pointer in dirty list of meta data, will dirty metadata rule corresponding with dirty list of meta data to data pool.It can be seen that, in this programme, due to when executing modification or creation operation to file, the dirty metadata pointer for the metadata modified is stored in dirty list of meta data, therefore when executing the operation of dirty metadata rule, only needing to scan dirty list of meta data, to save the dirty metadata rule time, can improve dirty metadata rule performance for dirty metadata rule to data pool；The invention also discloses a kind of metadata storage device, equipment and computer readable storage mediums, are equally able to achieve above-mentioned technical effect.
Description
Technical field
The present invention relates to metadata rule technical fields, more specifically to one kind based on distributed type assemblies storage system
Metadata storing method, device, equipment and the computer readable storage medium of system.
Background technique
It is widely applied in the case where distributed type assemblies are stored in mass data scene, such as in high-performance, video monitoring, broadcasting and TV matchmaker money
Etc. scenes.Currently, the performance of storage is the target that distributed storage is unanimously chased, wherein for mass small documents scene, first number
Then become bottleneck according to performance.After document creation or modification, metadata needs to carry out rule storage.Currently, rule is to be with catalogue
Unit scans entire catalogue, dirty metadata is carried out rule, if quantity of documents is very huge under catalogue, which needs to consume
Take the plenty of time, rule performance can be gradually reduced with becoming larger for catalogue.
Therefore, the dirty metadata rule time how is saved, dirty metadata rule performance is improved, is that those skilled in the art need
It solves the problems, such as.
Summary of the invention
The purpose of the present invention is to provide a kind of metadata storing method based on distributed cluster storage system, device,
Equipment and computer readable storage medium save the dirty metadata rule time to realize, improve dirty metadata rule performance.
To achieve the above object, the embodiment of the invention provides following technical solutions:
A kind of metadata storing method based on distributed cluster storage system, comprising:
When first object file is by modification or creation, by being stored in directory listing with the first object file pair
The dirty metadata pointer answered is stored in dirty list of meta data；
When executing the operation of dirty metadata rule, the dirty metadata pointer in the dirty list of meta data is only scanned, it will be with
The corresponding dirty metadata rule of the dirty list of meta data is to data pool.
Wherein, described will be after dirty metadata rule to data pool corresponding with the dirty list of meta data, further includes:
By the dirty metadata pointer of rule to data pool is set to Clean state in the dirty list of meta data.
Wherein, this programme further include:
Judge whether the metadata cached in meta data server is more than predetermined threshold；
If so, deleting the metadata pointer and corresponding metadata that metadata pointer state is Clean.
Wherein, this programme further include:
When the second file destination is modified, judge whether to be found and described second according to the dirty list of meta data
The corresponding metadata of file destination；
If can, it is modified according to the dirty list of meta data pair metadata corresponding with second file destination,
And metadata pointer corresponding with second file destination in the dirty list of meta data is set to Dirty state；Wherein, it sets
Metadata pointer for Dirty state is dirty metadata pointer；
If cannot, judge whether that first number corresponding with second file destination can be found according to the directory listing
According to；If can find, modified according to the directory listing pair metadata corresponding with second file destination, and will
Metadata pointer corresponding with second file destination is set to Dirty state in the directory listing, and the dirty member is added
Data list；
If searching less than metadata and modification of the load with second file destination from data pool.
A kind of metadata storage device based on distributed cluster storage system, comprising:
Dirty metadata pointer memory module is used for when first object file is by modification or creation, will be in directory listing
The dirty metadata pointer corresponding with the first object file of storage is stored in dirty list of meta data；
Metadata rule module, for only scanning in the dirty list of meta data when executing the operation of dirty metadata rule
Dirty metadata pointer, will dirty metadata rule corresponding with the dirty list of meta data to data pool.
Wherein, this programme further include:
First pointer state setup module, for will dirty metadata rule corresponding with the dirty list of meta data to data
After pond, by the dirty metadata pointer of rule to data pool is set to Clean state in the dirty list of meta data.
Wherein, this programme further include:
First judgment module, for judging whether the metadata cached in meta data server is more than predetermined threshold；
Removing module when the metadata for caching in meta data server is more than predetermined threshold, is deleted metadata and is referred to
The metadata pointer and corresponding metadata that needle-shaped state is Clean.
Wherein, this programme further include:
Second judgment module, for when the second file destination is modified, judge whether can be according to the dirty metadata column
Table finds metadata corresponding with second file destination；
First operation module, it is corresponding with second file destination for that can be found according to the dirty list of meta data
When metadata, modified according to the dirty list of meta data pair metadata corresponding with second file destination；And by institute
It states metadata pointer corresponding with second file destination in dirty list of meta data and is set to Dirty state；Wherein, it is set to
The metadata pointer of Dirty state is dirty metadata pointer；
Third judgment module, it is corresponding with second file destination for that cannot be found according to the dirty list of meta data
Metadata when, judge whether that metadata corresponding with second file destination can be found according to the directory listing；
Second operation module, for metadata corresponding with second file destination can be searched according to the directory listing
When, it is modified according to the directory listing pair metadata corresponding with second file destination；And by the directory listing
In metadata pointer corresponding with second file destination be set to Dirty state, and the dirty list of meta data is added；
Third operation module, for first number corresponding with second file destination cannot to be searched according to the directory listing
According to when, loaded from data pool and the metadata of second file destination and modification.
A kind of metadata storage devices based on distributed cluster storage system, comprising:
Memory, for storing computer program；
Processor is realized when for executing the computer program such as the step of above-mentioned metadata storing method.
A kind of computer readable storage medium is stored with computer program on the computer readable storage medium, described
It realizes when computer program is executed by processor such as the step of above-mentioned metadata storing method.
By above scheme it is found that a kind of metadata based on distributed cluster storage system provided in an embodiment of the present invention
Storage method, comprising: when first object file is by modification or creation, by being stored in directory listing with the first object
The corresponding dirty metadata pointer of file is stored in dirty list of meta data；When executing the operation of dirty metadata rule, only scan described dirty
Dirty metadata pointer in list of meta data, will dirty metadata rule corresponding with the dirty list of meta data to data pool.
As it can be seen that in the present solution, due to when executing modification to file or creation operates, first number for will having been modified
According to dirty metadata pointer be stored in dirty list of meta data, therefore when executing the operation of dirty metadata rule, not needing scanning includes
The directory listing of all metadata pointers, it is only necessary to scan dirty list of meta data can by dirty metadata rule to data pool, from
And the dirty metadata rule time is saved, improve dirty metadata rule performance；The invention also discloses one kind to be based on distributed type assemblies
Metadata storage device, equipment and the computer readable storage medium of storage system, are equally able to achieve above-mentioned technical effect.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is a kind of metadata storing method process based on distributed cluster storage system disclosed by the embodiments of the present invention
Schematic diagram；
Fig. 2 is metadata rule overall flow schematic diagram disclosed by the embodiments of the present invention；
Fig. 3 is a kind of metadata memory device structure based on distributed cluster storage system disclosed by the embodiments of the present invention
Schematic diagram.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
The embodiment of the invention discloses a kind of metadata storing method based on distributed cluster storage system, device, set
Standby and computer readable storage medium saves the dirty metadata rule time to realize, improves dirty metadata rule performance.
Referring to Fig. 1, a kind of metadata storing method based on distributed cluster storage system provided in an embodiment of the present invention,
Include:
S101, when first object file is by modification or creation, by being stored in directory listing with the first object
The corresponding dirty metadata pointer of file is stored in dirty list of meta data；
Specifically, in the prior art, after document creation or modification, metadata needs to carry out rule storage, at present rule
It is to scan the directory listing of entire catalogue as unit of catalogue, Dirty metadata is subjected to rule；It is understood that the mesh
The metadata pointer that All Files are stored in record list, if quantity of documents is very huge under catalogue, this scanning directory column
The form of table will necessarily waste the plenty of time.
Therefore in the present embodiment, the dirty list of meta data for storing dirty metadata pointer is increased, the dirty metadata
Pointer is for being directed toward corresponding dirty metadata；That is, when first object file is modified or created, to first
After the metadata of file destination is modified, set necessary not only for by the metadata pointer of the first object file in directory listing
It is set to Dirty state, it is also necessary to the dirty metadata pointer of first object file is stored in dirty list of meta data, it is dirty in progress in this way
When metadata rule, the dirty metadata pointer in dirty list of meta data can be only scanned, to will quickly refer to dirty metadata
The corresponding dirty metadata rule of needle is to data pool.
It is understood that the metadata is dirty metadata after metadata is modified, correspondingly, the finger of the metadata
Needle is dirty metadata pointer, also, metadata pointer determines whether it is dirty metadata pointer particular by pointer state,
Such as: if metadata pointer state is Clean state, illustrate that the corresponding metadata of metadata pointer is not modified, namely
The metadata pointer is clean metadata pointer；If the metadata pointer state is Dirty state, illustrate that the metadata refers to
The corresponding metadata of needle is the dirty metadata modified, then the metadata pointer is dirty metadata pointer.
S102, execute dirty metadata rule operation when, only scan the dirty metadata pointer in the dirty list of meta data,
Will dirty metadata rule corresponding with the dirty list of meta data to data pool.
Wherein, described will be after dirty metadata rule to data pool corresponding with the dirty list of meta data, further includes: will
The dirty metadata pointer of rule to data pool is set to Clean state in the dirty list of meta data.
Specifically, at this moment only needing to scan in dirty list of meta data if necessary to execute rule operation to dirty metadata
Dirty metadata pointer, will dirty metadata rule corresponding with dirty list of meta data to data pool；Also, will be with dirty metadata column
After the corresponding dirty metadata rule to data pool of table, need for the dirty metadata pointer of rule to data pool to be set to
Clean state needs exist for the pointer that dirty metadata pointer is set to Clean state, specifically includes in directory listing and dirty member
Pointer in data list.
Further, since the catalogue of storage file is multistage, that is to say, that be also possible that second level under first class catalogue
Catalogue is also possible that three-level catalogue under second-level directory, therefore, modifies to the file in wherein any first class catalogue, must
It so will affect the file information under other grade of catalogue, such as: it modifies to the metadata under three-level catalogue, necessarily will affect two
Grade catalogue, that is to say, that can modify to the metadata under second-level directory；Therefore, this programme can create and every first class catalogue
The corresponding dirty list of meta data of directory listing, first class catalogue every so just has a dirty list of meta data, if the same level is dirty
List of meta data has modified, then the dirty list of meta data of its corresponding parent directory just needs corresponding modification；Such as: under three-level catalogue
Dirty list of meta data in metadata be modified, at this moment set Dirty for pointer state corresponding in dirty list of meta data
After state, it is also necessary to the metadata under second-level directory be modified, and will be corresponding in the dirty list of meta data of second-level directory
Pointer is set as Dirty state.
It in summary it can be seen, since this programme is when executing modification or creation operation to file, just will be modified
The dirty metadata pointer of metadata be stored in dirty list of meta data, therefore when executing the operation of dirty metadata rule, do not need
Scanning includes the directory listing of all metadata pointers, it is only necessary to which scanning dirty list of meta data can be by dirty metadata rule to number
According to pond, to save the dirty metadata rule time, dirty metadata rule performance is improved.
Based on the above embodiment, in the present embodiment, further includes:
Judge whether the metadata cached in meta data server is more than predetermined threshold；
If so, deleting the metadata pointer and corresponding metadata that metadata pointer state is Clean.
Specifically, in the present embodiment, can set a predetermined threshold, the effect of the predetermined threshold is for judging first number
It is whether excessive according to the metadata cached in server；If the metadata cached in meta data server is more than predetermined threshold,
In needing to cache rule to data pool metadata delete, i.e., deletion pointer state be Clean metadata pointer with
And corresponding metadata.
It is understood that if by after the metadata deletion of rule in caching, it is likely that occur taking from metadata
Be engaged in device in search less than corresponding metadata the case where, therefore in the present solution, further include:
When the second file destination is modified, judge whether to be found and described second according to the dirty list of meta data
The corresponding metadata of file destination；
If can, it is modified according to the dirty list of meta data pair metadata corresponding with second file destination,
And metadata pointer corresponding with second file destination in the dirty list of meta data is set to Dirty state；Wherein, it sets
Metadata pointer for Dirty state is dirty metadata pointer；
If cannot, judge whether that first number corresponding with second file destination can be found according to the directory listing
According to；If can find, modified according to the directory listing pair metadata corresponding with second file destination, and will
Metadata pointer corresponding with second file destination is set to Dirty state in the directory listing, and the dirty member is added
Data list；
If searching less than metadata and modification of the load with second file destination from data pool.
Specifically, this programme is when finding metadata, due to being already provided with dirty list of meta data, and dirty list of meta data
Compared with directory listing, the metadata pointer of storage is less, therefore preferentially can search metadata from dirty list of meta data,
The time is searched so as to shorten, just executes modification operation if can find；If can not find out, looked into from directory listing
It looks for, if can find from directory listing, continues to execute modification operation；If from dirty list of meta data and directory listing
When cannot find metadata namely metadata and reading cache failure, illustrating the metadata, rule is to disk, in caching
It is not present, at this moment needs to be loaded directly into metadata from disk, thereby executing modification operation.
Referring to fig. 2, the metadata rule overall flow schematic diagram provided for this programme, Dir (the Directory mesh in Fig. 2
Record) it is directory listing in the present embodiment, for the metadata pointer of All Files under storage catalogue, Dir_list
(Dirty_list) it is dirty list of meta data in the present embodiment；If there is file modification or creation in catalogue, by dirty data
Pointer is directly stored in Dirty_list list, such as workflow graph A；It, will be corresponding in Dirty_list list in metadata rule
Metadata whole rule, and dirty data pointer is set to Clean state；Further, if during metadata rule, first number
According to being changed, it can search and modify corresponding metadata in Dir_list list, if searched from Dir_list list
Less than then going to search in Dir and modify corresponding metadata.
But after exceeding the caching upper limit due to the metadata cache in meta data server, it will be deleted the member of rule
Data, therefore, if cannot find corresponding metadata from Dir_list list and Dir, i.e., metadata is not in metadata
When in the caching of server, then directly from read in data pool namely figure in process C.
It in summary it can be seen, this programme is when executing modification to file or creation operates, first number for will having been modified
According to dirty metadata pointer be stored in dirty list of meta data, therefore when executing the operation of dirty metadata rule, not needing scanning includes
The directory listing of all metadata pointers, it is only necessary to scan dirty list of meta data can by dirty metadata rule to data pool, from
And the dirty metadata rule time is saved, improve dirty metadata rule；Also, it is provided with predetermined threshold, if metadata cache
It is excessive, then the case where needing to delete the metadata of rule, read failure if there is metadata, then directly from data pool
Middle load, to save the memory space of meta data server.
This metadata storage performance optimization method is that metadata is stored as unit of catalogue apoplexy involving the solid organs data list to magnetic
Disk；Metadata is read from disk as unit of catalogue, has great improvement for promoting cluster small documents process performance, especially
For big catalogue scene, there can be better improvement.It is understood that this method can not only be applied to distributed storage
In system, it can be applied in unified storage.
Metadata storage device provided in an embodiment of the present invention is introduced below, metadata storage dress described below
Setting can be cross-referenced with above-described metadata storing method.
Referring to Fig. 3, a kind of metadata storage device based on distributed cluster storage system provided in an embodiment of the present invention,
Include:
Dirty metadata pointer memory module 100 is used for when first object file is by modification or creation, by directory listing
The dirty metadata pointer corresponding with the first object file of middle storage is stored in dirty list of meta data；
Metadata rule module 200, for only scanning the dirty list of meta data when executing the operation of dirty metadata rule
In dirty metadata pointer, will dirty metadata rule corresponding with the dirty list of meta data to data pool.
Wherein, this programme further include:
First pointer state setup module, for will dirty metadata rule corresponding with the dirty list of meta data to data
After pond, by the dirty metadata pointer of rule to data pool is set to Clean state in the dirty list of meta data.
Wherein, this programme further include:
First judgment module, for judging whether the metadata cached in meta data server is more than predetermined threshold；
Removing module when the metadata for caching in meta data server is more than predetermined threshold, is deleted metadata and is referred to
The metadata pointer and corresponding metadata that needle-shaped state is Clean.
Wherein, this programme further include:
Second judgment module, for when the second file destination is modified, judge whether can be according to the dirty metadata column
Table finds metadata corresponding with second file destination；
First operation module, it is corresponding with second file destination for that can be found according to the dirty list of meta data
When metadata, modified according to the dirty list of meta data pair metadata corresponding with second file destination；And by institute
It states metadata pointer corresponding with second file destination in dirty list of meta data and is set to Dirty state；Wherein, it is set to
The metadata pointer of Dirty state is dirty metadata pointer；
Third judgment module, it is corresponding with second file destination for that cannot be found according to the dirty list of meta data
Metadata when, judge whether that metadata corresponding with second file destination can be found according to the directory listing；
Second operation module, for metadata corresponding with second file destination can be searched according to the directory listing
When, it is modified according to the directory listing pair metadata corresponding with second file destination；And by the directory listing
In metadata pointer corresponding with second file destination be set to Dirty state, and the dirty list of meta data is added；
Third operation module, for first number corresponding with second file destination cannot to be searched according to the directory listing
According to when, loaded from data pool and the metadata of second file destination and modification.
The embodiment of the invention also provides a kind of metadata storage devices based on distributed cluster storage system, comprising:
Memory, for storing computer program；
Processor realizes the metadata storage side as described in above method embodiment when for executing the computer program
The step of method.
The embodiment of the invention also provides a kind of computer readable storage medium, deposited on the computer readable storage medium
Computer program is contained, realizes that the metadata as described in above method embodiment is deposited when the computer program is executed by processor
The step of method for storing.
Wherein, the storage medium may include: USB flash disk, mobile hard disk, read-only memory (Read-Only Memory,
ROM), random access memory (Random Access Memory, RAM), magnetic or disk etc. are various can store program
The medium of code.
Each embodiment in this specification is described in a progressive manner, the highlights of each of the examples are with other
The difference of embodiment, the same or similar parts in each embodiment may refer to each other.
The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention.
Various modifications to these embodiments will be readily apparent to those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, of the invention
It is not intended to be limited to the embodiments shown herein, and is to fit to and the principles and novel features disclosed herein phase one
The widest scope of cause.
Claims (10)
1. a kind of metadata storing method based on distributed cluster storage system characterized by comprising
It is corresponding with the first object file by what is stored in directory listing when first object file is by modification or creation
Dirty metadata pointer is stored in dirty list of meta data；
Execute dirty metadata rule operation when, only scan the dirty metadata pointer in the dirty list of meta data, will with it is described
The corresponding dirty metadata rule of dirty list of meta data is to data pool.
2. metadata storing method according to claim 1, which is characterized in that it is described will be with the dirty list of meta data pair
After the dirty metadata rule to data pool answered, further includes:
By the dirty metadata pointer of rule to data pool is set to Clean state in the dirty list of meta data.
3. metadata storing method according to claim 2, which is characterized in that further include:
Judge whether the metadata cached in meta data server is more than predetermined threshold；
If so, deleting the metadata pointer and corresponding metadata that metadata pointer state is Clean.
4. metadata storing method according to claim 3, which is characterized in that further include:
When the second file destination is modified, judge whether to be found according to the dirty list of meta data and second target
The corresponding metadata of file；
If can, it is modified according to the dirty list of meta data pair metadata corresponding with second file destination, and will
Metadata pointer corresponding with second file destination is set to Dirty state in the dirty list of meta data；Wherein, it is set to
The metadata pointer of Dirty state is dirty metadata pointer；
If cannot, judge whether that metadata corresponding with second file destination can be found according to the directory listing；If
It can find, then be modified according to the directory listing pair metadata corresponding with second file destination, and will be described
Metadata pointer corresponding with second file destination is set to Dirty state in directory listing, and the dirty metadata is added
List；
If searching less than metadata and modification of the load with second file destination from data pool.
5. a kind of metadata storage device based on distributed cluster storage system characterized by comprising
Dirty metadata pointer memory module, for will be stored in directory listing when first object file is by modification or creation
Dirty metadata pointer corresponding with the first object file be stored in dirty list of meta data；
Metadata rule module, for only scanning dirty in the dirty list of meta data when executing the operation of dirty metadata rule
Metadata pointer, will dirty metadata rule corresponding with the dirty list of meta data to data pool.
6. metadata storage device according to claim 5, which is characterized in that further include:
First pointer state setup module, for will dirty metadata rule corresponding with the dirty list of meta data to data pool it
Afterwards, by the dirty metadata pointer of rule to data pool is set to Clean state in the dirty list of meta data.
7. metadata storage device according to claim 6, which is characterized in that further include:
First judgment module, for judging whether the metadata cached in meta data server is more than predetermined threshold；
Removing module deletes metadata pointer shape when the metadata for caching in meta data server is more than predetermined threshold
The metadata pointer and corresponding metadata that state is Clean.
8. metadata storage device according to claim 7, which is characterized in that further include:
Second judgment module can be looked into for judging whether when the second file destination is modified according to the dirty list of meta data
Find metadata corresponding with second file destination；
First operation module, for first number corresponding with second file destination can be found according to the dirty list of meta data
According to when, modified according to the dirty list of meta data pair metadata corresponding with second file destination；And it will be described dirty
Metadata pointer corresponding with second file destination is set to Dirty state in list of meta data；Wherein, it is set to Dirty shape
The metadata pointer of state is dirty metadata pointer；
Third judgment module, for member corresponding with second file destination cannot to be found according to the dirty list of meta data
When data, judge whether that metadata corresponding with second file destination can be found according to the directory listing；
Second operation module, when for metadata corresponding with second file destination can be searched according to the directory listing,
It is modified according to the directory listing pair metadata corresponding with second file destination；And by the directory listing with
The corresponding metadata pointer of second file destination is set to Dirty state, and the dirty list of meta data is added；
Third operation module, for metadata corresponding with second file destination cannot to be searched according to the directory listing
When, it is loaded from data pool and the metadata of second file destination and modification.
9. a kind of metadata storage devices based on distributed cluster storage system characterized by comprising
Memory, for storing computer program；
Processor realizes such as Claims 1-4 described in any item metadata storage sides when for executing the computer program
The step of method.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program realizes such as Claims 1-4 described in any item metadata storing methods when the computer program is executed by processor
The step of.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811043806.3A CN109144425A (en) | 2018-09-07 | 2018-09-07 | A kind of metadata storing method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811043806.3A CN109144425A (en) | 2018-09-07 | 2018-09-07 | A kind of metadata storing method, device, equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109144425A true CN109144425A (en) | 2019-01-04 |
Family
ID=64823696
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811043806.3A Pending CN109144425A (en) | 2018-09-07 | 2018-09-07 | A kind of metadata storing method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109144425A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111158594A (en) * | 2019-12-27 | 2020-05-15 | 浪潮电子信息产业股份有限公司 | Metadata disk-dropping method, device, equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101573370B1 (en) * | 2007-11-21 | 2015-12-01 | 바이올린 메모리 인코포레이티드 | Method and system for storage of data in non-volatile media |
| CN106201918A (en) * | 2016-07-14 | 2016-12-07 | 合肥易立迅科技有限公司 | A kind of method and system quickly discharged based on big data quantity and extensive caching |
| WO2017113213A1 (en) * | 2015-12-30 | 2017-07-06 | 华为技术有限公司 | Method and device for processing access request, and computer system |
| CN107402870A (en) * | 2017-07-31 | 2017-11-28 | 郑州云海信息技术有限公司 | The processing method and processing device of daily record section in a kind of meta data server |
-
2018
- 2018-09-07 CN CN201811043806.3A patent/CN109144425A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101573370B1 (en) * | 2007-11-21 | 2015-12-01 | 바이올린 메모리 인코포레이티드 | Method and system for storage of data in non-volatile media |
| WO2017113213A1 (en) * | 2015-12-30 | 2017-07-06 | 华为技术有限公司 | Method and device for processing access request, and computer system |
| CN106201918A (en) * | 2016-07-14 | 2016-12-07 | 合肥易立迅科技有限公司 | A kind of method and system quickly discharged based on big data quantity and extensive caching |
| CN107402870A (en) * | 2017-07-31 | 2017-11-28 | 郑州云海信息技术有限公司 | The processing method and processing device of daily record section in a kind of meta data server |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111158594A (en) * | 2019-12-27 | 2020-05-15 | 浪潮电子信息产业股份有限公司 | Metadata disk-dropping method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107169083B (en) | Mass vehicle data storage and retrieval method and device for public security card port and electronic equipment | |
| CN101446984B (en) | Method and device for storing files and method and device for deleting files | |
| US10013440B1 (en) | Incremental out-of-place updates for index structures | |
| CN101763437B (en) | Method and device for realizing high-speed buffer storage | |
| CN102411628B (en) | Android system-based file scanning method and device | |
| RU2646349C2 (en) | Data processing method and system | |
| CN102567427B (en) | Method and device for processing object data | |
| CN103595797B (en) | Caching method for distributed storage system | |
| CN111198856B (en) | File management method, device, computer equipment and storage medium | |
| CN107241444B (en) | Distributed cache data management system, method and device | |
| KR101674176B1 (en) | Method and apparatus for fsync system call processing using ordered mode journaling with file unit | |
| CN110134335B (en) | RDF data management method and device based on key value pair and storage medium | |
| EP3789883A1 (en) | Storage fragment managing method and terminal | |
| CN110888837A (en) | Object storage small file merging method and device | |
| CN103942301B (en) | Distributed file system oriented to access and application of multiple data types | |
| CN104991949A (en) | Mobile terminal and file management system and method thereof | |
| US10628305B2 (en) | Determining a data layout in a log structured storage system | |
| JP6788002B2 (en) | Data storage methods and devices for mobile devices | |
| CN111125002A (en) | Data backup filing method and system based on distributed storage | |
| US20220342888A1 (en) | Object tagging | |
| CN108280123B (en) | HBase column polymerization method | |
| CN109144425A (en) | A kind of metadata storing method, device, equipment and computer readable storage medium | |
| CN101483668A (en) | Network storage and access method, device and system for hot spot data | |
| CN113326262A (en) | Data processing method, device, equipment and medium based on key value database | |
| CN112328587A (en) | Data processing method and device for ElasticSearch |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190104 |