CN102110146B - Key-value storage-based distributed file system metadata management method - Google Patents
Key-value storage-based distributed file system metadata management method Download PDFInfo
- Publication number
- CN102110146B CN102110146B CN2011100392697A CN201110039269A CN102110146B CN 102110146 B CN102110146 B CN 102110146B CN 2011100392697 A CN2011100392697 A CN 2011100392697A CN 201110039269 A CN201110039269 A CN 201110039269A CN 102110146 B CN102110146 B CN 102110146B
- Authority
- CN
- China
- Prior art keywords
- key
- subobject
- metadata
- catalogue
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007726 management method Methods 0.000 title claims abstract description 20
- 238000000034 method Methods 0.000 claims abstract description 26
- 238000012217 deletion Methods 0.000 claims description 28
- 230000037430 deletion Effects 0.000 claims description 28
- 230000015572 biosynthetic process Effects 0.000 claims description 4
- 230000000737 periodic effect Effects 0.000 claims description 3
- 230000004048 modification Effects 0.000 abstract description 6
- 238000012986 modification Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 17
- 238000005516 engineering process Methods 0.000 description 3
- 230000002567 autonomic effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a key-value storage-based distributed file system metadata management method, and mainly relates to the field of file systems. The method comprises the following steps of: storing files and metadata of a catalog and storing sub objects of the catalog in a bitmap mode by a server; modifying the metadata of the sub objects; and periodically scanning the operation log by a background process, and performing garbage reclamation on the overtime operation. Compared with the traditional distributed file system, the key-value storage-based distributed file system metadata storage management method has better expansibility; and meanwhile, because the management mode of log record and garbage reclamation is adopted, the consistency of the metadata is kept in the metadata modification process.
Description
Technical field
The present invention relates to field of filesystems, particularly a kind of metadata of distributed type file system memory management method based on key assignments key-value storage.
Background technology
The metadata of file system comprises the metadata of catalogue and file, and bibliographic structure etc.The metadata of file comprises: filename, and authority, the time, file size, and the distribution of data block etc.The metadata of catalogue comprises: main attribute such as directory name, authority, time, and bibliographic structure related contents such as son file that it comprised and sub-directory.A difficult point of file system metadata management is that metadata access efficiently should be provided, and for the modification of metadata interface flexibly is provided again.
Distributed file system replaces traditional local file system because of the advantage of its mass memory at internet arena just gradually.Ceph is the typical distributed file system, the metadata that it adopts object-based distributed storage cluster (RADOS, Reliable Autonomic Distributed Object Storage) to come memory file system; RADOS can expand for Ceph provides, object storage reliably, but Ceph catalogue also as an independent object, the related information of the subobject of catalogue all is stored in the directory object; When a lot of processes were created a large amount of subobject in certain catalogue at Ceph concomitantly, in order to guarantee the consistance of metadata, these operations can only be carried out in serial, cause the system performance bottleneck.
The key-value storage can be for the metadata of distributed type file system storage administration brings better expansibility, if the data of handling continue to increase, it is just passable to add machine, does not have system bottleneck problem.But the metadata of distributed type file system storage tube that is based on key-value storage is comprehended and brought some new challenges: the operation of metadata of distributed type file system relates generally to a plurality of steps; Relate to a plurality of key-value operations, and the transaction operation of a plurality of key-value is not supported in the key-value storage.Therefore how guaranteeing in certain step, to have no progeny, can realize garbage-cleaning through certain measure, guarantee the consistance of metadata, is technical issues that need to address.
Summary of the invention
The technical matters that (one) will solve
The technical matters that the present invention will solve is: how a kind of metadata of distributed type file system management method based on key assignments key-value storage is provided, and guarantees in the operating process to metadata, keep the consistance of metadata.
(2) technical scheme
For solving the problems of the technologies described above, the present invention provides a kind of metadata of distributed type file system management method based on key assignments key-value storage, and this method may further comprise the steps:
S1: the metadata of server stores file and catalogue, and with the subobject of the mode storage directory of bitmap; Said subobject comprises son file and sub-directory;
S2: to the operation of making amendment of the metadata of subobject; Before carrying out said retouching operation, with the directory content sign dirhdl-ID of the parent directory of said subobject and the bitmap index structure key key of said subobject, be value value with the current time, record in the Operation Log;
S3: the said Operation Log of background process periodic scanning, carry out garbage reclamation to overtime operation.
Preferably, three key-value tables of storage in the said server: log sheet log.tbl, the table meta.tbl of unit and order table entry.tbl;
The base attribute record of the metadata of said file and catalogue, and the directory content recording storage of the metadata of said catalogue is in the said table meta.tbl of unit;
The incidence relation of said catalogue and its subobject is stored among the said order table entry.tbl.
Preferably, the base attribute record of the metadata of said file is key key with file identification doc-ID, serves as value value with the base attribute information of file;
The base attribute record of the metadata of said catalogue is key key with catalogue sign dir-ID, serves as value value with the base attribute information of catalogue;
The directory content record of the metadata of said catalogue is key key with directory content sign dirhdl-ID, serves as value value with the bitmap structure of the subobject that catalogue was comprised.
Preferably, the incidence relation of said catalogue and its subobject comprises:
The said catalogue sign dir-ID and first separating character constitute key key, and said directory content sign dirhdl-ID is value value, and both constitute first association;
Said directory content sign dirhdl-ID constitutes key key through the bitmap index that second separating character connects said subobject, subobject sign and subobject name formation value value, and both constitute second association;
Said directory content sign dirhdl-ID connects said subobject name through the 3rd separating character and constitutes key key, and the subobject bitmap index is value value, and both constitute the 3rd association.
Preferably, among the said step S2, said metadata to subobject is made amendment and is comprised: create the metadata of said subobject and the metadata of the said subobject of deletion.
Preferably, the metadata of creating said subobject specifically may further comprise the steps:
S2.1.1: the base attribute record of creating said subobject; If said subobject is a catalogue, it is related with first to create its directory content record;
S2.1.2: write operation daily record;
S2.1.3: create said the 3rd association, second association, revise the directory content record of the parent directory of said subobject;
S2.1.4: deletion action daily record.
The metadata of preferably, deleting said subobject specifically may further comprise the steps:
S2.2.1: write operation daily record;
S2.2.2: revise the directory content record of the parent directory of said subobject, delete said the 3rd association, second association;
S2.2.3: deletion action daily record;
S2.2.4: the base attribute record of deleting said subobject; If said subobject is a catalogue, it is related with first to delete its directory content record.
Preferably, among the said step S3, said garbage reclamation specifically may further comprise the steps:
S3.1.1: the directory content record of revising said parent directory;
S3.1.2: resolve the key key in the said Operation Log, obtain said subobject sign and said subobject name, delete said the 3rd association, second association;
S3.1.3: the base attribute record of deletion subobject; If said subobject is a catalogue, resolve and obtain its directory content sign, it is related with first to delete its directory content record;
S3.1.4: the overtime record of deletion from Operation Log.
Preferably, among the said step S2, said key key connects the directory content sign dirhdl-ID of said parent directory by the 4th separating character and the bitmap index of said subobject constitutes.
Preferably, among the said step S3, whether decision operation overtime may further comprise the steps:
S3.0.1: resolve said key key, obtain running time opTime;
S3.0.2: deduct said running time opTime with current time curTime, judge that whether its difference is greater than overtime standard value timeout; If greater than, operation overtime then; Otherwise, operate not overtime.
(3) beneficial effect
The invention provides a kind of metadata of distributed type file system memory management method based on key assignments key-value storage; It adopts key assignments key-value storage that the metadata of distributed file system is managed, and has than traditional distributed file system better expansibility.Simultaneously, it adopts the management mode of log record and garbage reclamation, in to the metadata modification process, has kept the consistance of metadata.
Description of drawings
Fig. 1 is the said metadata of distributed type file system memory management method process flow diagram based on key assignments key-value storage of the embodiment of the invention;
Fig. 2 is the said metadata of distributed type file system storage node composition based on key assignments key-value storage of the embodiment of the invention;
Fig. 3 is the said establishment subobject of an embodiment of the invention process data structural drawing;
Fig. 4 is the said deletion subobject of an embodiment of the invention process data structural drawing;
Fig. 5 is the said garbage reclamation process data of an embodiment of the invention structural drawing.
Embodiment
Below in conjunction with accompanying drawing and embodiment, specific embodiments of the invention describes in further detail.Following examples are used to explain the present invention, but are not used for limiting scope of the present invention.
Fig. 1 is the said metadata of distributed type file system management method process flow diagram based on key assignments key-value storage of the embodiment of the invention, and as shown in Figure 1, this method may further comprise the steps:
S1: the metadata of server stores file and catalogue, and with the subobject of the mode storage directory of bitmap; Said subobject comprises son file and sub-directory.
S2: to the operation of making amendment of the metadata of subobject; Before carrying out said retouching operation, with the directory content sign dirhdl-ID of the parent directory of said subobject and the bitmap index structure key key of said subobject, be value value with the current time, record in the Operation Log.Said key key and value value are stored among the log sheet log.tbl of said server as log record.
S3: the said Operation Log of background process periodic scanning, carry out garbage reclamation to overtime operation.
The method of the embodiment of the invention adopts key assignments key-value storage, and the metadata of distributed file system is managed, and it has than traditional distributed file system better expansibility.Simultaneously, the garbage reclamation among log record among the step S2 and the step S3 has guaranteed in to the metadata modification process, to remain the consistance of metadata.
Fig. 2 is the said metadata of distributed type file system storage node composition based on key assignments key-value storage of the embodiment of the invention, and is as shown in Figure 2, and its expression catalogue I comprises catalogue b and file c.Three key-value tables of storage in the said server: log sheet log.tbl, the table meta.tbl1 of unit and order table entry.tbl2.The base attribute record of the metadata of catalogue I, catalogue b and file c, and the directory content recording storage of the metadata of catalogue I and catalogue b is in the table meta.tbl1 of unit.The incidence relation of catalogue I and catalogue b, file c is stored among the said order table entry.tbl2.
In the base attribute record of the metadata of file c, key key is 5, and value value is the base attribute information of file c; The base attribute information of said file comprises: information such as access rights, creation-time, access time and distribution of content.
The base attribute record of the metadata of catalogue I, key key are 1, and value value is the base attribute information of catalogue I; The base attribute information of said catalogue comprises: information such as access rights, creation-time, access time.
The directory content record of the metadata of catalogue I, key key are 2, and value value is the bitmap structure of the subobject that comprised of catalogue I.Among Fig. 2 in the bitmap structure of catalogue I [1100...] set be 1 two corresponding catalogue b of difference and file c.
The incidence relation of catalogue I and its subobject catalogue b and file c comprises:
The catalogue of catalogue I sign dir-ID i.e. 1 and first separating character " $ " constitutes key key, and the directory content sign dirhdl-ID of catalogue I promptly 2 is value value, and both constitute first association, write down among the corresponding diagram 2 order table entry.tbl 2 < 1+ " $ ", 2 >.Said first separating character can also wait other characters for " * ", " # ", " ", but it must distinguish over other special characters of the embodiment of the invention.
It is that 2 bitmap indexs through the said subobject catalogue b of second separating character " " connection are 0 formation key key that the directory content of catalogue I identifies dirhdl-ID; The catalogue of catalogue b sign dir-ID promptly 3 with title " b " the formation value value of catalogue b; Both constitute second association; Record < 2+ " "+0, { 3, " b " }>among the corresponding diagram 2 order table entry.tbl 2.Said second separating character can also wait other characters for " * ", " # ", " $ ", but it must distinguish over other special characters of the embodiment of the invention.
The directory content sign dirhdl-ID of catalogue I i.e. 2 titles " b " through the 3rd separating character " # " connexon object directory b constitutes key key; The bitmap index of catalogue b promptly 0 is value value; Both constitute the 3rd association; Record < 2+ " # "+" b ", 0>among the corresponding diagram 2 order table entry.tbl 2.Said the 3rd separating character can also wait other characters for " * ", " ", " $ ", but it must distinguish over other special characters of the embodiment of the invention.
Preferably, among the said step S2, said metadata to subobject is made amendment and is comprised: create the metadata of said subobject and the metadata of the said subobject of deletion.
Fig. 3 is the said establishment subobject of embodiment of the invention process data structural drawing, and is as shown in Figure 3, and the metadata of the subobject catalogue b of the I that creaties directory specifically may further comprise the steps:
S2 ' .1.1: the base attribute record of the b that creaties directory; Like StepC1 in the form 1; Unit's table meta.tbl 1 increase record among Fig. 33, DIR...} >.
The directory content record of b of createing directory is first related with it; Like StepC2 and StepC3 in the form 1; Unit's table meta.tbl 1 increases record < 4, { [0000...] }>in the corresponding diagram 3, and order table entry.tbl 2 increases record < 3+ " $ ", 4>among Fig. 3.
S2 ' .1.2: write operation daily record; Like StepC4 in the form 1; Increase record < 2+ " ^ "+0, curTime>in the corresponding diagram 3 among the log sheet log.tbl3.
S2 ' .1.3: create said the 3rd association, second association, revise the directory content record of catalogue I; Like StepC5～StepC7 in the form 1; Order table entry.tbl 2 increases record < 2+ " # "+" b ", 0>and < 2+ " "+0, { 3, " b " }>in the corresponding diagram 3, and amendment record < 2, { [0000...] }>is < 2, { [1000...] }>among the table meta.tbl1 of unit.
S2 ' .1.4: deletion action daily record; Like StepC8 in the form 1; Deletion record among the log sheet log.tbl 3 in the corresponding diagram 3 < 2+ " ^ "+0, curTime>returns to the preceding state of creation operation.
Table 1 is created the metadata correlation step of subobject
Fig. 4 is the said deletion subobject of embodiment of the invention process data structural drawing, and is as shown in Figure 4, and the metadata of the subobject catalogue b of the I that deltrees specifically may further comprise the steps:
S2 ' .2.1: write operation daily record; Like StepR0～StepR1 in the form 2; Increase record < 2+ " ^ "+0, curTime>in the corresponding diagram 4 among the log sheet log.tbl 3.Step StepR0 be used for according to the directory content of catalogue I sign dirhdl-ID promptly 2 with by the file key key inquiry by name order table entry.tbl 2 of deletion subobject catalogue b, to obtain the bitmap index of catalogue b, its index value is 0.
S2 ' .2.2: revise the directory content record of said catalogue I, delete said second association, the 3rd association; Like StepR2StepR4 in the form 2; Order table entry.tbl 2 deletion records < 2+ " # "+" b ", 0>and < 2+ " "+0, { 3, " b " }>in the corresponding diagram 4, amendment record < 2, { [1000...] }>is < 2, { [0000...] }>among the table meta.tbl1 of unit.
S2 ' .2.3: deletion action daily record; Like StepR5 in the form 2; Deletion record among the log sheet log.tbl 3 in the corresponding diagram 4 < 2+ " ^ "+0, curTime>returns to the preceding state of deletion action.
S2 ' .2.4: the base attribute record of the b that deltrees, directory content write down related with first; Like StepR6StepR8 in the form 2; Order table entry.tbl 2 deletion records < 3+ " $ ", 4>in the corresponding diagram 3, meta.tbl 1 deletion record < 4, { [0000...] }>of unit table and < 3, { DIR...}>among Fig. 3.
The metadata correlation step of table 2 deletion subobject
Fig. 5 is the said garbage reclamation process data of embodiment of the invention structural drawing, and is as shown in Figure 5,
The subobject catalogue b process of I of supposing to create directory is overtime, and its garbage reclamation process specifically may further comprise the steps:
S3 ' .1.1: the directory content record of revising catalogue I; Like Step1 in the form 3; In the corresponding diagram 5 among the unit table meta.tbl 1 amendment record < 2, { [1000...] }>be < 2, { [0000...] } >.
S3 ' .1.2: the key key in the parse operation daily record, the catalogue sign dir-ID that obtains catalogue b promptly 3 with its title " b ", delete said the 3rd association, second association; Like Step2～Step3 in the form 3; Order table entry.tbl 2 deletion records < 2+ " # "+" b ", 0>and < 2+ " "+0, { 3, " b " }>in the corresponding diagram 5.
S3 ' .1.3: resolving the directory content sign dirhdl-ID that obtains catalogue b is 4, and its first related and base attribute record deleted in the directory content record of the b that deltrees; Like Step4～Step7 in the form 3; Deletion record < 4, { [0000...] }>and < 3, { DIR...}>among the unit table meta.tbl 1 in the corresponding diagram 5, deletion record among the order table entry.tbl 2 < 3+ " $ ", 4 >.
S3 ' .1.4: the overtime record of deletion from Operation Log; Like Step8 in the form 3; Deletion record among the log sheet log.tbl 3 in the corresponding diagram 5 < 2+ " ^ "+0, timestamp >.
Table 3 garbage reclamation process correlation step
Preferably, among the said step S2, said key key connects the directory content sign dirhdl-ID of said parent directory by the 4th separating character " ^ " and the bitmap index of said subobject constitutes.Said the 4th separating character can also wait other characters for " * ", " ", " $ ", but it must distinguish over other special characters of the embodiment of the invention.
Preferably, among the said step S3, whether decision operation overtime may further comprise the steps:
S3.0.1: resolve said key key, obtain running time opTime;
S3.0.2: deduct said running time opTime with current time curTime, judge that whether its difference is greater than overtime standard value timeout; If greater than, operation overtime then; Otherwise, operate not overtime.
Garbage reclamation among log record among the step S2 and the step S3 has guaranteed in to the metadata modification process, to remain the consistance of metadata.
Above embodiment only is used to explain the present invention; And be not limitation of the present invention; The those of ordinary skill in relevant technologies field under the situation that does not break away from the spirit and scope of the present invention, can also be made various variations and modification; Therefore all technical schemes that are equal to also belong to category of the present invention, and scope of patent protection of the present invention should be defined by the claims.
Claims (7)
1. the metadata of distributed type file system management method based on key assignments key-value storage is characterized in that, said method comprising the steps of:
S1: the metadata of server stores file and catalogue, and with the subobject of the mode storage directory of bitmap; Said subobject comprises son file and sub-directory;
Three key-value tables of storage in the said server: log sheet log.tbl, the table meta.tbl of unit and order table entry.tbl;
The base attribute record of the metadata of said file and catalogue, and the directory content recording storage of the metadata of said catalogue is in the said table meta.tbl of unit;
The incidence relation of said catalogue and its subobject is stored among the said order table entry.tbl;
The incidence relation of said catalogue and its subobject comprises:
The said catalogue sign dir-ID and first separating character constitute key key, and said directory content sign dirhdl-ID is value value, and both constitute first association;
Said directory content sign dirhdl-ID constitutes key key through the bitmap index that second separating character connects said subobject, subobject sign and subobject name formation value value, and both constitute second association;
Said directory content sign dirhdl-ID connects said subobject name through the 3rd separating character and constitutes key key, and the subobject bitmap index is value value, and both constitute the 3rd association;
S2: to the operation of making amendment of the metadata of subobject; Before carrying out said retouching operation, with the directory content sign dirhdl-ID of the parent directory of said subobject and the bitmap index structure key key of said subobject, be value value with the current time, record in the Operation Log;
S3: the said Operation Log of background process periodic scanning, carry out garbage reclamation to overtime operation;
Said garbage reclamation specifically may further comprise the steps:
S3.1.1: the directory content record of revising said parent directory;
S3.1.2: resolve the key key in the said Operation Log, obtain said subobject sign and said subobject name, delete said the 3rd association, second association;
S3.1.3: the base attribute record of deletion subobject; If said subobject is a catalogue, resolve and obtain its directory content sign, it is related with first to delete its directory content record;
S3.1.4: the overtime record of deletion from Operation Log.
2. the metadata of distributed type file system management method based on key assignments key-value storage as claimed in claim 1 is characterized in that,
The base attribute record of the metadata of said file is key key with file identification doc-ID, serves as value value with the base attribute information of file;
The base attribute record of the metadata of said catalogue is key key with catalogue sign dir-ID, serves as value value with the base attribute information of catalogue;
The directory content record of the metadata of said catalogue is key key with directory content sign dirhdl-ID, serves as value value with the bitmap structure of the subobject that catalogue was comprised.
3. the metadata of distributed type file system management method based on key assignments key-value storage as claimed in claim 1 is characterized in that, among the said step S2,
Said metadata to subobject is made amendment and is comprised: create the metadata of said subobject and the metadata of the said subobject of deletion.
4. the metadata of distributed type file system management method based on key assignments key-value storage as claimed in claim 3 is characterized in that the metadata of creating said subobject specifically may further comprise the steps:
S2.1.1: the base attribute record of creating said subobject; If said subobject is a catalogue, it is related with first to create its directory content record;
S2.1.2: write operation daily record;
S2.1.3: create said the 3rd association, second association, revise the directory content record of the parent directory of said subobject;
S2.1.4: deletion action daily record.
5. the metadata of distributed type file system management method based on key assignments key-value storage as claimed in claim 3 is characterized in that the metadata of deleting said subobject specifically may further comprise the steps:
S2.2.1: write operation daily record;
S2.2.2: revise the directory content record of the parent directory of said subobject, delete said the 3rd association, second association;
S2.2.3: deletion action daily record;
S2.2.4: the base attribute record of deleting said subobject; If said subobject is a catalogue, it is related with first to delete its directory content record.
6. the metadata of distributed type file system management method based on key assignments key-value storage as claimed in claim 1; It is characterized in that; Among the said step S2, said key key connects the directory content sign dirhdl-ID of said parent directory by the 4th separating character and the bitmap index of said subobject constitutes.
7. the metadata of distributed type file system management method based on key assignments key-value storage as claimed in claim 1 is characterized in that, among the said step S3, whether decision operation overtime may further comprise the steps:
S3.0.1: resolve said key key, obtain running time opTime;
S3.0.2: deduct said running time opTime with current time curTime, judge that whether its difference is greater than overtime standard value timeout; If greater than, operation overtime then; Otherwise, operate not overtime.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011100392697A CN102110146B (en) | 2011-02-16 | 2011-02-16 | Key-value storage-based distributed file system metadata management method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011100392697A CN102110146B (en) | 2011-02-16 | 2011-02-16 | Key-value storage-based distributed file system metadata management method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102110146A CN102110146A (en) | 2011-06-29 |
| CN102110146B true CN102110146B (en) | 2012-11-14 |
Family
ID=44174307
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011100392697A Active CN102110146B (en) | 2011-02-16 | 2011-02-16 | Key-value storage-based distributed file system metadata management method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102110146B (en) |
Families Citing this family (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103544156B (en) * | 2012-07-10 | 2019-04-09 | 腾讯科技（深圳）有限公司 | File memory method and device |
| CN103049574B (en) * | 2013-01-04 | 2015-12-09 | 中国科学院高能物理研究所 | Realize key assignments file system and the method for file dynamic copies |
| CN103914483B (en) * | 2013-01-07 | 2018-09-25 | 深圳市腾讯计算机系统有限公司 | File memory method, device and file reading, device |
| CN104572711A (en) * | 2013-10-18 | 2015-04-29 | 腾讯科技（深圳）有限公司 | Distribution document-type data storage and extraction method and device |
| CN103744882B (en) * | 2013-12-20 | 2018-05-25 | 浪潮(北京)电子信息产业有限公司 | A kind of browse film segment table based on key-value pair shows method and device |
| CN104850565B (en) * | 2014-02-19 | 2018-05-22 | 北京京东尚科信息技术有限公司 | A kind of metadata management method based on K-V storage systems |
| CN104883381B (en) * | 2014-05-27 | 2018-09-04 | 陈杰 | The data access method and system of distributed storage |
| CN104156278B (en) * | 2014-08-01 | 2017-06-27 | 江苏大学 | A kind of FileVersion control system and its method |
| CN104572920A (en) * | 2014-12-27 | 2015-04-29 | 北京奇虎科技有限公司 | Data arrangement method and data arrangement device |
| CN105988899B (en) * | 2015-03-05 | 2019-04-02 | 广州市动景计算机科技有限公司 | The method and apparatus for realizing data buffer storage |
| CN105302889B (en) * | 2015-10-16 | 2019-06-04 | 北京奇虎科技有限公司 | The conversion method and device of data store organisation |
| CN105719454B (en) * | 2016-01-28 | 2019-03-15 | 海穗信息技术(上海)有限公司 | A kind of expansible ZigBee data transmission set and method |
| CN106202367B (en) * | 2016-07-07 | 2019-03-05 | 腾讯科技（深圳）有限公司 | A kind of processing method and processing device of object information |
| CN107870940B (en) * | 2016-09-28 | 2021-06-18 | 杭州海康威视数字技术股份有限公司 | File storage method and device |
| CN106843912B (en) * | 2017-03-22 | 2021-01-01 | 武汉斗鱼网络科技有限公司 | Page information acquisition method and device |
| CN106951557B (en) * | 2017-03-31 | 2020-02-07 | 北京希嘉创智教育科技有限公司 | Log association method and device and computer system applying log association method and device |
| WO2019024060A1 (en) * | 2017-08-03 | 2019-02-07 | 华为技术有限公司 | Data storage method and device, and storage medium |
| US11016932B2 (en) * | 2017-09-21 | 2021-05-25 | Alibaba Group Holding Limited | Systems, methods, and apparatuses for simplifying filesystem operations utilizing a key-value storage system |
| CN107741833B (en) * | 2017-10-30 | 2021-02-05 | 广东乐心医疗电子股份有限公司 | Storage method and device of nestable key-value |
| CN109960212B (en) * | 2017-12-25 | 2020-07-31 | 北京京东乾石科技有限公司 | Task sending method and device |
| CN109117093B (en) * | 2018-08-20 | 2021-10-01 | 赛凡信息科技（厦门）有限公司 | Method for ensuring consistency of data, flow and capacity in distributed object storage |
| CN110874353A (en) * | 2018-09-04 | 2020-03-10 | 阿里巴巴集团控股有限公司 | Data model creating method and device and data table accessing method and device |
| CN109379254B (en) * | 2018-11-07 | 2021-02-09 | 视联动力信息技术股份有限公司 | Network connection detection method and system based on video conference |
| CN111694808B (en) * | 2019-03-15 | 2023-12-29 | 阿里巴巴集团控股有限公司 | Data processing method and device and computing equipment |
| CN111143366B (en) * | 2019-12-27 | 2020-12-01 | 焦点科技股份有限公司 | High-efficiency storage method for massive large object data |
| CN114722021A (en) * | 2020-09-30 | 2022-07-08 | 华为技术有限公司 | Metadata management method and electronic equipment |
| CN112597070B (en) * | 2020-11-16 | 2022-10-21 | 新华三大数据技术有限公司 | Object recovery method and device |
| CN113032356B (en) * | 2021-03-31 | 2023-05-26 | 中电科航空电子有限公司 | Cabin distributed file storage system and implementation method |
| CN114356241B (en) * | 2021-12-28 | 2023-11-03 | 天翼云科技有限公司 | Small object data storage method, small object data storage device, electronic equipment and readable medium |
| CN114415971B (en) * | 2022-03-25 | 2022-09-23 | 阿里云计算有限公司 | Data processing method and device |
| CN115904885B (en) * | 2023-03-10 | 2023-05-09 | 浪潮电子信息产业股份有限公司 | Log management method, device, electronic equipment and computer readable storage medium |
| CN117539841B (en) * | 2024-01-09 | 2024-04-23 | 深圳市合纵天下信息科技有限公司 | Metadata management system of distributed file system and operation method thereof |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101196929A (en) * | 2007-12-29 | 2008-06-11 | 中国科学院计算技术研究所 | Metadata management method for splitting name space |
| CN101567003A (en) * | 2009-05-27 | 2009-10-28 | 清华大学 | Method for managing and allocating resource in parallel file system |
| CN101697168A (en) * | 2009-10-22 | 2010-04-21 | 中国科学技术大学 | Method and system for dynamically managing metadata of distributed file system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7647463B2 (en) * | 2006-11-16 | 2010-01-12 | International Business Machines Corporation | Apparatus, system and method for detection of mismatches in continuous remote copy using metadata |
| CN101697526A (en) * | 2009-10-10 | 2010-04-21 | 中国科学技术大学 | Method and system for load balancing of metadata management in distributed file system |
-
2011
- 2011-02-16 CN CN2011100392697A patent/CN102110146B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101196929A (en) * | 2007-12-29 | 2008-06-11 | 中国科学院计算技术研究所 | Metadata management method for splitting name space |
| CN101567003A (en) * | 2009-05-27 | 2009-10-28 | 清华大学 | Method for managing and allocating resource in parallel file system |
| CN101697168A (en) * | 2009-10-22 | 2010-04-21 | 中国科学技术大学 | Method and system for dynamically managing metadata of distributed file system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102110146A (en) | 2011-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102110146B (en) | Key-value storage-based distributed file system metadata management method | |
| CN104933133B (en) | Meta-data snap in distributed file system stores and accesses method | |
| CN102169507B (en) | Implementation method of distributed real-time search engine | |
| US10509701B2 (en) | Performing data backups using snapshots | |
| US7769792B1 (en) | Low overhead thread synchronization system and method for garbage collecting stale data in a document repository without interrupting concurrent querying | |
| CN101542483B (en) | Apparatus and method for resource reclamation in data storage systems | |
| US20120179689A1 (en) | Directory tree search | |
| US7634517B1 (en) | System and method for dynamically updating a document repository without interrupting concurrent querying | |
| CN105677826A (en) | Resource management method for massive unstructured data | |
| CN103473296A (en) | Recycle bin mechanism and recycle bin system applicable to cloud computing | |
| CN102567427A (en) | Method and device for processing object data | |
| CN109726177A (en) | A kind of mass file subregion indexing means based on HBase | |
| CN100424699C (en) | Attribute extensible object file system | |
| CN102012933A (en) | Distributed file system and method for storing data and providing services by utilizing same | |
| WO2018141304A1 (en) | Flash file system and data management method thereof | |
| US20140244606A1 (en) | Method, apparatus and system for storing, reading the directory index | |
| CN102289526B (en) | HFS+file system data recovery method and device | |
| CN102223410A (en) | Searching method of intelligent information terminal and terminal | |
| CN103268291B (en) | The method of persistence index metadata is postponed in flash-memory storage system | |
| CN105045850A (en) | Method for recovering junk data in cloud storage log file system | |
| CN103473298A (en) | Data archiving method and device and storage system | |
| CN104239438A (en) | File information storage method and file information read-write method based on separate storage | |
| CN102541982B (en) | Method for organizing and accessing metadata file log | |
| US7617226B1 (en) | Document treadmilling system and method for updating documents in a document repository and recovering storage space from invalidated documents | |
| CN102567415A (en) | Control method and device of database |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |