CN107341267A - A kind of distributed file system access method and platform - Google Patents
A kind of distributed file system access method and platform Download PDFInfo
- Publication number
- CN107341267A CN107341267A CN201710607025.1A CN201710607025A CN107341267A CN 107341267 A CN107341267 A CN 107341267A CN 201710607025 A CN201710607025 A CN 201710607025A CN 107341267 A CN107341267 A CN 107341267A
- Authority
- CN
- China
- Prior art keywords
- small documents
- reading
- file
- behavioral data
- threshold value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application discloses a kind of distributed file system access method and platform, this method includes：Obtain the write request of client；Judge whether the file size of file to be written corresponding to said write request is less than default file size threshold value；If it is, the file type for judging the file to be written is small documents, and by the file cache to be written to distributed redis clusters.The present invention writes file as cache pool, client using distributed redis clusters from the storage cluster of rear end disk, and file is read from redis clusters.From the foregoing, it will be observed that be directed to write once read many, file is smaller and in the case of not deleting, and caches small documents to redis clusters in write-in, is directly read when reading small documents from redis clusters, reduce the delay for reading file；The storage cluster of redis clusters and rear end disk is kept apart, the write-in behavior in rear end disk is not interfered with the reading behavior of redis clusters, reduces the pressure of rear end disk, improve file system performance.
Description
Technical field
The present invention relates to file access technical field, more particularly to a kind of distributed file system access method and platform.
Background technology
In distributed cloud storage field, in order to reduce file system capacity redundancy, generally use correcting and eleting codes carry out data storage,
Will a data by entangle delete algorithm write polylith disk.When a file needs to be read, it is necessary to read number from polylith disk
According to final data could be obtained, so cause to need polylith disk to simultaneously participate in when reading.When needing write once read many, largely
Read requests can take the input and output (IO) of disk, cause the bandwidth of write request to be affected, file system pressure is very
Greatly, especially when the file to be written of write request is smaller, the performance of file system seems lower.
In the prior art, delayed using fast solid-state disk (SSD) to do the high speed in rear end serial ports (SATA) hard disc data pond
Rush storage pool (cache).Client writes file from cache ponds, and file is read from SATA data pools；When cache ponds write full,
File data is moved in SATA data pools；When the reading of certain file is more frequent, this document is determined as hot spot data, and
This document is moved in cache ponds from SATA data pools, accelerates again reading off for this document.
In this way, if presently written pressure is larger, as the migration of hot spot data has influence on file system
The performance of system.
The content of the invention
In view of this, it is an object of the invention to provide a kind of distributed file system access method and platform, can subtract
The pressure of small rear end disk, the delay for reading file is reduced, lift file system performance.Its concrete scheme is as follows：
A kind of distributed file system access method, including：
Obtain the write request of client；
Judge whether the file size of file to be written corresponding to said write request is less than default file size threshold value；
If it is, the file type for judging the file to be written is small documents, and by the file cache to be written
To distributed redis clusters.
Preferably, methods described further comprises：
The reading behavior that the client is directed to small documents is recorded, obtains reading behavioral data accordingly；
Judge whether the reading behavioral data meets preparatory condition, if it is, the reading behavioral data is corresponding
Small documents cache to the distributed redis clusters.
Preferably, it is described to judge whether the reading behavioral data meets the process of preparatory condition, including：
Judge whether reading frequency is more than the first predeterminated frequency threshold value, if it is, judging that the reading behavioral data is expired
The foot preparatory condition.
Preferably, it is described to judge whether the reading behavioral data meets the process of preparatory condition, including：
Judge whether reading times are more than the first preset times threshold value, if it is, judging that the reading behavioral data is expired
The foot preparatory condition.
Preferably, it is described to judge whether the reading behavioral data meets the process of preparatory condition, including：
If the reading frequency is more than the second predeterminated frequency threshold value, it is default to judge whether reading times are more than second
Frequency threshold value, if it is, judging that the reading behavioral data meets the preparatory condition.
Preferably, it is described to judge whether the reading behavioral data meets the process of preparatory condition, including：
Judge whether reading times are more than the 3rd preset times threshold value；
If the reading times are more than the 3rd preset times threshold value, it is default to judge whether reading frequency is more than the 3rd
Frequency threshold, if it is, judging that the reading behavioral data meets the preparatory condition.
Preferably, methods described further comprises：
Determine the small documents of the client current request reading and/or ask the small documents read in history, obtain phase
The target small documents answered；
It is determined that the small documents associated with the target small documents, obtain associating small documents；
The association small documents are cached to the distributed redis clusters.
Preferably, process that is described to determine the small documents associated with the target small documents, obtaining associating small documents, bag
Include：
It is determined that the first kind small documents and/or second class small documents associated with the target small documents, obtain the pass
Join small documents；
Wherein, the first kind small documents are the small documents for having with the target small documents same keyword；
The second class small documents are the store path identical small documents with the target small documents.
The present invention also provides a kind of distributed file system access platform, including：
Acquisition module, for obtaining the write request of client；
Size judge module, it is pre- whether the file size for judging file to be written corresponding to said write request is less than
If file size threshold value；
First cache module, for judging that the file size of the file to be written is less than institute when the size judge module
Default file size threshold value is stated, by the file cache to be written to distributed redis clusters.
Preferably, the platform further comprises：
Data model analysis module, the reading behavior of small documents is directed to for recording the client, is read accordingly
Take behavioral data；
Behavior judge module, for judging whether the reading behavioral data meets preparatory condition；
Second cache module, for judging that the reading behavioral data meets the default bar when the behavior judge module
Part, small documents corresponding to the reading behavioral data are cached to the distributed redis clusters.
Preferably, the platform further comprises：
First determining module, for determining the small documents of the client current request reading and/or request reading in history
The small documents taken, obtain corresponding target small documents；
Second determining module, for determining the small documents associated with the target small documents, obtain associating small documents；
3rd cache module, for the association small documents to be cached to the distributed redis clusters.
Distributed file system access method and platform disclosed by the invention, using distributed redis clusters as caching
Pond, client write file from the storage cluster of rear end disk, and file is read from redis clusters.For write once read many, file compared with
It is small and in the case of not deleting, small documents are cached to redis clusters in write-in, directly from redis collection when reading small documents
Group reads, and reduces the delay for reading file；The storage cluster of redis clusters and rear end disk is kept apart, in redis clusters
Reading behavior does not interfere with the write-in behavior in rear end disk, reduces the pressure of rear end disk, improves file system
Energy.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
The embodiment of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis
The accompanying drawing of offer obtains other accompanying drawings.
Fig. 1 is a kind of flow chart of distributed file system access method disclosed in the embodiment of the present invention；
Fig. 2 is the flow chart of another distributed file system access method disclosed in the embodiment of the present invention；
Fig. 3 is the flow chart of another distributed file system access method disclosed in the embodiment of the present invention；
Fig. 4 is a kind of structure chart of distributed file system access platform disclosed in the embodiment of the present invention；
Fig. 5 is the structure chart of another distributed file system access platform disclosed in the embodiment of the present invention；
Fig. 6 is the structure chart of another distributed file system access platform disclosed in the embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, rather than whole embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained every other under the premise of creative work is not made
Embodiment, belong to the scope of protection of the invention.
Shown in Figure 1 the embodiment of the invention discloses a kind of distributed file system access method, Fig. 1 is the present invention
A kind of flow chart of distributed file system access method, comprises the following steps disclosed in embodiment：
Step S11：Obtain the write request of client.
Specifically, the write-in behavior of client occurs in the storage cluster of rear end disk.
Step S12：Judge whether the file size of file to be written corresponding to said write request is big less than default file
Small threshold value.
In embodiments of the present invention, the fileinfo to be written included by write request, determines file to be written, leads to
Cross default size threshold value and judge whether file to be written is small documents.Wherein, the judgment threshold of small documents can according to file
The actual performance of system makes the embodiment of the present invention to be played more in the small documents scene of write once read many come the threshold value determined
Good effect.
Step S13：If it is, the file type for judging the file to be written is small documents, and will be described to be written
File cache is to distributed redis clusters.
Wherein, distributed redis clusters are a kind of distributed memory key-value pair data storehouse, and at the event of redis clusters
It is faster than general storage cluster to manage speed.By small documents from the storage cluster asynchronous write distribution redis clusters of rear end, Jin Erke
To read small documents from distributed redis clusters, it is possible to reduce read the delay of small documents.
A kind of distributed file system access method disclosed in the embodiment of the present invention, using distributed redis clusters conduct
Cache pool, client write file from the storage cluster of rear end disk, and file is read from redis clusters.For write once read many, text
Part is smaller and in the case of not deleting, and caches small documents to redis clusters in write-in, when reading small documents directly from
Redis clusters are read, and reduce the delay for reading file；The storage cluster of redis clusters and rear end disk is kept apart,
The reading behavior of redis clusters does not interfere with the write-in behavior in rear end disk, reduces the pressure of rear end disk, improves
File system performance.
In order to further lift the performance of file system, the embodiment of the present invention discloses another distributed file system and accessed
Method.Shown in Figure 2, Fig. 2 is the flow chart of another distributed file system access method disclosed in the embodiment of the present invention,
Comprise the following steps：
Step S21：The reading behavior that the client is directed to small documents is recorded, obtains reading behavioral data accordingly.
In the embodiment of the present invention, reading behavior can include the behavior read in history, i.e., read behavioral data accordingly
For the reading behavioral data of accumulation, for example, being recorded in past one week, client have read n times to certain file, if with day
For unit of account, then the reading frequency of this document is N/7.
Step S22：Judge whether the reading behavioral data meets preparatory condition, if it is, by the reading behavior
Small documents corresponding to data are cached to the distributed redis clusters.
Specifically, data of the client to certain file access situation can be can show that to be any by reading behavioral data.It is logical
In the case of often, it is reading times and/or reading frequency to read behavioral data.
Now, it is above-mentioned to judge whether the reading behavioral data meets the process of preparatory condition, four kinds of schemes can be passed through
To realize, including first judges that scheme, second judge that scheme, the 3rd judge scheme and the 4th judge scheme.Wherein：
First judgement scheme includes：Judge whether reading frequency is more than the first predeterminated frequency threshold value, if it is, judging institute
State reading behavioral data and meet the preparatory condition.
Second judgement scheme includes：Judge whether reading times are more than the first preset times threshold value, if it is, judging institute
State reading behavioral data and meet the preparatory condition.
3rd judgement scheme includes：
Judge whether reading frequency is more than the second predeterminated frequency threshold value；
If the reading frequency is more than the second predeterminated frequency threshold value, it is default to judge whether reading times are more than second
Frequency threshold value, if it is, judging that the reading behavioral data meets the preparatory condition.
4th judgement scheme includes：
Judge whether reading times are more than the 3rd preset times threshold value；
If the reading times are more than the 3rd preset times threshold value, it is default to judge whether reading frequency is more than the 3rd
Frequency threshold, if it is, judging that the reading behavioral data meets the preparatory condition.
Predetermined threshold value herein is the threshold value for the most suitable current file system that can be changed according to actual conditions.When certain text
When the reading times and/or reading frequency of part are more than corresponding threshold value, it is believed that this document turns into what client often accessed
" hot spot data ", this document is cached into distributed redis clusters, from distribution when next client reads this document
Redis clusters are read, it is possible to reduce read delay.It should be noted that it is reading frequency and reading time when reading behavioral data
During number, the two all should be greater than corresponding threshold value, and due to now considering two kinds of data, can suitably be dropped according to actual conditions
Low threshold numerical value.
In order to further lift file system reading performance, the embodiment of the present invention discloses a kind of distributed field system again
System access method.Shown in Figure 3, Fig. 3 is another distributed file system access method disclosed in the embodiment of the present invention
Flow chart, comprise the following steps：
Step S31：Determine the small documents of the client current request reading and/or ask the small text read in history
Part, obtain corresponding target small documents.
In the embodiment of the present invention, when the source of target small documents is only the small documents that current request is read, it is only necessary to obtain
Current read requests；When the small documents that the source of target small documents includes asking to read in history, it is necessary to record client
Read requests historical data.
Step S32：It is determined that the small documents associated with the target small documents, obtain associating small documents.
Usually, it is determined that the foundation of association small documents can be to have same keyword and/or identical with target small documents
The small documents of store path.By taking same keyword as an example, when the small documents that client request is read are to include television type number
According to text document when, the association small documents that are retrieved by keyword " television set " can be comprising keyword " television set "
Television structure figure, or other small documents related to television set.And the small documents of identical store path are usually
The associated small documents that user divides according to itself custom.The embodiment of the present invention determines target small documents by relevance, now
Target small documents be next small documents that client may access, target small documents are cached into redis clusters,
Reduce the delay for the reading behavior that client may be carried out by predicting, further improve the current performance of file system.
Certainly, in actual applications, other modes can also be used to determine association small documents, to this embodiment of the present invention not
Limit.
Correspondingly, the embodiment of the present invention also provides a kind of distributed file system access platform, and shown in Figure 5, Fig. 4 is
A kind of structure chart of distributed file system access platform, the platform include disclosed in the embodiment of the present invention：
Acquisition module 41, for obtaining the write request of client；
Size judge module 42, for judging whether the file size of file to be written corresponding to said write request is less than
Default file size threshold value；
First cache module 43, for judging that the file size of the file to be written is less than when the size judge module
The default file size threshold value, by the file cache to be written to distributed redis clusters.
Shown in Figure 5, Fig. 5 is the structure of another distributed file system access platform disclosed in the embodiment of the present invention
Figure, the platform include：
Data model analysis module 51, the reading behavior of small documents is directed to for recording the client, obtained corresponding
Read behavioral data；
Behavior judge module 52, for judging whether the reading behavioral data meets preparatory condition；
Second cache module 53, for judging that the reading behavioral data meets described preset when the behavior judge module
Condition, small documents corresponding to the reading behavioral data are cached to the distributed redis clusters.
Shown in Figure 6, Fig. 6 is the structure of another distributed file system access platform disclosed in the embodiment of the present invention
Figure, the platform include：
First determining module 61, for determining the small documents of the client current request reading and/or asking in history
The small documents of reading, obtain corresponding target small documents；
Second determining module 62, for determining the small documents associated with the target small documents, obtain associating small documents；
3rd determining module 63, for the association small documents to be cached to the distributed redis clusters.
It may be referred on the more specifical course of work of modules in above-mentioned distributed file system access platform
Corresponding contents disclosed in previous embodiment, are no longer repeated herein.
Distributed file system access method disclosed in the embodiment of the present invention and platform, made using distributed redis clusters
For cache pool, client writes file from the storage cluster of rear end disk, and file is read from redis clusters.For write once read many,
File is smaller and in the case of not deleting, and caches small documents to redis clusters in write-in, when reading small documents directly from
Redis clusters are read, and reduce the delay for reading file；The storage cluster of redis clusters and rear end disk is kept apart,
The reading behavior of redis clusters does not interfere with the write-in behavior in rear end disk, reduces the pressure of rear end disk, improves
File system performance.
Finally, it is to be noted that, herein, such as first and second or the like relational terms be used merely to by
One entity or operation make a distinction with another entity or operation, and not necessarily require or imply these entities or operation
Between any this actual relation or order be present.Moreover, term " comprising ", "comprising" or its any other variant meaning
Covering including for nonexcludability, so that process, method, article or equipment including a series of elements not only include that
A little key elements, but also the other element including being not expressly set out, or also include for this process, method, article or
The intrinsic key element of equipment.In the absence of more restrictions, the key element limited by sentence "including a ...", is not arranged
Except other identical element in the process including the key element, method, article or equipment being also present.
Distributed file system access method provided by the present invention and platform are described in detail above, herein
Apply specific case to be set forth the principle and embodiment of the present invention, the explanation of above example is only intended to help
Understand the method and its core concept of the present invention；Meanwhile for those of ordinary skill in the art, according to the thought of the present invention,
There will be changes in specific embodiments and applications, in summary, this specification content should not be construed as to this
The limitation of invention.
Claims (11)
- A kind of 1. distributed file system access method, it is characterised in that including：Obtain the write request of client；Judge whether the file size of file to be written corresponding to said write request is less than default file size threshold value；If it is, the file type for judging the file to be written is small documents, and the file cache to be written is extremely divided Cloth redis clusters.
- 2. according to the method for claim 1, it is characterised in that further comprise：The reading behavior that the client is directed to small documents is recorded, obtains reading behavioral data accordingly；Judge whether the reading behavioral data meets preparatory condition, if it is, will be small corresponding to the reading behavioral data File cache is to the distributed redis clusters.
- 3. according to the method for claim 2, it is characterised in that described to judge whether the reading behavioral data meets to preset The process of condition, including：Judge whether reading frequency is more than the first predeterminated frequency threshold value, if it is, judging that the reading behavioral data meets institute State preparatory condition.
- 4. according to the method for claim 2, it is characterised in that described to judge whether the reading behavioral data meets to preset The process of condition, including：Judge whether reading times are more than the first preset times threshold value, if it is, judging that the reading behavioral data meets institute State preparatory condition.
- 5. according to the method for claim 2, it is characterised in that described to judge whether the reading behavioral data meets to preset The process of condition, including：Judge whether reading frequency is more than the second predeterminated frequency threshold value；If the reading frequency is more than the second predeterminated frequency threshold value, judge whether reading times are more than the second preset times Threshold value, if it is, judging that the reading behavioral data meets the preparatory condition.
- 6. according to the method for claim 2, it is characterised in that described to judge whether the reading behavioral data meets to preset The process of condition, including：Judge whether reading times are more than the 3rd preset times threshold value；If the reading times are more than the 3rd preset times threshold value, judge whether reading frequency is more than the 3rd predeterminated frequency Threshold value, if it is, judging that the reading behavioral data meets the preparatory condition.
- 7. method according to any one of claim 1 to 6, it is characterised in that further comprise：Determine the small documents of the client current request reading and/or ask the small documents read in history, obtain corresponding Target small documents；It is determined that the small documents associated with the target small documents, obtain associating small documents；The association small documents are cached to the distributed redis clusters.
- 8. according to the method for claim 7, it is characterised in that described to determine the small text associated with the target small documents Part, obtain associating the process of small documents, including：It is determined that the first kind small documents and/or second class small documents associated with the target small documents, it is small to obtain the association File；Wherein, the first kind small documents are the small documents for having with the target small documents same keyword；The second class small documents are the store path identical small documents with the target small documents.
- A kind of 9. distributed file system access platform, it is characterised in that including：Acquisition module, for obtaining the write request of client；Size judge module, for judging whether the file size of file to be written corresponding to said write request is less than default text Part size threshold value；First cache module, the file size for judging the file to be written when the size judge module are less than described pre- If file size threshold value, by the file cache to be written to distributed redis clusters.
- 10. platform according to claim 9, it is characterised in that further comprise：Data model analysis module, the reading behavior of small documents is directed to for recording the client, obtains corresponding read and go For data；Behavior judge module, for judging whether the reading behavioral data meets preparatory condition；Second cache module, for judging that the reading behavioral data meets the preparatory condition when the behavior judge module, Small documents corresponding to the reading behavioral data are cached to the distributed redis clusters.
- 11. the platform according to claim 9 or 10, it is characterised in that further comprise：First determining module, for determining the small documents of the client current request reading and/or asking what is read in history Small documents, obtain corresponding target small documents；Second determining module, for determining the small documents associated with the target small documents, obtain associating small documents；3rd cache module, for the association small documents to be cached to the distributed redis clusters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710607025.1A CN107341267A (en) | 2017-07-24 | 2017-07-24 | A kind of distributed file system access method and platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710607025.1A CN107341267A (en) | 2017-07-24 | 2017-07-24 | A kind of distributed file system access method and platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107341267A true CN107341267A (en) | 2017-11-10 |
Family
ID=60217289
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710607025.1A Pending CN107341267A (en) | 2017-07-24 | 2017-07-24 | A kind of distributed file system access method and platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107341267A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108089825A (en) * | 2018-01-11 | 2018-05-29 | 郑州云海信息技术有限公司 | A kind of storage system based on distributed type assemblies |
| CN108268215A (en) * | 2017-12-30 | 2018-07-10 | 广东技术师范学院 | A kind of sudden access recognition methods of disk |
| CN108415962A (en) * | 2018-02-06 | 2018-08-17 | 国家电网公司 | A kind of cloud storage system |
| CN109325001A (en) * | 2018-08-20 | 2019-02-12 | 郑州云海信息技术有限公司 | The method, device and equipment of small documents is deleted based on meta data server |
| CN109492425A (en) * | 2018-09-30 | 2019-03-19 | 南京中铁信息工程有限公司 | A kind of worm technical application method on a distributed |
| CN110750498A (en) * | 2018-07-19 | 2020-02-04 | 成都华为技术有限公司 | Object access method, device and storage medium |
| CN110908965A (en) * | 2019-11-07 | 2020-03-24 | 北京浪潮数据技术有限公司 | Object storage management method, device, equipment and storage medium |
| CN111352590A (en) * | 2020-02-26 | 2020-06-30 | 上海霄云信息科技有限公司 | File storage method and equipment |
| CN112328549A (en) * | 2020-10-29 | 2021-02-05 | 无锡先进技术研究院 | Small file storage method, electronic device and storage medium |
| CN112486943A (en) * | 2020-12-28 | 2021-03-12 | 福州大学 | Distributed file storage system and method based on FastDFS + Redis |
| CN113485978A (en) * | 2021-06-23 | 2021-10-08 | 华泰证券股份有限公司 | Method and system for improving read-write throughput of file storage NAS and storage |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008065342A1 (en) * | 2006-12-01 | 2008-06-05 | David Irvine | Data maps |
| CN102332027A (en) * | 2011-10-15 | 2012-01-25 | 西安交通大学 | Mass non-independent small file associated storage method based on Hadoop |
| CN104331428A (en) * | 2014-10-20 | 2015-02-04 | 暨南大学 | Storage and access method of small files and large files |
| CN104715064A (en) * | 2015-03-31 | 2015-06-17 | 北京奇虎科技有限公司 | Method and server for marking keywords on webpage |
| CN105138673A (en) * | 2015-09-08 | 2015-12-09 | 浙江风向标科技有限公司 | Mass storage system and method based on Internet of Things |
| CN105426127A (en) * | 2015-11-13 | 2016-03-23 | 浪潮(北京)电子信息产业有限公司 | File storage method and apparatus for distributed cluster system |
| CN105677904A (en) * | 2016-02-04 | 2016-06-15 | 杭州数梦工场科技有限公司 | Distributed file system based small file storage method and device |
| CN105843841A (en) * | 2016-03-07 | 2016-08-10 | 青岛理工大学 | Small file storing method and system |
| CN106021335A (en) * | 2016-05-06 | 2016-10-12 | 北京奇虎科技有限公司 | A database accessing method and device |
-
2017
- 2017-07-24 CN CN201710607025.1A patent/CN107341267A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008065342A1 (en) * | 2006-12-01 | 2008-06-05 | David Irvine | Data maps |
| CN102332027A (en) * | 2011-10-15 | 2012-01-25 | 西安交通大学 | Mass non-independent small file associated storage method based on Hadoop |
| CN104331428A (en) * | 2014-10-20 | 2015-02-04 | 暨南大学 | Storage and access method of small files and large files |
| CN104715064A (en) * | 2015-03-31 | 2015-06-17 | 北京奇虎科技有限公司 | Method and server for marking keywords on webpage |
| CN105138673A (en) * | 2015-09-08 | 2015-12-09 | 浙江风向标科技有限公司 | Mass storage system and method based on Internet of Things |
| CN105426127A (en) * | 2015-11-13 | 2016-03-23 | 浪潮(北京)电子信息产业有限公司 | File storage method and apparatus for distributed cluster system |
| CN105677904A (en) * | 2016-02-04 | 2016-06-15 | 杭州数梦工场科技有限公司 | Distributed file system based small file storage method and device |
| CN105843841A (en) * | 2016-03-07 | 2016-08-10 | 青岛理工大学 | Small file storing method and system |
| CN106021335A (en) * | 2016-05-06 | 2016-10-12 | 北京奇虎科技有限公司 | A database accessing method and device |
Non-Patent Citations (2)
| Title |
|---|
| 丁建立 等: "基于NoSQL的海量航空物流小文件分布式多级存储方法", 《计算机应用研究》 * |
| 刘高军 等: "基于Redis的海量小文件分布式存储方法研究", 《计算机工程与科学》 * |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108268215A (en) * | 2017-12-30 | 2018-07-10 | 广东技术师范学院 | A kind of sudden access recognition methods of disk |
| CN108089825B (en) * | 2018-01-11 | 2020-07-07 | 郑州云海信息技术有限公司 | Storage system based on distributed cluster |
| CN108089825A (en) * | 2018-01-11 | 2018-05-29 | 郑州云海信息技术有限公司 | A kind of storage system based on distributed type assemblies |
| CN108415962A (en) * | 2018-02-06 | 2018-08-17 | 国家电网公司 | A kind of cloud storage system |
| CN110750498A (en) * | 2018-07-19 | 2020-02-04 | 成都华为技术有限公司 | Object access method, device and storage medium |
| CN110750498B (en) * | 2018-07-19 | 2023-01-06 | 成都华为技术有限公司 | Object access method, device and storage medium |
| CN109325001A (en) * | 2018-08-20 | 2019-02-12 | 郑州云海信息技术有限公司 | The method, device and equipment of small documents is deleted based on meta data server |
| CN109325001B (en) * | 2018-08-20 | 2021-06-29 | 郑州云海信息技术有限公司 | Method, device and equipment for deleting small files based on metadata server |
| CN109492425A (en) * | 2018-09-30 | 2019-03-19 | 南京中铁信息工程有限公司 | A kind of worm technical application method on a distributed |
| CN109492425B (en) * | 2018-09-30 | 2021-12-28 | 南京中铁信息工程有限公司 | Method for applying work write-once read-many technology on distributed file system |
| CN110908965A (en) * | 2019-11-07 | 2020-03-24 | 北京浪潮数据技术有限公司 | Object storage management method, device, equipment and storage medium |
| CN111352590A (en) * | 2020-02-26 | 2020-06-30 | 上海霄云信息科技有限公司 | File storage method and equipment |
| CN111352590B (en) * | 2020-02-26 | 2023-06-09 | 上海霄云信息科技有限公司 | File storage method and device |
| CN112328549A (en) * | 2020-10-29 | 2021-02-05 | 无锡先进技术研究院 | Small file storage method, electronic device and storage medium |
| CN112486943A (en) * | 2020-12-28 | 2021-03-12 | 福州大学 | Distributed file storage system and method based on FastDFS + Redis |
| CN113485978A (en) * | 2021-06-23 | 2021-10-08 | 华泰证券股份有限公司 | Method and system for improving read-write throughput of file storage NAS and storage |
| CN113485978B (en) * | 2021-06-23 | 2023-07-21 | 华泰证券股份有限公司 | Method, system and memory for improving read-write throughput capacity of file storage NAS |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107341267A (en) | A kind of distributed file system access method and platform | |
| CN102156738B (en) | Method for processing data blocks, and data block storage equipment and system | |
| US8996799B2 (en) | Content storage system with modified cache write policies | |
| US9141554B1 (en) | Methods and apparatus for data processing using data compression, linked lists and de-duplication techniques | |
| US20130080679A1 (en) | System and method for optimizing thermal management for a storage controller cache | |
| CN104025059B (en) | For the method and system that the space of data storage memory is regained | |
| CN101604226B (en) | Method for building dynamic buffer pool to improve performance of storage system based on virtual RAID | |
| US10289566B1 (en) | Handling data that has become inactive within stream aware data storage equipment | |
| US20160188227A1 (en) | Method and apparatus for writing data into solid state disk | |
| US9182912B2 (en) | Method to allow storage cache acceleration when the slow tier is on independent controller | |
| CN107302561B (en) | A kind of hot spot data Replica placement method in cloud storage system | |
| CN105468298B (en) | A kind of key assignments storage method based on log-structured merging tree | |
| CN103942157A (en) | Method and system for data processing in computing storage environment | |
| US11042324B2 (en) | Managing a raid group that uses storage devices of different types that provide different data storage characteristics | |
| CN107633045B (en) | Method and system for counting tenant data capacity in cloud storage service | |
| CN108089825A (en) | A kind of storage system based on distributed type assemblies | |
| US20140181332A1 (en) | Monitoring Record Management Method and Device | |
| CN115617742A (en) | Data caching method, system, equipment and storage medium | |
| CN104834478A (en) | Data writing and reading method based on heterogeneous hybrid storage device | |
| CN105915595B (en) | Method for cluster storage system to access data and cluster storage system | |
| CN104375782A (en) | Read-write solution for tens of millions of small file data | |
| US11461250B2 (en) | Tuning data storage equipment based on comparing observed I/O statistics with expected I/O statistics which are defined by operating settings that control operation | |
| CN108958667A (en) | A kind of method for reading data based on distributed memory system, system and device | |
| US20200125283A1 (en) | Data deduplication using kvssd | |
| Liu et al. | A delayed container organization approach to improve restore speed for deduplication systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171110 |