CN104111804A - Distributed file system - Google Patents
Distributed file system Download PDFInfo
- Publication number
- CN104111804A CN104111804A CN201410295985.5A CN201410295985A CN104111804A CN 104111804 A CN104111804 A CN 104111804A CN 201410295985 A CN201410295985 A CN 201410295985A CN 104111804 A CN104111804 A CN 104111804A
- Authority
- CN
- China
- Prior art keywords
- large file
- file
- server
- metadata
- caching server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a distributed file system which comprises a big file storage server, a big file metadata management server and a cache server, wherein the big file storage server is used for storing split big file data blocks, and big files are files of which the size is larger than the preset size; the big file metadata management server is used for storing metadata of the big files, storing mapping information of the big file data blocks on the big file storage server, managing the namespace of the big files and processing the requesting information of a user; the cache server is used for storing small files, metadata of the small files and caching part of frequently visited big files, and the small files are the files of which the size is smaller than or equal to the preset size. According to the distributed file system, the big files and the small files are stored separately, the big files are stored on the big file storage server in a block mode, the small files are stored on the cache server and the reading and writing efficiency of the big files and the small files is improved effectively.
Description
Technical field
The present invention relates to computer memory technical field, more specifically, relate to a kind of distributed file system.
Background technology
Universal and perfect along with cloud computing, individual or business data are stored in high in the clouds by increasing user, these data not only comprise that large file also comprises small documents, these class data have that data volume is large, reading frequency higher than write frequency, need to carry out the features such as quick-searching.
At present, the file system that cloud service provider is used is mainly divided into two classes: network file system(NFS) (Network File System, NFS) and distributed file system (Hadoop Distributed File System, HDFS).Network file system(NFS) refers to that cloud service provider carries out virtual partition on server, the disk space of dividing one carries out file storage to user, and the each reading and writing of files of user all needs first to login remote dummy server, on virtual disk, carry out file read-write, the defect of this type systematic is that all customer data is all kept on same server, if server failure produces significant impact to user's normal running.
Distributed file system refers to utilize multiple servers to carry out cluster jointly to store service provider the file system of data, user need to send request in the time of reading and writing of files, background server is processed user's request and request results is returned to user, current most popular distributed file system is HDFS, but this system has two major defects: cannot efficient storage a large amount of small documents and only have single name node to carry out global administration.In order to improve these defects researcher afterwards, multiple file system is proposed, but but there is defect separately, as for mass small documents, storage has proposed TFS(Taobao File System), it is merged into a large file by a large amount of small documents and is stored on data server, the method is significantly not progressive compared with HDFS, only small documents is integrated into a large file piece and is stored on data server, and increased a name server for subsequent use.And name server for subsequent use is not participated in the user's request of processing directly, only have name server for subsequent use after name server is delayed machine just to replace name server to process user's request.The method defect is that name server is mainly responsible for processing user request, and storage space fixes, and increasing when data volume, its performance will become the bottleneck of restricted T FS development.And in the time that name server catastrophic failure causes loss of data, name server for subsequent use needs one side and name server synchrodata, an edge responses user request, now the load of name server for subsequent use is excessive.In MapR file system, file data blocks and metadata are kept on node simultaneously, overcome the bottleneck of single name server, but large file and small documents have been stored together simultaneously, waste storage resources and be not easy to management.
There is the difficult problem that cannot effectively store small documents and solve single management node in current distributed file system.User's file data is various, and size is different, and the file system storage efficiency of cloud service end is most important, also directly affects failure response and the resume speed of file system.Distributed file system reasonable in design, the fault that recovery file storing process occurs rapidly, has extremely important meaning and actual application value.
Summary of the invention
The present invention is at least one defect (deficiency) overcoming described in above-mentioned prior art, and a kind of distributed file system that can effectively store small documents is provided.
For solving the problems of the technologies described above, technical scheme of the present invention is as follows:
A kind of distributed file system, comprising:
Large document storage server is for storing the large file piece after fractionation, and wherein large file is to be greater than default big or small file;
Large file metadata management server, for store large file metadata, store large file piece on large document storage server map information, manage the NameSpace of large file and process user's solicited message;
Caching server, for storing the gentle large large file of nonresident portion visit capacity of metadata of small documents, small documents, wherein, small documents is to be less than or equal to default big or small file.
In such scheme, described large document storage server comprises some, and large file metadata management server comprises at least three, and caching server comprises at least three.
In such scheme, between at least three large file metadata management servers, adopt adaptive mode to store the map information of large file piece on large file metadata and large document storage server and bear the Processing tasks of user's request;
Between at least three caching servers, adopt self-adaptation, dynamically adjustment mode to store data and process user's request.
In such scheme, on any large file metadata and large document storage server, the map information of large file piece is stored at least 2 large file metadata management servers.
In such scheme, on caching server, be provided with meta-data preservation district for storing the metadata of the large file of storing on small documents metadata and caching server, for storing the small documents conservation zone of small documents and the large file cache district for the large large file of buffer memory part visit capacity.
In such scheme, be provided with counter in caching server, for realizing large file access classification mechanism, specific implementation process is: in the time that user reads and writes certain large file by this caching server request, this large file access amount adds 1;
Visit capacity threshold values is set;
Visit capacity is called the large file of frequent access higher than the large file of visit capacity threshold values;
Caching server sorts according to visit capacity from high to low for the large file of frequent access.
In such scheme, the storage mode of the large file that in caching server, storage area visit capacity is large is:
In the time that the buffer area space of caching server is enough, the metadata that caching server directly adds new large file in large file cache district and add new large file in meta-data preservation district;
In the time of the large file cache district of caching server insufficient space, if caching server need to add the often large file of access of a new user, large the frequent access that in large file cache district, visit capacity is minimum file is deleted until space is enough, then new large file is added to large file cache district.
In such scheme, described caching server is preserved small documents metadata in permanent mode, with daily record form permanent storage small documents, preserves the often large file metadata of access with update mode.
In such scheme, when after a large file metadata management server fault wherein, system guides user to ask other large file metadata management servers to be processed at once, until the large file metadata management server of fault recovers normal;
When the large file metadata management server recovering after fault is for empty, other large file metadata management servers and the synchronous fault of this large file metadata management server front large file metadata and the large file piece map information identical with it.
In such scheme, when after a caching server fault wherein, system guides user to ask other caching server processing at once, until the caching server of fault recovers normal;
If the caching server recovering after fault is sky, small documents and the small documents metadata identical with this caching server before other caching servers and the synchronous fault of this caching server.
Compared with prior art, the beneficial effect of technical solution of the present invention is:
(1) distributed file system of the present invention is by large file and separately storage of small documents, and large file carries out piecemeal and is stored on large document storage server, and small documents is stored on caching server.In the time that user need to read and write small documents, then direct access cache server makes respective operations, and read-write efficiency visits again the mode of data storage server far above traditional first accesses meta-data management server like this.And if user need to read and write large file and first accesses large file metadata management server, obtaining large document storage server corresponding to access after positional information.This system can effectively be preserved large file and small documents, and has improved the read-write efficiency of file.
(2) system of the present invention adopts at least 3 caching servers and at least 3 large file metadata management servers, wherein interconnected between the each server of same level, can effectively break through the bottleneck of traditional single management server, in the time that a large number of users is accessed low volume data simultaneously, system by self-adaptation, dynamically adjust mode and can carry out load balancing to multiple servers, avoid occurring that certain station server processes and storage capacity deficiency but the situation of the overweight machine of delaying of Processing tasks efficiently solves the various difficult problems that single management node brings because of self.
(3) distributed file system of the present invention is in the time that certain large file metadata management server and/or caching server break down, can guide in time user to ask other large file metadata management servers to be processed, can ensure that user is unaffected for the normal running of stored file, and the loss of data causing due to fault, system of the present invention utilizes other servers to recover the data of failed server by synchronization mechanism, can effectively improve fault recovery efficiency.
Brief description of the drawings
Fig. 1 is the schematic diagram that in a kind of distributed file system specific embodiment of the present invention, large document storage server is connected with large file metadata management server.
Fig. 2 is the schematic diagram that in a kind of distributed file system specific embodiment of the present invention, large file metadata management server is connected with caching server.
Fig. 3 is caching server internal zone dividing schematic diagram in a kind of distributed file system specific embodiment of the present invention.
Embodiment
Accompanying drawing, only for exemplary illustration, can not be interpreted as the restriction to this patent;
For better explanation the present embodiment, some parts of accompanying drawing have omission, zoom in or out, and do not represent the size of actual product;
To those skilled in the art, in accompanying drawing some known features and explanation thereof may to omit be understandable.
In description of the invention, it will be appreciated that, term " first ", " second " be only for describing object, and can not be interpreted as the quantity of instruction or hint relative importance or implicit indicated technical characterictic.Thus, one or more these features can be expressed or impliedly be comprised to the feature of " first " of restriction, " second ".In description of the invention, except as otherwise noted, the implication of " multiple " is two or more.
In description of the invention, it should be noted that, unless otherwise clearly defined and limited, term " installation ", " connection " should be interpreted broadly, and for example, can be to be fixedly connected with, and can be also to removably connect, or connect integratedly; Can be mechanical connection, can be also electrical connection; Can be to be directly connected, can be also indirectly to connect by intermediary, can say the connection of two element internals.For the ordinary skill in the art, can concrete condition understand above-mentioned term at concrete meaning of the present invention.
Below in conjunction with drawings and Examples, technical scheme of the present invention is described further.
Embodiment 1
As illustrated in fig. 1 and 2, be the Organization Chart of a kind of distributed file system specific embodiment of the present invention.Referring to Fig. 1 and 2, a kind of distributed file system of this specific embodiment specifically comprises large document storage server, large file metadata management server, caching server, large document storage server, large file metadata management server, caching server three link together in turn, wherein caching server is used for connecting user, to receive user's request, processing user asks and returns to request results to user, in this process, cannot process user when caching server asks conventionally user can be asked issue large file metadata management server and asks to process, the result that request is processed also will return to user by caching server, now caching server does not do any processing to the result of user's request and request processing, only be used as the retransmission unit of data.
Large document storage server is for storing the large file piece after fractionation, and wherein large file is to be greater than default big or small file; In preset procedures, conventionally specify a threshold values by keeper, from actual angle and with reference to alternative document system, this threshold values is generally 1M, is large file in the time that file size exceedes this threshold values, when less than or equal to this threshold values, is small documents; Wherein, the size of large file piece is normally fixed, and can obtain by keeper is default.After large document storage server save data, conventionally also the metadata of preserved large file piece is sent to large file metadata management server;
Large file metadata management server, for the mapping relations of preserving, managing the large file piece of preserving on large document storage server, the metadata of preserving large file piece for preserving large document storage server, for being responsible for processing user's request and the NameSpace for management document;
Caching server, for storing the gentle large large file of nonresident portion visit capacity of metadata of small documents, small documents, wherein, small documents is to be less than or equal to default big or small file.
Based on the scheme of this specific embodiment, distributed file system of the present invention is carried out piecemeal by large file and is stored on large document storage server, small documents is stored on caching server, access small documents by caching server, access large file by large document storage server and large file metadata management server, large file and small documents are implemented separately to preserve, can effectively preserve small documents on the one hand.Distinguish on the other hand and the method that in prior art, large file is preserved together with small documents, effectively saved storage resources, and greatly facilitated the management of file.
In specific implementation process, in order to solve the problem of single management node, this specific embodiment is provided with some large document storage servers, at least three large file metadata management servers and at least three caching servers in this distributed file system, all large file metadata management servers are mutually interconnected, mutually interconnected between all caching servers.In Organization Chart as illustrated in fig. 1 and 2, distributed file system is provided with 12 large document storage servers, 3 large file metadata management servers and 3 caching servers.Server performance of the same type is identical under normal circumstances, and dissimilar server performance can differentiation.
Wherein, between at least three large file metadata management servers, adopt self-adaptation, dynamically adjustment mode to store the map information of large file piece on large file metadata and large document storage server and bear the Processing tasks of user's request.Particularly, carry out data Storage and Processing user while asking, at metadata and the map information thereof of all large file metadata management server performances and the relatively strong more much file data blocks of large file metadata management server stores of storage capacity, and bear more user and ask Processing tasks, performance is stored metadata and the map information thereof of relative less large file piece with the relatively weak large file metadata management server of storage capacity, and bears relatively less user and ask Processing tasks.
Wherein, between at least three caching servers, adopt self-adaptation, dynamically adjustment mode to store data and process user's request.Particularly, carry out data Storage and Processing user while asking, in all caching servers, store more data for performance and the relatively strong caching server of storage capacity, and bear more user and ask Processing tasks, store relatively less data and bear relative less task for performance weak caching server relative to storage capacity.
Based on such scheme, to the storage of large file, small documents, the mode that system by self-adaptation, is dynamically adjusted is carried out the load balancing of server, make data storage and user between the relatively weak server of server that performance and storage capacity are relatively high and performance and storage capacity ask processing to reach a relatively reasonable distribution, can effectively break through the bottleneck of traditional single management server, avoid occurring that certain station server processes and storage capacity deficiency but the situation of the overweight machine of delaying of Processing tasks because of self.
In specific implementation process, in order to ensure the normal running of user in server failure situation, in such scheme, originally be embodied in the mode that adopts redundancy backup between large file metadata management server, on any large file metadata and large document storage server, the map information of large file piece is stored at least 2 large file metadata management servers.In the time of a large file metadata management server fault wherein, can also be from other large file metadata management servers obtaining information.
In specific implementation process, as shown in Figure 3, meta-data preservation district, small documents conservation zone and large file cache district are set on caching server, the metadata that meta-data preservation district is responsible for preserving the metadata of the inner small documents of depositing of this caching server and is buffered in the large file on caching server, small documents metadata is persistence, unless small documents is deleted or the system failure by user, and large file metadata is taked more new-type deletion; Small documents conservation zone, with the form storage small documents of daily record, this partial data is persistence, unless deleted or the system failure by user; Large file cache district, the large large file of storage user's visit capacity, this partial data is more new-type deletion, not persistence;
In concrete implementation process, counter is set on caching server, for realizing slow large file access classification mechanism, its specific implementation process is: in the time that user accesses certain large file by this caching server, this large file access amount adds 1; Set in advance visit capacity threshold values for large file; Visit capacity is called the large file of frequent access higher than the large file of visit capacity threshold values; Caching server sorts according to visit capacity from high to low for the large file of frequent access.
One preferred embodiment in, the storage mode of the large file that in caching server, storage area visit capacity is large is:
In the time that the buffer area space of caching server is enough, the metadata that caching server directly adds new large file in large file cache district and add new large file in meta-data preservation district;
In the time of the large file cache district of caching server insufficient space, if caching server need to add the often large file of access of a new user, large the frequent access that in large file cache district, visit capacity is minimum file is deleted until space is enough, then new large file is added to large file cache district.
In such scheme, all users of user representative to relevant in distributed file system of the present invention, are often referred to the user who there is no buffer memory or very little buffer zone, as wearable smart machines that makes such as intelligent watchs.
In specific implementation process, in distributed file system of the present invention, recovery mechanism is proposed, comprise caching server fault recovery and large two kinds of situations of file metadata management server fault recovery, particularly:
When after a large file metadata management server fault wherein, system guides user to ask other large file metadata management servers to be processed at once, until the large file metadata management server of fault recovers normal;
When the large file metadata management server recovering after fault is for empty, other large file metadata management servers and the synchronous fault of this large file metadata management server front large file metadata and the large file piece map information identical with it.
When after a caching server fault wherein, system guides user to ask other caching server processing at once, until the caching server of fault recovers normal;
If the caching server recovering after fault is sky, small documents and the small documents metadata identical with this caching server before other caching servers and the synchronous fault of this caching server.
Describe distributed file system of the present invention in detail below in conjunction with an instantiation.
As shown in Figure 1, distributed file system comprises 12 large document storage server 1-12, 33 caching server 1-3 and 3 large file metadata management server 1-3, large file metadata management server 1 is preserved large document storage server 1, 2, 3, 4, 5, 6, 9, the metadata of stored information on 10, large file metadata management server 2 is preserved large document storage server 1, 2, 5, 6, 7, 8, 11, the metadata of stored information on 12, large file metadata management server 3 is preserved large document storage server 3, 4, 7, 8, 9, 10, 11, the metadata of stored information on 12,
As shown in Figure 2,12 users' of supposing the system storage small documents, 3 caching server storage modes for: caching server 1 is preserved small documents and the metadata thereof that user 1,2,3,4,5,6,9,10 stores; Caching server 2 is preserved small documents and the metadata thereof that user 1,2,5,6,7,8,11,12 stores; Caching server 3 is preserved small documents and the metadata thereof that user 3,4,7,8,9,10,11,12 stores.Wherein, in three caching servers, also mark off step by step certain region and carry out the often large file of access of cache user.
As shown in Figure 2, caching server failover procedure is: after caching server 1 breaks down, user's request is assigned to caching server 2 and caching server 3 is processed, after caching server 1 recovers when it is during because of cause specific obliterated data, the small documents that the synchronous relevant user 1,2,5,6 of caching server 2 stores and metadata thereof be to caching server 1, and the synchronous small documents of storing about user 3,4,9,10 of caching server 3 and metadata thereof are to caching server 1.
As shown in Figure 1, large file metadata management server failover procedure is: after large file metadata management server 1 breaks down, user's request is assigned to large file metadata management server 2 and large file metadata management server 3 is processed, after large file metadata management server 1 recovers when it is during because of cause specific obliterated data, the synchronous large document storage server 1 of large file metadata management server 2, 2, 5, 6 relevant data-mapping and metadata are given large file metadata management server 1, the synchronous large document storage server 3 of large file metadata management server 3, 4, 9, 10 relevant data-mapping and metadata are given large file metadata management server 1.
The corresponding same or analogous parts of same or analogous label;
In accompanying drawing, describe position relationship for only for exemplary illustration, can not be interpreted as the restriction to this patent;
Obviously, the above embodiment of the present invention is only for example of the present invention is clearly described, and is not the restriction to embodiments of the present invention.For those of ordinary skill in the field, can also make other changes in different forms on the basis of the above description.Here without also giving exhaustive to all embodiments.All any amendments of doing within the spirit and principles in the present invention, be equal to and replace and improvement etc., within all should being included in the protection domain of the claims in the present invention.
Claims (10)
1. a distributed file system, is characterized in that, comprising:
Large document storage server is for storing the large file piece after fractionation, and wherein large file is to be greater than default big or small file;
Large file metadata management server, for store large file metadata, store large file piece on large document storage server map information, manage the NameSpace of large file and process user's solicited message;
Caching server, for storing the gentle large large file of nonresident portion visit capacity of metadata of small documents, small documents, wherein, small documents is to be less than or equal to default big or small file.
2. distributed file system according to claim 1, is characterized in that, described large document storage server comprises some, and large file metadata management server comprises at least three, and caching server comprises at least three.
3. distributed file system according to claim 2, it is characterized in that, between at least three large file metadata management servers, adopt the mode of self-adaptation, dynamically adjustment to store the map information of large file piece on large file metadata and large document storage server and bear the Processing tasks of user's request;
Between at least three caching servers, adopt self-adaptation, dynamically adjustment mode to store data and process user's request.
4. distributed file system according to claim 1, is characterized in that, on any large file metadata and large document storage server, the map information of large file piece is stored at least 2 large file metadata management servers.
5. distributed file system according to claim 4, it is characterized in that, on caching server, be provided with meta-data preservation district for storing the metadata of the large file of storing on small documents metadata and caching server, for storing the small documents conservation zone of small documents and the large file cache district for the large large file of buffer memory part visit capacity.
6. distributed file system according to claim 5, it is characterized in that, in caching server, be provided with counter, for realizing large file access classification mechanism, specific implementation process is: in the time that user reads and writes certain large file by this caching server request, this large file access amount adds 1;
Visit capacity threshold values is set;
Visit capacity is called the large file of frequent access higher than the large file of visit capacity threshold values;
Caching server sorts according to visit capacity from high to low for the large file of frequent access.
7. distributed file system according to claim 6, is characterized in that, the storage mode of the large file that in caching server, storage area visit capacity is large is:
In the time that the buffer area space of caching server is enough, the metadata that caching server directly adds new large file in large file cache district and add new large file in meta-data preservation district;
In the time of the large file cache district of caching server insufficient space, if caching server need to add the often large file of access of a new user, large the frequent access that in large file cache district, visit capacity is minimum file is deleted until space is enough, then new large file is added to large file cache district.
8. according to the distributed file system described in claim 1 to 7 any one, it is characterized in that, described caching server is preserved small documents metadata in permanent mode, with daily record form permanent storage small documents, preserves the often large file metadata of access with update mode.
9. distributed file system according to claim 2, it is characterized in that, when after a large file metadata management server fault wherein, system guides user to ask other large file metadata management servers to be processed at once, until the large file metadata management server of fault recovers normal;
When the large file metadata management server recovering after fault is for empty, other large file metadata management servers and the synchronous fault of this large file metadata management server front large file metadata and the large file piece map information identical with it.
10. distributed file system according to claim 2, is characterized in that, when after a caching server fault wherein, system guides user to ask other caching server processing at once, until the caching server of fault recovers normal;
If the caching server recovering after fault is sky, small documents and the small documents metadata identical with this caching server before other caching servers and the synchronous fault of this caching server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410295985.5A CN104111804B (en) | 2014-06-27 | 2014-06-27 | A kind of distributed file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410295985.5A CN104111804B (en) | 2014-06-27 | 2014-06-27 | A kind of distributed file system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104111804A true CN104111804A (en) | 2014-10-22 |
| CN104111804B CN104111804B (en) | 2017-10-31 |
Family
ID=51708610
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410295985.5A Active CN104111804B (en) | 2014-06-27 | 2014-06-27 | A kind of distributed file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104111804B (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105049178A (en) * | 2015-07-23 | 2015-11-11 | 柳州易旺科技有限公司 | Multi-user information screening method |
| CN105095511A (en) * | 2015-09-08 | 2015-11-25 | 浪潮(北京)电子信息产业有限公司 | File processing method, apparatus and system based on distributed system |
| CN105094695A (en) * | 2015-06-29 | 2015-11-25 | 浪潮(北京)电子信息产业有限公司 | Storing method and system |
| CN105141666A (en) * | 2015-07-29 | 2015-12-09 | 江苏天联信息科技发展有限公司 | Information data storing method and device |
| CN105138632A (en) * | 2015-08-20 | 2015-12-09 | 浪潮(北京)电子信息产业有限公司 | Organization and management method for file data and file management server |
| CN105511802A (en) * | 2015-11-24 | 2016-04-20 | 北京达沃时代科技有限公司 | Buffer memory writing method and apparatus and synchronizing method and apparatus for disk cache region |
| CN105516240A (en) * | 2015-11-23 | 2016-04-20 | 浪潮(北京)电子信息产业有限公司 | Dynamic optimization framework and method for read-write performance of cluster storage system |
| CN105608193A (en) * | 2015-12-23 | 2016-05-25 | 深圳市深信服电子科技有限公司 | Data management method and apparatus for distributed file system |
| CN106020713A (en) * | 2015-09-16 | 2016-10-12 | 展视网（北京）科技有限公司 | File storage method based on buffer area |
| CN106802950A (en) * | 2017-01-16 | 2017-06-06 | 郑州云海信息技术有限公司 | A kind of method of distributed file system small documents write buffer optimization |
| CN108011584A (en) * | 2016-10-28 | 2018-05-08 | 丰郅(上海)新能源科技有限公司 | Photovoltaic cell is monitored on-line and intelligent management system |
| CN108089825A (en) * | 2018-01-11 | 2018-05-29 | 郑州云海信息技术有限公司 | A kind of storage system based on distributed type assemblies |
| CN108089888A (en) * | 2016-11-21 | 2018-05-29 | 杨正 | A kind of application operation method and system based on file system |
| CN108153491A (en) * | 2017-12-22 | 2018-06-12 | 深圳市瑞驰信息技术有限公司 | A kind of storage method and framework for closing part server |
| CN109002260A (en) * | 2018-07-02 | 2018-12-14 | 深圳市茁壮网络股份有限公司 | A kind of data cached processing method and processing system |
| CN109656874A (en) * | 2018-11-28 | 2019-04-19 | 烟台蓝洋电子科技有限责任公司 | Binary file management system implementation method |
| CN112394876A (en) * | 2019-08-14 | 2021-02-23 | 深圳市特思威尔科技有限公司 | Large file storage/reading method, storage/reading device and computer equipment |
| CN114116634A (en) * | 2022-01-26 | 2022-03-01 | 苏州浪潮智能科技有限公司 | Caching method and device and readable storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101520805A (en) * | 2009-03-25 | 2009-09-02 | 中兴通讯股份有限公司 | Distributed file system and file processing method thereof |
| CN102882983A (en) * | 2012-10-22 | 2013-01-16 | 南京云创存储科技有限公司 | Rapid data memory method for improving concurrent visiting performance in cloud memory system |
| CN103078906A (en) * | 2012-12-26 | 2013-05-01 | 爱迪科特(北京)科技有限公司 | Document transparent moving method |
-
2014
- 2014-06-27 CN CN201410295985.5A patent/CN104111804B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101520805A (en) * | 2009-03-25 | 2009-09-02 | 中兴通讯股份有限公司 | Distributed file system and file processing method thereof |
| CN102882983A (en) * | 2012-10-22 | 2013-01-16 | 南京云创存储科技有限公司 | Rapid data memory method for improving concurrent visiting performance in cloud memory system |
| CN103078906A (en) * | 2012-12-26 | 2013-05-01 | 爱迪科特(北京)科技有限公司 | Document transparent moving method |
Cited By (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105094695B (en) * | 2015-06-29 | 2018-09-04 | 浪潮(北京)电子信息产业有限公司 | A kind of storage method and system |
| CN105094695A (en) * | 2015-06-29 | 2015-11-25 | 浪潮(北京)电子信息产业有限公司 | Storing method and system |
| CN105049178A (en) * | 2015-07-23 | 2015-11-11 | 柳州易旺科技有限公司 | Multi-user information screening method |
| CN105141666A (en) * | 2015-07-29 | 2015-12-09 | 江苏天联信息科技发展有限公司 | Information data storing method and device |
| CN105141666B (en) * | 2015-07-29 | 2018-12-14 | 江苏天联信息科技发展有限公司 | Information data storing method and device |
| CN105138632A (en) * | 2015-08-20 | 2015-12-09 | 浪潮(北京)电子信息产业有限公司 | Organization and management method for file data and file management server |
| CN105095511A (en) * | 2015-09-08 | 2015-11-25 | 浪潮(北京)电子信息产业有限公司 | File processing method, apparatus and system based on distributed system |
| CN106020713A (en) * | 2015-09-16 | 2016-10-12 | 展视网（北京）科技有限公司 | File storage method based on buffer area |
| CN105516240A (en) * | 2015-11-23 | 2016-04-20 | 浪潮(北京)电子信息产业有限公司 | Dynamic optimization framework and method for read-write performance of cluster storage system |
| CN105511802A (en) * | 2015-11-24 | 2016-04-20 | 北京达沃时代科技有限公司 | Buffer memory writing method and apparatus and synchronizing method and apparatus for disk cache region |
| CN105511802B (en) * | 2015-11-24 | 2018-06-05 | 北京达沃时代科技股份有限公司 | The method and apparatus of write buffer and the synchronous method and device in disk buffering area |
| CN105608193A (en) * | 2015-12-23 | 2016-05-25 | 深圳市深信服电子科技有限公司 | Data management method and apparatus for distributed file system |
| CN105608193B (en) * | 2015-12-23 | 2019-03-26 | 深信服科技股份有限公司 | The data managing method and device of distributed file system |
| CN108011584B (en) * | 2016-10-28 | 2020-06-26 | 丰郅(上海)新能源科技有限公司 | Photovoltaic cell on-line monitoring and intelligent management system |
| CN108011584A (en) * | 2016-10-28 | 2018-05-08 | 丰郅(上海)新能源科技有限公司 | Photovoltaic cell is monitored on-line and intelligent management system |
| CN108089888B (en) * | 2016-11-21 | 2019-09-13 | 杨正 | It is a kind of that operation method and system are applied based on file system |
| CN108089888A (en) * | 2016-11-21 | 2018-05-29 | 杨正 | A kind of application operation method and system based on file system |
| US11016785B2 (en) | 2016-11-21 | 2021-05-25 | Zheng Yang | Method and system for mirror image package preparation and application operation |
| CN106802950A (en) * | 2017-01-16 | 2017-06-06 | 郑州云海信息技术有限公司 | A kind of method of distributed file system small documents write buffer optimization |
| CN108153491A (en) * | 2017-12-22 | 2018-06-12 | 深圳市瑞驰信息技术有限公司 | A kind of storage method and framework for closing part server |
| CN108089825B (en) * | 2018-01-11 | 2020-07-07 | 郑州云海信息技术有限公司 | Storage system based on distributed cluster |
| CN108089825A (en) * | 2018-01-11 | 2018-05-29 | 郑州云海信息技术有限公司 | A kind of storage system based on distributed type assemblies |
| CN109002260A (en) * | 2018-07-02 | 2018-12-14 | 深圳市茁壮网络股份有限公司 | A kind of data cached processing method and processing system |
| CN109002260B (en) * | 2018-07-02 | 2021-08-13 | 深圳市茁壮网络股份有限公司 | Processing method and processing system for cache data |
| CN109656874A (en) * | 2018-11-28 | 2019-04-19 | 烟台蓝洋电子科技有限责任公司 | Binary file management system implementation method |
| CN109656874B (en) * | 2018-11-28 | 2024-03-08 | 山东蓝洋智能科技有限公司 | Method for implementing file management system in dual system |
| CN112394876A (en) * | 2019-08-14 | 2021-02-23 | 深圳市特思威尔科技有限公司 | Large file storage/reading method, storage/reading device and computer equipment |
| CN112394876B (en) * | 2019-08-14 | 2024-02-23 | 深圳市特思威尔科技有限公司 | Large file storage/reading method, storage/reading device and computer equipment |
| CN114116634A (en) * | 2022-01-26 | 2022-03-01 | 苏州浪潮智能科技有限公司 | Caching method and device and readable storage medium |
| CN114116634B (en) * | 2022-01-26 | 2022-04-22 | 苏州浪潮智能科技有限公司 | Caching method and device and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104111804B (en) | 2017-10-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104111804A (en) | Distributed file system | |
| US20200186602A1 (en) | Continuous backup of data in a distributed data store | |
| US10437721B2 (en) | Efficient garbage collection for a log-structured data store | |
| US10229011B2 (en) | Log-structured distributed storage using a single log sequence number space | |
| US20190188406A1 (en) | Dynamic quorum membership changes | |
| CN105183839A (en) | Hadoop-based storage optimizing method for small file hierachical indexing | |
| KR101771246B1 (en) | System-wide checkpoint avoidance for distributed database systems | |
| US10747746B2 (en) | Efficient read replicas | |
| KR101914019B1 (en) | Fast crash recovery for distributed database systems | |
| US9507843B1 (en) | Efficient replication of distributed storage changes for read-only nodes of a distributed database | |
| US10659225B2 (en) | Encrypting existing live unencrypted data using age-based garbage collection | |
| US9547706B2 (en) | Using colocation hints to facilitate accessing a distributed data storage system | |
| US9405643B2 (en) | Multi-level lookup architecture to facilitate failure recovery | |
| CN106021031B (en) | A kind of the deletion data reconstruction method and device of BTRFS file system | |
| CN107798130A (en) | A kind of Snapshot Method of distributed storage | |
| CN103905540A (en) | Object storage data distribution mechanism based on two-sage Hash | |
| CN110347651A (en) | Method of data synchronization, device, equipment and storage medium based on cloud storage | |
| CN103037004A (en) | Implement method and device of cloud storage system operation | |
| AU2013403132A1 (en) | Data storage method, data storage apparatus, and storage device | |
| CN103501319A (en) | Low-delay distributed storage system for small files | |
| US10152493B1 (en) | Dynamic ephemeral point-in-time snapshots for consistent reads to HDFS clients | |
| CN104735110A (en) | Metadata management method and system | |
| CN113377868A (en) | Offline storage system based on distributed KV database | |
| CN112334891A (en) | Centralized storage for search servers | |
| CN105187565A (en) | Method for utilizing network storage data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |