CN102143215A - Network-based PB level cloud storage system and processing method thereof - Google Patents

Network-based PB level cloud storage system and processing method thereof Download PDF

Info

Publication number
CN102143215A
CN102143215A CN2011100227416A CN201110022741A CN102143215A CN 102143215 A CN102143215 A CN 102143215A CN 2011100227416 A CN2011100227416 A CN 2011100227416A CN 201110022741 A CN201110022741 A CN 201110022741A CN 102143215 A CN102143215 A CN 102143215A
Authority
CN
China
Prior art keywords
storage
server
storage server
file
main control
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN2011100227416A
Other languages
Chinese (zh)
Other versions
CN102143215B (en
Inventor
陈卫卫
吴海佳
胡谷雨
董继光
刘鹏
田浪军
胥光辉
潘志松
李清
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
PLA University of Science and Technology
Original Assignee
PLA University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by PLA University of Science and Technology filed Critical PLA University of Science and Technology
Priority to CN2011100227416A priority Critical patent/CN102143215B/en
Publication of CN102143215A publication Critical patent/CN102143215A/en
Application granted granted Critical
Publication of CN102143215B publication Critical patent/CN102143215B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Abstract

The invention relates to a network-based PB level cloud storage system and a processing method thereof. The network-based PB level cloud storage system comprises a main control server, a storage server cluster and a client side which are operated through an internal exchange, wherein the main control server is used for providing the directory information and the meta data information for a cloud storage client side and monitoring the storage server cluster; the storage server cluster comprises a plurality of storage servers used for data storage; the client side is used for providing the virtual disc service for a cloud storage client machine, submitting an operation request of the cloud storage client machine to a virtual disc to the main control server and reading/writing the file data from the storage server, and a client-side module is arranged on the cloud storage client machine. The invention is easy to expanded and is convenient to manage, solves the problem of storage island, can construct a cloud storage file system provided with the dynamic load balancing capability and the online backup and automatic recovery functions and capable of being expanded on demand on a general-purpose low-performance PC (personal computer) cluster.

Description

A kind of based on network PB level cloud storage system and processing method thereof
Technical field
The invention belongs to the mass memory technical field, especially have in the mass memory that dynamic load leveling, online backup and automatic fault are switched, the cloud memory file system of capacity extension function as required, specifically a kind of based on network PB (1PB=1024 * 1024GB) level cloud storage system and processing method thereof.
Background technology
In current cybertimes, memory technology is had a revolutionary change.Cause the demand of this revolutionary variation at first to show the rapid expansion of memory capacity.In digitlization and network interconnection epoch, in multi-user's parallel environment, the widespread deployment of large-scale application system has proposed huge challenge to the performance and the function of mass storage system (MSS), but mainly show as high-performance, extensibility sharing, adaptivity, manageability, and high reliability and availability.
The mass storage system (MSS) that makes up based on special purpose memory devices comprises direct-connected storage (DAS), the storage of attached net (NSA), storage area network (SAN).These mass memories all depend on expensive special purpose memory devices, and have problems such as complex management, expansion difficulty.
The distributed memory system that makes up based on distributed file system can be deployed on the all-purpose computer.Traditional distributed file system comprises NFS, AFS, GPFS etc.NFS is present most widely used distributed file system, but its poor expandability is based on the Single-Server model, even many nfs servers are arranged in the network, between them also without any contact, independent operating separately.Along with the growth of system users scale, nfs server will become system bottleneck gradually.AFS, GPFS etc. provide the independence and the fault tolerance of memory location mostly, but these file system are all at the science calculation Design, to bandwidth and delay requirement than higher, and lack the support that server is dynamically added and withdraws from, the also not detection of back-level server inefficacy, along with the expansion of system scale, maintenance cost can sharply increase.
Along with popularizing of cloud computing thought, the cloud storage becomes mass data storage with efficient administration and the solution of cheapness.The more famous Google File System (GFS) that Google is arranged with and increase income and realize Hadoop Distributed File System (HDFS).GFS designs for handling based on the data parallel of big file and streaming visit, is not suitable for the general persistent storage of file.The chain type copy update mode of HDFS will cause systematic function seriously to descend, and HDFS does not at present also support the dynamic adding of storage server and withdraws from.Prominent domestic Taobao has also developed Taobao File System (TFS), and TFS has done special optimization at the random read-write access performance of mass small documents, makes it also be not suitable for use in general mass memory.
General cloud memory file system is worked in coordination with memory device general in a large number, cheap in the cluster, low performance by the cloud memory file system, and high performance stores service externally is provided jointly.The user need not to be concerned about the concrete deposit position of data, file layout, only needs can enjoy and the highly reliable stores service of the magnanimity of local hard drive indifference by cloud storage client virtualization software.
There are a plurality of modules in the cloud memory file system, each intermodule carries out data interaction by network, therefore the operating lag that can bring the user to experience in order to shorten or to eliminate the cloud memory file system operating lag that causes owing to network delay, needs to use caching technology.At present, representative buffer scheduling method comprises: FIFO, LRU, LFU, Hyper-G, SIZE, GD-Size, LRU-SP, LEC etc.These buffer scheduling methods only consider how to improve the hit rate of buffer memory, but do not consider to reduce the number of times of searching of the adjustment number of times of buffer memory sequence and cache object.But in the cloud memory file system,,, will cause system constantly to adjust the buffer memory sequence, thereby cause certain delay if use above buffer scheduling method owing to there is the average frequent concurrent visit of a large amount of probability distribution.
In order to improve the ability of cloud memory file system concurrent processing task, need the working load balancing technique.Early stage load-balancing algorithm has at random (Random) algorithm and poll (Round-Robin) algorithm, and they calculate simply, efficient is high, it is wider to use.But these two kinds of algorithms are not considered the actual loading situation of back-end server, and the performance difference between the server, thereby when the concurrent access number of system increases, can not guarantee the load balancing between each server.Also have based on current minimum linking number (the Least Connection First) algorithm that enlivens linking number in the server in addition, select the minimum server of the current request number of handling as transmitting object.But active request connection number can not reflect the difference on the server process load fully, can handle more request as the server that disposal ability is strong in the unit interval.Minimum linking number (the Weighted Least Connection First) algorithm of WRR (Weighted Round-Robin) algorithm and weighting had appearred again afterwards, by come the difference between the balance server for the different different weights of server configuration.But static weights information can not show dynamic load characteristic, along with system operation time increases, will cause the imbalance of cluster load Distribution.
Therefore, existing cloud memory file system exists expansion difficulty, complex management, is prone to problems such as storage isolated island, can not satisfy user demand.
Summary of the invention
The objective of the invention is at the existing expansion difficulty of present traditional mass storage system (MSS) (DAS, NAS, SAN etc.), complex management, be prone to the problem of storage isolated island, a kind of based on network PB level cloud storage system FFS (Formicary File System) and processing method thereof are proposed, can on the PC of general low performance cluster, make up and have dynamic load leveling ability, online backup and auto restore facility and the cloud storage system that can expand as required.
Technical scheme of the present invention is:
A kind of based on network PB level cloud storage system, it comprises main control server, storage server cluster and client, they carry out exchanges data by the inner exchanging machine;
Main control server is used for providing directory information and metadata information to cloud storage client, and the storage server cluster is monitored;
The storage server cluster comprises many storage servers that are used for storage, and it is provided with the AC-RU buffer memory, preserves several file handles of opening recently;
Client is used for providing the virtual disk service to the cloud storage client, the cloud storage client is submitted to main control server to the operation requests of virtual disk, and from storage server read/write file data, described client modules is deployed on the cloud storage client, it is provided with the AC-RU buffer memory, preserves the directory information of visit recently and the metadata information of file.
Client of the present invention comprises mailbox server, ftp server, Web server, database server or streaming media server for needing to use the large server of magnanimity cloud stores service.
Directory information of the present invention comprises path, date created, the directory attribute of this catalogue; Metadata information comprises IP address, port numbers and the corresponding GUID of path, establishment/modification date, file attribute, file size, place primary storage server and the back-up storage server of this document.
A kind of processing method of based on network PB level cloud storage system adopts the cloud memory file system, and it may further comprise the steps:
(a), adopt many storage servers to set up the storage server cluster, periodically send heartbeat message after each storage server starts to main control server, comprise the current state of this storage server in the heartbeat message; Be provided with one in the internal memory of main control server corresponding to the tabulation of the logical node of a plurality of storage servers, receive behind the heartbeat message of each storage server in upgrading tabulation logical node when main control server corresponding to each storage server;
(b), when under the client-access virtual disk during any catalogue, cloud storage client is initiated request to main control server, obtain subdirectory information and the file metadata information under the catalogue of asking, and subdirectory information and the file metadata information that is obtained is kept in the customer end A C-RU buffer memory;
(c), cloud storage client-requested creates the operation of file or catalogue, wherein;
I, when cloud storage client has the request of creating file, client is submitted to main control server with this request, whether the catalogue that the file place is created in the main control server inspection exists, if there is not the then non-existent answer of Returning catalogue; If exist, then distribute GUID for this document, from the online storage service device, select load lighter two primary storage server and back-up storage servers as this document, and the file of notifying these two storage servers to create GUID by name, two storage servers are all created success and are then returned the successful answer of establishment to client;
Ii, when cloud storage client has the request that creaties directory, client is submitted to main control server with this request, main control server checks whether the parent directory of newly-built catalogue exists, if do not exist, then returns the non-existent answer of parent directory; If exist, then set up the new directory node, and add in the subdirectory tabulation of parent directory node, and return the successful answer of establishment to client;
(d), cloud storage client-requested reads file, written document, deleted file, duplicates/move or Rename file operation, wherein:
When i, cloud storage client-requested is read file, at first from the AC-RU buffer memory of cloud storage client, search the metadata information of this document, if metadata information exists in the buffer memory, then the metadata information by file finds corresponding primary storage server, and the primary storage server requests is read concrete a certain section the data of this document; If metadata information does not exist in the buffer memory, then at first send metadata request, and will add the customer end A C-RU buffer memory from the metadata information that main control server obtains to main control server;
When ii, cloud storage client-requested written document, at first from the AC-RU buffer memory of cloud storage client, search the metadata information of this document, if metadata information exists in the buffer memory, then cloud storage client finds corresponding primary storage server and back-up storage server by the metadata information of file, and write concrete a certain section the data of this document to their requests, after all returning the response of writing success etc. primary storage server and back-up storage server, the success of this time write operation, otherwise think and write failure; If metadata information does not exist in the buffer memory, then at first send metadata request, and will add the customer end A C-RU buffer memory from the metadata information that main control server obtains to main control server;
Iii, during cloud storage client-requested deleted file, at first send file deletion requests to main control server, main control server finds the master file storage server and the backup file storage server of concrete preservation this document according to the metadata information of this document, and this metadata information deleted from bibliographic structure, return the successful response of deletion to cloud storage client simultaneously, main control server will be notified primary storage server and back-up storage server deletion this document of this document, two storage servers execute file deletion action after receiving the file delete instruction of main control server then;
Iv, cloud storage client-requested duplicates/and the move operation is the combination of establishment/read/write/delete file operation;
V, Rename file are to revise file metadata information under the bibliographic structure in main control server; (e), the cloud storage client-requested operation that deltrees, wherein: when cloud storage client-requested deltrees, at first whether there is the information of being deleted catalogue in the viewing client-side AC-RU buffer memory,, sends the directory delete request to main control server then from buffer memory if exist then it is deleted; If there is no, then directly send the directory delete request to main control server, its subdirectory of traversal was tabulated and listed files after main control server was received the directory delete request, recursively deleted all subdirectories and file, this directory delete the most at last under this catalogue; In the recurrence delete procedure, when certain subdirectory is a leaf node in the bibliographic structure, then directly deletion, otherwise the recurrence delete procedure takes place; When deltreeing down file, carry out according to step (d) .iii method.
In the step of the present invention (c), selecting load lighter two from the online storage service device is to adopt the system load balancing method as the primary storage server of this document and the method for back-up storage server; The load calculating method of each storage server is:
l i＝a i×(u i+d i)/n i (1)
Wherein, l iBe the load value of i platform storage server in the storage server cluster, a iThe disk free space percentage of storage server for this reason, u iThe network interface card upstream rate of storage server for this reason, d iBe its downstream rate, n iBe the maximum rate of this network interface card, u i, d iAnd n iUnit all be kbps.
System load balancing method of the present invention also comprises: main control server periodically calculates the total work load and the data distribution degree of unbalance of current storage server cluster; When total work load is lower than certain set point L Threshold, and data distribution degree of unbalance is higher than certain set point U ThresholdThe time, then the heaviest storage server of memory load is carried out the data migration, data are moved to the lightest storage server of memory load.
Always the work computing formula of load of storage server cluster is as follows:
L = Σ i = 1 N [ ( u i + d i ) / n i ] (2)
Wherein, N represents the sum of storage server in the storage server cluster, u i, d i, n iThe same formula of meaning (1);
Data distribution degree of unbalance represents that with the variance of each storage server disk free space percentage in the storage server cluster its computing formula is as follows:
U = Σ i = 1 N ( a i - a ‾ ) 2 N (3)
Wherein, a iThe same formula of meaning (1), the same formula of the meaning of N (2), The mathematic expectaion of each storage server disk free space percentage in the expression storage server cluster;
L ThresholdMethod by the sampling assessment determines that concrete grammar is: main control server is safeguarded a variables L Max, the initial value of this variable is 0, has preserved always the work maximum sampled value of load L of storage server cluster since the main control server self-starting, L Threshold=L Max* 20%;
U ThresholdMethod by sample statistics determines that concrete grammar is: All Files deposits N platform storage server as analog sample in the mode of poll at first selected certain large server, calculates the data distribution degree of unbalance U of N platform storage server under this situation RR, U Threshold=U RR* 80%.
Processing method of the present invention also comprises: the method for supervising of the extended method of storage server cluster and storage server cluster,
Wherein, the extended method of storage server cluster may further comprise the steps: new storage server is added in the storage server cluster, after starting, this storage server will periodically send heartbeat message to main control server, comprise the current state of this storage server in the heartbeat message, main control server receives behind this heartbeat message the logical node of newly-built this storage server in internal memory, and this logical node is added in the tabulation of online storage service device;
The method for supervising of storage server cluster may further comprise the steps: main control server is kept the logical node tabulation of online storage server in internal memory, the state that the last heartbeat of in store respective stored server is reported in each logical node in the tabulation; Be provided with a monitored variable state=1 in each logical node, the passive acceptance of main control server is from the heartbeat message of storage server, in case receive the heartbeat of certain storage server, then upgrade the state information in its counterlogic node, and the state variable of this node is set to 2; Main control server is periodically checked each node in the logical node tabulation, proof cycle equals the heart beat cycle of storage server, during inspection, the state value of tested node is subtracted 1, if the value of state is 0, then make the judgement of " this storage server goes offline ", and the log-on data automatic restore procedure.
The current state that comprises this storage server in the heartbeat message of the present invention, these state informations comprise CPU usage, memory usage, network interface card up/downstream rate and disk expend.
In the step of the present invention (b): the directory information of being preserved in the customer end A C-RU buffer memory comprises path, date created, the directory attribute of this catalogue; Metadata information comprises IP address, port numbers and the corresponding GUID of path, establishment/modification date, file attribute, file size, place primary storage server and the back-up storage server of this document.
Beneficial effect of the present invention:
Relate to storage server cluster automatic expansion method, dynamic load balancing method, online backup and automatic recovery method, AC-RU (Adjust-Controlled Recently-Used) caching method, client virtual disk method in based on network PB level cloud storage system of the present invention and the processing method thereof.Wherein, storage server cluster automatic expansion method has solved the problem of traditional mass storage system (MSS) expansion difficulty.The unbalanced problem of load Distribution can appear in the cloud storage server cluster that dynamic load balancing method has solved isomery after long-time running.Online backup and automatic recovery method have solved the integrity problem when adopting general low performance PC cluster that the cloud stores service is provided.The AC-RU caching method has solved in the cloud memory file system the frequent problem of adjusting of buffer memory sequence that occurs when using traditional buffer scheduling strategy.
The invention solves the problem that there is the expansion upper limit in traditional mass storage system (MSS) and stores isolated island.The present invention is by main control server management cloud storage pool, the cloud storage pool is invented the memory device of a magnanimity, in the cloud storage pool, add new storage server and can realize capacity extension, and there is not the expansion upper limit in this expansion, avoided the generation of storage isolated island.
In the present invention, heartbeat mechanism by storage server, main control server can be grasped global state in real time, utilizes global state can carry out the balanced and task load balancing of memory load accurately, effectively raises the concurrent task disposal ability of cloud memory file system.
The present invention is by being provided with the AC-RU buffer memory in cloud storage client and storage server end, catalogue and metadata request quantity that cloud storage client is initiated to main control server have been significantly reduced, reduce the frequency of storage server opening/closing file, effectively raised the response speed of system.The AC-RU buffer memory has overcome the problem of the frequent adjustment of buffer memory sequence that can occur under the applied environment that traditional buffer scheduling mechanism is concurrent at height, task is frequently switched, and has further improved the response speed of system.
The present invention has solved and has used cheap common apparatus to make up the integrity problem that cloud storage system brings by the online backup and automatic Restoration Mechanism of FFS, has guaranteed reliability of data storage.
Description of drawings
Fig. 1 is FFS cloud storage system physical topology figure;
Fig. 2 is a FFS cloud storage system building-block of logic;
Fig. 3 is the schematic diagram of FFS monitoring storage server cluster state;
Fig. 4 is the monitoring flow process of main control server to the storage server cluster;
Fig. 5 is the process of FFS client write data;
Fig. 6 is the process of FFS deleted file;
Fig. 7 is the process of FFS restore data;
Fig. 8 is an AC-RU buffer scheduling algorithm.
Fig. 9 is a FFS client virtual disk read-write flow chart.
Figure 10 is bibliographic structure and storage server logical node tabulation schematic diagram.
Figure 11 is Archive sit and storage server logical node
Embodiment
The present invention is further illustrated below in conjunction with drawings and Examples.
As shown in Figure 1, a kind of based on network PB level cloud storage system, it comprises main control server, storage server cluster and client, they carry out exchanges data by the inner exchanging machine;
Main control server is used for providing directory information and metadata information to cloud storage client, and the storage server cluster is monitored;
The storage server cluster comprises many storage servers that are used for storage, and it is provided with the AC-RU buffer memory, preserves several file handles of opening recently;
Client is used for providing the virtual disk service to the cloud storage client, the cloud storage client is submitted to main control server to the operation requests of virtual disk, and from storage server read/write file data, described client modules is deployed on the cloud storage client, it is provided with the AC-RU buffer memory, preserves the directory information of visit recently and the metadata information of file.
The service of client virtual disk:
In order to shield the visit details of client application to the cloud memory file system, protection has software resource now, and the access mode that the cloud memory file system provides client application is answered the conformance with standard file access mode, i.e. the disk read-write mode.As shown in Figure 9, FFS design and support conduct interviews in the disk mode from Windows and linuxn client.
Application process is converted into access request to the Fuse driver module to the access request of file in the cloud memory file system under Linux, be converted into the access request to the Dokan driver module under Windows.Fuse is the kit of increasing income of Application and Development layer file system under the Linux, its Fuse by kernel mode drives and calls the open file read-write interface of giving user's attitude, the developer only need realize user's attitude file read-write interface of Fuse in user's attitude, then can realize the exploitation of file system.Dokan is the kit of increasing income that is similar to Fuse under the Windows.
Cloud storage client of the present invention comprises mailbox server, ftp server, Web server, database server or streaming media server for needing to use the large server of magnanimity cloud stores service.
Directory information of the present invention comprises path, date created, the directory attribute of this catalogue; The IP address, port that metadata information is meant primary storage server that file is specifically deposited and back-up storage server with and corresponding GUID.
A kind of processing method of based on network PB level cloud storage system, it may further comprise the steps:
(a), adopt many storage servers to set up the storage server cluster, periodically send heartbeat message after each storage server starts to main control server, comprise the current state of this storage server in the heartbeat message; Be provided with one in the internal memory of main control server corresponding to the tabulation of the logical node of a plurality of storage servers, receive the logical node that in the tabulation of internal memory, upgrades behind the heartbeat message of each storage server corresponding to each storage server when main control server;
(b), when under the client-access virtual disk during any catalogue, cloud storage client is initiated request to main control server, obtain subdirectory information and the file metadata information under the catalogue of asking, and subdirectory information and the file metadata information that is obtained is kept in the customer end A C-RU buffer memory;
(c), cloud storage client-requested creates the operation of file or catalogue, wherein;
I, when cloud storage client has the request of creating file, client is submitted to main control server with this request, whether the catalogue that the file place is created in the main control server inspection exists, if there is not the then non-existent answer of Returning catalogue; If exist, then distribute GUID for this document, from the online storage service device, select load lighter two primary storage server and back-up storage servers as this document, and the file of notifying these two storage servers to create GUID by name, two storage servers are all created success and are then returned the successful answer of establishment to client;
Ii, when cloud storage client has the request that creaties directory, client is submitted to main control server with this request, main control server checks whether the parent directory of newly-built catalogue exists, if do not exist, then returns the non-existent answer of parent directory; If exist, then set up the new directory node, and add in the subdirectory tabulation of parent directory node, and return the successful answer of establishment to client;
(d), cloud storage client-requested reads file, written document, deleted file, duplicates/move or Rename file operation, wherein:
When i, cloud storage client-requested is read file, at first from the AC-RU buffer memory of cloud storage client, search the metadata information of this document, if there is the primary storage server that then finds correspondence, and the primary storage server requests is read concrete a certain section the data of this document by the metadata information of file; At first do not send metadata request if do not exist then in the buffer memory, and will add from the metadata information that main control server obtains the customer end A C-RU buffer memory to main control server.
When ii, cloud storage client-requested written document, at first from the AC-RU buffer memory of cloud storage client, search the metadata information of this document, if exist the storage of cloud then client to find corresponding primary storage server and back-up storage server by the metadata information of file, and their requests are write concrete a certain section the data of this document, after all returning the response of writing success etc. primary storage server and back-up storage server, the success of this time write operation, otherwise think and write failure; At first do not send metadata request if do not exist then in the buffer memory, and will add from the metadata information that main control server obtains the customer end A C-RU buffer memory to main control server.
Iii, during cloud storage client-requested deleted file, at first send file deletion requests to main control server, main control server finds the master file storage server and the backup file storage server of concrete preservation this document according to the metadata information of this document, and this metadata information deleted from bibliographic structure, return the successful response of deletion to cloud storage client simultaneously, main control server will be notified primary storage server and back-up storage server deletion this document of this document, two storage servers execute file deletion action after receiving the file delete instruction of main control server then.
Iv, cloud storage client-requested duplicates/and the move operation is the combination operation of establishment/read/write/deleted file;
V, Rename file are to revise file metadata information under the bibliographic structure in main control server;
Bibliographic structure:
Bibliographic structure is a kind of tree-shaped logical construction, the information that saves contents and file metadata information, and the subordinate relation between subdirectory and parent directory, file and the catalogue.
The interior nodes of tree-shaped logical construction all is a directory node, and information saves contents.Directory information comprises path, the directory attribute of this catalogue, the date created of catalogue, and points to the pointer list of subdirectory node and point to the pointer list of Archive sit.
The leaf node of tree-shaped logical construction may be a directory node, also may be Archive sit.If illustrating, directory node do not comprise subdirectory and file under this catalogue.If Archive sit is then preserved file metadata information.File metadata information comprises path, the file attribute of this document, the establishment/modification date of file, and the GUID of IP address, port numbers and this document that preserves the storage server of master file and backup file.
(e), cloud storage client-requested deltrees, duplicates/mobile catalogue, rename directory operation, wherein:
When i, cloud storage client-requested deltrees, at first whether there is the information of being deleted catalogue in the viewing client-side AC-RU buffer memory, if exist then it is deleted from buffer memory.Send the directory delete request to main control server then, its subdirectory of traversal was tabulated and listed files after main control server was received the directory delete request, recursively deleted all subdirectories and file, this directory delete the most at last under this catalogue.In the recurrence delete procedure, when certain subdirectory is a leaf node in the bibliographic structure, then directly deletion, otherwise the recurrence delete procedure takes place; When deltreeing down file, carry out according to step (d) .iii method.
Duplicate/mobile catalogue, rename directory with the file class of operation seemingly.
In the step of the present invention (c), selecting load lighter two from the online storage service device is to adopt the system load balancing method as the primary storage server of this document and the method for back-up storage server; The load calculating method of each storage server is:
l i＝a i×(u i+d i)/n i (1)
Wherein, l iBe the load value of i platform storage server in the storage server cluster, a iThe disk free space percentage of storage server for this reason, u iThe network interface card upstream rate of storage server for this reason, d iBe its downstream rate, n iBe the maximum rate of this network interface card, u i, d iAnd n iUnit all be kbps.
System load balancing method of the present invention also comprises: main control server periodically calculates the total work load and the data distribution degree of unbalance of current storage server cluster; When total work load is lower than certain set point L Threshold, and data distribution degree of unbalance is higher than certain set point U ThresholdThe time, then the heaviest storage server of memory load is carried out the data migration, data are moved to the lightest storage server of memory load.
Always the work computing formula of load of storage server cluster is as follows:
L = Σ i = 1 N [ ( u i + d i ) / n i ] (2)
Wherein, N represents the sum of storage server in the storage server cluster, u i, d i, n iThe same formula of meaning (1);
Data distribution degree of unbalance represents that with the variance of each storage server disk free space percentage in the storage server cluster its computing formula is as follows:
U = Σ i = 1 N ( a i - a ‾ ) 2 N (3)
Wherein, a iThe same formula of meaning (1), the same formula of the meaning of N (2),
Figure BDA0000044592430000123
The mathematic expectaion of each storage server disk free space percentage in the expression storage server cluster;
L ThresholdMethod by the sampling assessment determines that concrete grammar is: main control server is safeguarded a variables L Max, the initial value of this variable is 0, has preserved always the work maximum sampled value of load L of storage server cluster since the main control server self-starting, L Threshold=L Max* 20%;
U ThresholdMethod by sample statistics determines that concrete grammar is: All Files deposits N platform storage server as analog sample in the mode of poll at first selected certain large server, calculates the data distribution degree of unbalance U of N platform storage server under this situation RR, U Threshold=U RR* 80%.
In the experiment, it is example that sample is selected with the All Files on the campus network ftp server, according to statistics as can be known, has 25169 files on this ftp server, 261242 files, and the file size summation is 4.75TB.Minimum file is 0 byte in this group sample, and maximum file is 34.29GB, and wherein the size distribution of most files is at e 5～e 20In the bytes range.These files are saved in 50 storage servers successively in the mode of poll, and every storage server total capacity is 250GB, through calculating as can be known U RR=6.88, U Threshold=5.51.
Processing method of the present invention also comprises: the method for supervising of the extended method of storage server cluster and storage server cluster,
Wherein, the extended method of storage server cluster may further comprise the steps: new storage server is added in the storage server cluster, after starting, this storage server will periodically send heartbeat message to main control server, comprise the current state of this storage server in the heartbeat message, main control server receive behind this heartbeat message will be in internal memory the logical node of newly-built this storage server, and this logical node added in the tabulation of online storage service device;
The method for supervising of storage server cluster may further comprise the steps: main control server is kept the logical node tabulation of online storage server in internal memory, the state that the last heartbeat of in store respective stored server is reported in each logical node in the tabulation; Be provided with a monitored variable state=1 in each logical node, the passive acceptance of main control server is from the heartbeat message of storage server, in case receive the heartbeat of certain storage server, then upgrade the state information in its counterlogic node, and the state variable of this node is set to 2; Main control server is periodically checked each node in the logical node tabulation, proof cycle equals the heart beat cycle of storage server, during inspection, the state value of tested node is subtracted 1, if the value of state is 0, then make the judgement of " this storage server goes offline ", and the log-on data automatic restore procedure.
Comprise the current state of this storage server in the heartbeat message of the present invention, for CPU usage, memory usage, network interface card up/downstream rate and disk expend.
In the step of the present invention (b): the directory information of being preserved in the customer end A C-RU buffer memory comprises path, date created, the directory attribute of this catalogue; File metadata information comprises IP address, port numbers and the corresponding GUID of path, establishment/modification date, file attribute, file size, place primary storage server and the back-up storage server of this document.
During concrete enforcement:
As shown in Figure 2, be the building-block of logic of FFS cloud storage system.The FFS cloud storage system is made up of three big modules altogether: main control server, storage server, cloud storage client.Main control server is deployed on the server of a better performances, storage server is deployed on each node in the storage server cluster, cloud storage client deployment (database server as shown in FIG., ftp server, mail server, Web server, streaming media server etc.) on the application server that the mass data storage demand is arranged.It is interconnected that each node uses gigabit switch in the FFS cloud storage system, can expand by cascade when the gigabit switch port is not enough.
Communicate by Intranet IP address between each node of FFS cloud storage system.Such as, setting main control server IP is 192.168.0.1, the IP address of setting each storage server and cloud storage client is 192.168.x.x.Because cloud storage client deployment is on application server, application server need externally provide service, so application server need dispose an outer net IP address again.Each storage server and cloud storage client need be distributed to " the main control server IP address " in the configuration file Intranet IP address of main control server when field is revised as actual disposition.
Fig. 3 is the schematic diagram of FFS monitoring storage server cluster state.Storage server initiatively sends heartbeat message to main control server, comprises in the heartbeat message on storage server current C PU utilization rate, memory usage, the network interface card/downstream rate, disk total capacity and used state informations such as capacity, storage file number.Main control server is opened a listening port, and passive monitoring is from the heartbeat message of storage server.
Fig. 4 is the algorithm flow of main control server monitoring storage server cluster.Monitor procedure is made up of two parts.Shown in Fig. 4 left-half, be the process of the passive collection storage server of main control server state.Fig. 4 right half part is each storage server state of periodic test, Storage Server List is sorted by load is ascending, and the offline storage server is carried out data recover.
Fig. 5 is the process of cloud storage client written document.In FFS, each file is each storage portion on different storage servers all, and portion is a master file, and portion is a backup file.When cloud storage client written document, can simultaneously data be write master file and backup file, all write success when them, just think this write data success, otherwise think the written document failure.
Fig. 6 is the process of cloud storage client-requested deleted file.Cloud storage client is submitted to main control server with the request of deleted file, and main control server is submitted to the request of deleted file the master file storage server and the backup file storage server at the actual place of this document.When master file and backup file are all deleted success, then main control server is to the successful response of cloud storage client backspace file deletion.
Fig. 7 is after main control server detects certain storage server off-line, the process that performed data are recovered.Shown in Figure 10,11, writing down the listed files of being preserved in each storage server in the main control server, the item in the listed files is for pointing to the pointer of the file leaf node under the bibliographic structure.Preserve two pointers in the file leaf node, point to the primary storage server logic node and the back-up storage server logic node of this document respectively.The data of FFS are recovered automatically based on the online backup technology.Online backup is meant the FFS client when creating file, can create duplicate of the document simultaneously on two storage servers, and one of them is a master file, and one is backup file.In order to keep the consistency of two copies, FFS has adopted Read-one Write-all strategy: because the file read operation is a safety operation, only need get final product from the master file reading of data; For the file write operation, client need write all copies simultaneously with data, and obtains the successfully answer of writing of all copies, just is judged as the success of file write operation.This strategy is realized simple, and has guaranteed that as far as possible each copy all has latest data at any time, has also reduced server load simultaneously, because the data dissemination burden has been transferred to client by server, this has improved extensibility conversely again.Online backup mechanism has guaranteed that every part of file that is kept among the FFS all has two on all four copies under the normality.When certain storage server lost efficacy in the storage cluster, main control server can be found this failure node immediately by heartbeat mechanism, and the log-on data automatic restore procedure.As shown in figure 10, in main control server, keeping a bibliographic structure and a storage server logical node tabulation.Bibliographic structure is a kind of tree structure.In directory tree, intermediate node is the subdirectory node, the routing information of in store this catalogue, directory attribute information, date created information in the node; Leaf node is empty list node or Archive sit, the routing information of in store respective file, file attribute information, establishment/modification date in the Archive sit, and the logical node pointer of pairing two storage servers of this document and its master file corresponding on storage server and the GUID of backup file.GUID is file overall unique number by the main control server unified distribution when creating, and uses this numbering to preserve the problem that file can be avoided file conflict of the same name on storage server.Preserved the IP address of corresponding stored server and port numbers, state variable, CPU/ internal memory/network interface card/Disk State in the storage server logical node, and master file pointer list and backup file pointer list.As shown in figure 11, be the Data View of Archive sit and storage server logical node.In Archive sit, can find active/standby part of storage server logical node depositing this document by active/standby part of file pointer; In the storage server logical node, can find corresponding file node in the bibliographic structure by active/standby part of listed files.After main control server judges that certain storage server goes offline, with log-on data recovery process.Data recovery procedure can travel through the file pointer tabulation of preserving in this storage server logical node that goes offline, and finds corresponding file node under the bibliographic structure.Storage server is the primary storage server of this document if go offline, its back-up storage server of then from the file leaf node, looking for, with its back-up storage server updating is the primary storage server, and select the lighter storage server of load as the new back-up storage server of this document, this new back-up storage server of notice is created this document then, and reads the file restructure backup from the current primary storage server of this document; Storage server is the back-up storage server of this document if go offline, then select the lighter storage server of load as the new back-up storage server of this document, and notify this new back-up storage server to create this document, and read the file restructure backup from the primary storage server of this document.
Fig. 8 is the metadata cache of cloud storage client and the employed AC-RU dispatching algorithm of file handle buffer memory of storage server end.When new object adds buffer memory, add the head of buffer memory to, original object moves after successively in the buffer memory, if buffer memory was taken before new object added, will cause the object of buffer memory afterbody to be moved out of buffer memory after then new object adds.When existing object is hit in the buffer memory, if this object is in buffer memory the first half position, then do not carry out the adjustment of buffer memory sequence, if this object has been in half position behind the buffer memory, then this object is adjusted to the head of buffer memory.This shows, when the cache object that hits is in the first half of buffer memory, the adjustment of buffer memory sequence can't take place, the adjustment of buffer memory sequence only occurs in the adding of new object or the cache object that hits is in the back half of buffer memory.
The part that the present invention does not relate to prior art that maybe can adopt all same as the prior art is realized.

Claims (9)

1. a based on network PB level cloud storage system is characterized in that it comprises main control server, storage server cluster and client, and they carry out exchanges data by the inner exchanging machine;
Main control server is used for providing directory information and metadata information to cloud storage client, and the storage server cluster is monitored;
The storage server cluster comprises many storage servers that are used for storage, and it is provided with the AC-RU buffer memory, preserves several file handles of opening recently;
Client is used for providing the virtual disk service to the cloud storage client, the cloud storage client is submitted to main control server to the operation requests of virtual disk, and from storage server read/write file data, described client modules is deployed on the cloud storage client, it is provided with the AC-RU buffer memory, preserves the directory information of visit recently and the metadata information of file.
2. based on network PB level cloud storage system according to claim 1, it is characterized in that the large server of described client, comprise mailbox server, ftp server, Web server, database server or streaming media server for needs use magnanimity cloud stores service.
3. based on network PB level cloud storage system according to claim 1 is characterized in that described directory information comprises path, date created, the directory attribute of this catalogue; Metadata information comprises IP address, port numbers and the corresponding GUID of path, establishment/modification date, file attribute, file size, place primary storage server and the back-up storage server of this document.
4. the processing method of a based on network PB level cloud storage system adopts the described based on network PB level cloud storage system of claim 1, it is characterized in that it may further comprise the steps:
(a), adopt many storage servers to set up the storage server cluster, periodically send heartbeat message after each storage server starts to main control server, comprise the current state of this storage server in the heartbeat message; Be provided with one in the internal memory of main control server corresponding to the tabulation of the logical node of a plurality of storage servers, receive behind the heartbeat message of each storage server in upgrading tabulation logical node when main control server corresponding to each storage server;
(b), when under the client-access virtual disk during any catalogue, cloud storage client is initiated request to main control server, obtain subdirectory information and the file metadata information under the catalogue of asking, and subdirectory information and the file metadata information that is obtained is kept in the customer end A C-RU buffer memory;
(c), cloud storage client-requested creates the operation of file or catalogue, wherein;
I, when cloud storage client has the request of creating file, client is submitted to main control server with this request, whether the catalogue that the file place is created in the main control server inspection exists, if there is not the then non-existent answer of Returning catalogue; If exist, then distribute GUID for this document, from the online storage service device, select load lighter two primary storage server and back-up storage servers as this document, and the file of notifying these two storage servers to create GUID by name, two storage servers are all created success and are then returned the successful answer of establishment to client;
Ii, when cloud storage client has the request that creaties directory, client is submitted to main control server with this request, main control server checks whether the parent directory of newly-built catalogue exists, if do not exist, then returns the non-existent answer of parent directory; If exist, then set up the new directory node, and add in the subdirectory tabulation of parent directory node, and return the successful answer of establishment to client;
(d), cloud storage client-requested reads file, written document, deleted file, duplicates/move or Rename file operation, wherein:
When i, cloud storage client-requested is read file, at first from the AC-RU buffer memory of cloud storage client, search the metadata information of this document, if metadata information exists in the buffer memory, then the metadata information by file finds corresponding primary storage server, and the primary storage server requests is read concrete a certain section the data of this document; If metadata information does not exist in the buffer memory, then at first send metadata request, and will add the customer end A C-RU buffer memory from the metadata information that main control server obtains to main control server;
When ii, cloud storage client-requested written document, at first from the AC-RU buffer memory of cloud storage client, search the metadata information of this document, if metadata information exists in the buffer memory, then cloud storage client finds corresponding primary storage server and back-up storage server by the metadata information of file, and write concrete a certain section the data of this document to their requests, after all returning the response of writing success etc. primary storage server and back-up storage server, the success of this time write operation, otherwise think and write failure; If metadata information does not exist in the buffer memory, then at first send metadata request, and will add the customer end A C-RU buffer memory from the metadata information that main control server obtains to main control server;
Iii, during cloud storage client-requested deleted file, at first send file deletion requests to main control server, main control server finds the master file storage server and the backup file storage server of concrete preservation this document according to the metadata information of this document, and this metadata information deleted from bibliographic structure, return the successful response of deletion to cloud storage client simultaneously, main control server will be notified primary storage server and back-up storage server deletion this document of this document, two storage servers execute file deletion action after receiving the file delete instruction of main control server then;
Iv, cloud storage client-requested duplicates/and the move operation is the combination of establishment/read/write/delete file operation;
V, Rename file are to revise file metadata information under the bibliographic structure in main control server; (e), the cloud storage client-requested operation that deltrees, wherein: when cloud storage client-requested deltrees, at first whether there is the information of being deleted catalogue in the viewing client-side AC-RU buffer memory,, sends the directory delete request to main control server then from buffer memory if exist then it is deleted; If there is no, then directly send the directory delete request to main control server, its subdirectory of traversal was tabulated and listed files after main control server was received the directory delete request, recursively deleted all subdirectories and file, this directory delete the most at last under this catalogue; In the recurrence delete procedure, when certain subdirectory is a leaf node in the bibliographic structure, then directly deletion, otherwise the recurrence delete procedure takes place; When deltreeing down file, carry out according to step (d) .iii method.
5. the processing method of based on network PB level cloud storage system according to claim 4, it is characterized in that in the step (c) that selecting load lighter two from the online storage service device is to adopt the system load balancing method as the primary storage server of this document and the method for back-up storage server; The load calculating method of each storage server is:
l i＝a i×(u i+d i)/n i (1)
Wherein, l iBe the load value of i platform storage server in the storage server cluster, a iThe disk free space percentage of storage server for this reason, u iThe network interface card upstream rate of storage server for this reason, d iBe its downstream rate, n iBe the maximum rate of this network interface card, u i, d iAnd n iUnit all be kbps.
6. the processing method of based on network PB level cloud storage system according to claim 5 is characterized in that described system load balancing method also comprises: main control server periodically calculates the total work load and the data distribution degree of unbalance of current storage server cluster; When total work load is lower than certain set point L Threshold, and data distribution degree of unbalance is higher than certain set point U ThresholdThe time, then the heaviest storage server of memory load is carried out the data migration, data are moved to the lightest storage server of memory load.
Always the work computing formula of load of storage server cluster is as follows:
L = Σ i = 1 N [ ( u i + d i ) / n i ] (2)
Wherein, N represents the sum of storage server in the storage server cluster, u i, d i, n iThe same formula of meaning (1);
Data distribution degree of unbalance represents that with the variance of each storage server disk free space percentage in the storage server cluster its computing formula is as follows:
U = Σ i = 1 N ( a i - a ‾ ) 2 N (3)
Wherein, a iThe same formula of meaning (1), the same formula of the meaning of N (2),
Figure FDA0000044592420000043
The mathematic expectaion of each storage server disk free space percentage in the expression storage server cluster;
L ThresholdMethod by the sampling assessment determines that concrete grammar is: main control server is safeguarded a variables L Max, the initial value of this variable is 0, has preserved always the work maximum sampled value of load L of storage server cluster since the main control server self-starting, L Threshold=L Max* 20%;
U ThresholdMethod by sample statistics determines that concrete grammar is: All Files deposits N platform storage server as analog sample in the mode of poll at first selected certain large server, calculates the data distribution degree of unbalance U of N platform storage server under this situation RR, U Threshold=U RR* 80%.
7. based on network PB level cloud storage system according to claim 4 is characterized in that this processing method also comprises: the method for supervising of the extended method of storage server cluster and storage server cluster,
Wherein, the extended method of storage server cluster may further comprise the steps: new storage server is added in the storage server cluster, after starting, this storage server will periodically send heartbeat message to main control server, comprise the current state of this storage server in the heartbeat message, main control server receives behind this heartbeat message the logical node of newly-built this storage server in internal memory, and this logical node is added in the tabulation of online storage service device;
The method for supervising of storage server cluster may further comprise the steps: main control server is kept the logical node tabulation of online storage server in internal memory, the state that the last heartbeat of in store respective stored server is reported in each logical node in the tabulation; Be provided with a monitored variable state=1 in each logical node, the passive acceptance of main control server is from the heartbeat message of storage server, in case receive the heartbeat of certain storage server, then upgrade the state information in its counterlogic node, and the state variable of this node is set to 2; Main control server is periodically checked each node in the logical node tabulation, proof cycle equals the heart beat cycle of storage server, during inspection, the state value of tested node is subtracted 1, if the value of state is 0, then make the judgement of " this storage server goes offline ", and the log-on data automatic restore procedure.
8. according to the processing method of claim 4 or 7 described based on network PB level cloud storage systems, it is characterized in that comprising in the described heartbeat message current state of this storage server, these state informations comprise CPU usage, memory usage, network interface card up/downstream rate and disk expend.
9. the processing method of based on network PB level cloud storage system according to claim 4,, it is characterized in that in the step (b): the directory information of being preserved in the customer end A C-RU buffer memory comprises path, date created, the directory attribute of this catalogue; Metadata information comprises IP address, port numbers and the corresponding GUID of path, establishment/modification date, file attribute, file size, place primary storage server and the back-up storage server of this document.
CN2011100227416A 2011-01-20 2011-01-20 Network-based PB level cloud storage system and processing method thereof Expired - Fee Related CN102143215B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN2011100227416A CN102143215B (en) 2011-01-20 2011-01-20 Network-based PB level cloud storage system and processing method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN2011100227416A CN102143215B (en) 2011-01-20 2011-01-20 Network-based PB level cloud storage system and processing method thereof

Publications (2)

Publication Number Publication Date
CN102143215A true CN102143215A (en) 2011-08-03
CN102143215B CN102143215B (en) 2013-04-10

Family

ID=44410434

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2011100227416A Expired - Fee Related CN102143215B (en) 2011-01-20 2011-01-20 Network-based PB level cloud storage system and processing method thereof

Country Status (1)

Country Link
CN (1) CN102143215B (en)

Cited By (84)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102244685A (en) * 2011-08-11 2011-11-16 中国科学院软件研究所 Distributed type dynamic cache expanding method and system supporting load balancing
CN102355502A (en) * 2011-10-13 2012-02-15 罗立群 Remote access method for remotely accessing storage system into desktop operation system
CN102523301A (en) * 2011-12-26 2012-06-27 深圳市创新科信息技术有限公司 Method for caching data on client in cloud storage
CN102571968A (en) * 2012-01-17 2012-07-11 华中科技大学 Directional data forwarding method in cloud backup system
CN102710790A (en) * 2012-06-20 2012-10-03 深圳市远行科技有限公司 Memcached implementation method and system based on metadata management
CN102724314A (en) * 2012-06-20 2012-10-10 深圳市远行科技有限公司 Memcached client side based on metadata management
CN102724330A (en) * 2012-07-09 2012-10-10 南京理工大学常熟研究院有限公司 Agricultural wireless sensor network system
CN102752146A (en) * 2012-07-13 2012-10-24 中国科学院微电子研究所 Cluster topological graph generation method and server
CN102880531A (en) * 2012-09-27 2013-01-16 新浪网技术(中国)有限公司 Database backup system and backup method and slave database server of database backup system
CN102904949A (en) * 2012-10-08 2013-01-30 华中科技大学 Replica-based dynamic metadata cluster system
CN102904948A (en) * 2012-09-29 2013-01-30 南京云创存储科技有限公司 Super-large-scale low-cost storage system
CN102904952A (en) * 2012-10-12 2013-01-30 北京锐安科技有限公司 Self-adapting system and method for efficiently processing input of mass data to database
CN102946429A (en) * 2012-11-07 2013-02-27 浪潮电子信息产业股份有限公司 High-efficiency dynamic resource scheduling method based on cloud storage
CN103001809A (en) * 2012-12-25 2013-03-27 曙光信息产业(北京)有限公司 Service node state monitoring method for cloud storage system
CN103067485A (en) * 2012-12-25 2013-04-24 曙光信息产业(北京)有限公司 Disk monitoring method for cloud storage system
CN103067438A (en) * 2011-10-18 2013-04-24 巴比禄股份有限公司 Network storage system, data manipulation method in network storage system, and storage device
CN103179185A (en) * 2012-12-25 2013-06-26 中国科学院计算技术研究所 Method and system for creating files in cache of distributed file system client
CN103186349A (en) * 2011-12-27 2013-07-03 杭州信核数据科技有限公司 Block-level distributed storage system and data reading-writing method thereof
CN103218453A (en) * 2013-04-28 2013-07-24 南京龙渊微电子科技有限公司 Method and device for splitting file
CN103236168A (en) * 2013-04-23 2013-08-07 深圳职业技术学院 Traffic data online acquisition system and method
CN103268252A (en) * 2013-05-12 2013-08-28 南京载玄信息科技有限公司 Virtualization platform system based on distributed storage and achieving method thereof
CN103338252A (en) * 2013-06-27 2013-10-02 南京邮电大学 Distributed database concurrence storage virtual request mechanism
CN103365979A (en) * 2013-07-03 2013-10-23 交通银行股份有限公司 Long-distance double-center online processing method and system based on open database
WO2013177968A1 (en) * 2012-05-28 2013-12-05 华为技术有限公司 File storage system and device, and file access method
CN103546572A (en) * 2013-10-30 2014-01-29 北京荣之联科技股份有限公司 Cloud storage device and multi-cloud storage networking system and method
CN103581243A (en) * 2012-07-27 2014-02-12 永泰软件有限公司 System and method for massive streaming media storage
CN103618804A (en) * 2013-12-16 2014-03-05 北京航空航天大学 Performance difference-based load balancing method for distributed key value storage system
CN103631894A (en) * 2013-11-19 2014-03-12 浪潮电子信息产业股份有限公司 Dynamic copy management method based on HDFS
CN103701916A (en) * 2013-12-31 2014-04-02 赛凡信息科技（厦门）有限公司 Dynamic load balancing method of distributed storage system
CN103744809A (en) * 2013-12-23 2014-04-23 天泽信息产业股份有限公司 Method for dual-computer hot-standby of vehicle information management system on basis of VRRP
CN103944958A (en) * 2014-03-14 2014-07-23 中国科学院计算技术研究所 Wide area file system and implementation method
WO2014117513A1 (en) * 2013-01-29 2014-08-07 中兴通讯股份有限公司 Method and device for managing recycle bin of cloud storage
CN104219279A (en) * 2013-06-04 2014-12-17 国际商业机器公司 Modular architecture for extreme-scale distributed processing applications
CN104391930A (en) * 2014-11-21 2015-03-04 用友软件股份有限公司 Distributed file storage device and method
CN104424292A (en) * 2013-09-02 2015-03-18 联想(北京)有限公司 Data processing method and electronic device
CN104469391A (en) * 2014-11-21 2015-03-25 深圳市天威视讯股份有限公司 Cloud-platform-based digital television content distribution system and method
CN104702634A (en) * 2013-12-04 2015-06-10 腾讯科技（深圳）有限公司 Method, device and system for processing data operation request
CN104780192A (en) * 2014-01-15 2015-07-15 携程计算机技术（上海）有限公司 File synchronization method, single-point server and cluster system
CN105117166A (en) * 2015-07-30 2015-12-02 苏州绿豆豆软件科技有限公司 Cloud storage card apparatus based on FPGA, and realization method therefor
CN105187565A (en) * 2015-10-14 2015-12-23 四川携创信息技术服务有限公司 Method for utilizing network storage data
CN105227672A (en) * 2015-10-13 2016-01-06 国家电网公司 The method and system that data store and access
CN105718484A (en) * 2014-12-04 2016-06-29 中兴通讯股份有限公司 File writing method, file reading method, file deletion method, file query method and client
CN105871659A (en) * 2016-05-31 2016-08-17 乐视控股（北京）有限公司 Monitoring method and device of newly added server
CN105956491A (en) * 2016-05-19 2016-09-21 北京春鸿企业管理咨询有限公司 Method and system for performing multi-account backup on data of public cloud storage system
CN106060173A (en) * 2016-07-22 2016-10-26 恒业智能信息技术（深圳）有限公司 Cluster type photographing data storage system based on cloud storage
CN106156289A (en) * 2016-06-28 2016-11-23 北京百迈客云科技有限公司 The method of the data in a kind of read-write object storage system and device
CN106294496A (en) * 2015-06-09 2017-01-04 北京京东尚科信息技术有限公司 A kind of data migration method based on hadoop cluster and instrument
CN106302757A (en) * 2016-08-19 2017-01-04 成都全码特时代科技有限公司 A kind of server cluster system based on cloud storage
CN106598503A (en) * 2016-12-23 2017-04-26 郑州云海信息技术有限公司 Method and device for optimizing storage space occupied by virtual disk under cloud platform
CN106790422A (en) * 2016-12-02 2017-05-31 北京锐安科技有限公司 A kind of data buffer storage cluster and data retrieval method for WEB application
CN106791908A (en) * 2016-11-25 2017-05-31 上海熙菱信息技术有限公司 A kind of live video stream storage method for supporting cloud platform using double buffering
CN106792825A (en) * 2016-12-29 2017-05-31 河北远东通信系统工程有限公司 A kind of management method of distributed network equipment
CN107205049A (en) * 2017-07-28 2017-09-26 郑州云海信息技术有限公司 A kind of network attached storage method and storage service apparatus
CN107229425A (en) * 2017-06-02 2017-10-03 浙江宇视科技有限公司 A kind of date storage method and device
CN107247714A (en) * 2016-06-01 2017-10-13 国家电网公司 A kind of small documents access system and method based on distributed storage technology
EP3261302A4 (en) * 2016-04-29 2018-03-21 Huawei Technologies Co., Ltd. Storage network element discovery method and device
CN107861690A (en) * 2017-11-22 2018-03-30 安徽伊尔思环境科技有限公司 A kind of big data optimizes project environmental impact assessment cloud storage system
CN107948248A (en) * 2017-11-01 2018-04-20 平安科技（深圳）有限公司 Distributed storage method, control server and computer-readable recording medium
CN108132815A (en) * 2017-12-25 2018-06-08 郑州云海信息技术有限公司 A kind of method, apparatus and storage medium of configuration file dynamic load
CN108243206A (en) * 2016-12-23 2018-07-03 航天星图科技（北京）有限公司 A kind of server cluster area management method
CN108809984A (en) * 2018-06-13 2018-11-13 苏州若依玫信息技术有限公司 A kind of cloud computing intelligent security system based on time domain
WO2019001280A1 (en) * 2017-06-29 2019-01-03 中兴通讯股份有限公司 Heterogeneous virtual computing resource management method, related device, and storage medium
WO2019020084A1 (en) * 2017-07-27 2019-01-31 杭州海康威视数字技术股份有限公司 Data storage method and system
CN109359100A (en) * 2018-09-28 2019-02-19 北京赛博贝斯数据科技有限责任公司 The visual modeling method of PB grades of historical datas and online data calculated in real time
CN109462642A (en) * 2018-10-30 2019-03-12 新华三技术有限公司成都分公司 Data processing method and device
CN109495540A (en) * 2018-10-15 2019-03-19 深圳市金证科技股份有限公司 A kind of method, apparatus of data processing, terminal device and storage medium
CN109658284A (en) * 2018-12-19 2019-04-19 国网北京市电力公司 The appraisal procedure and system of equipment state
CN109885552A (en) * 2019-02-18 2019-06-14 天固信息安全系统(深圳)有限责任公司 The metadata dynamic management approach and distributed file system of distributed file system
CN110300131A (en) * 2018-03-21 2019-10-01 北京金风科创风电设备有限公司 Routing method, device, equipment and system for multiple services of wind power plant
CN110535829A (en) * 2019-07-24 2019-12-03 视联动力信息技术股份有限公司 A kind of data interactive method and a kind of view networked system
CN111078120A (en) * 2018-10-18 2020-04-28 深信服科技股份有限公司 Data migration method and system of distributed file system and related components
CN111581020A (en) * 2020-04-22 2020-08-25 上海天玑科技股份有限公司 Method and device for data recovery in distributed block storage system
CN111625401A (en) * 2020-05-29 2020-09-04 浪潮电子信息产业股份有限公司 Data backup method and device based on cluster file system and readable storage medium
CN111767282A (en) * 2020-06-12 2020-10-13 咪咕文化科技有限公司 MongoDB-based storage system, data insertion method and storage medium
CN111901174A (en) * 2020-07-30 2020-11-06 腾讯科技（深圳）有限公司 Service state notification method, related device and storage medium
CN112068780A (en) * 2020-09-10 2020-12-11 国家卫星气象中心(国家空间天气监测预警中心) Data unified management system based on heterogeneous storage
CN112236758A (en) * 2018-05-31 2021-01-15 微软技术许可有限责任公司 Cloud storage distributed file system
CN112398871A (en) * 2020-12-01 2021-02-23 厦门市美亚柏科信息股份有限公司 Mass file transmission method and system
CN112445770A (en) * 2020-11-30 2021-03-05 清远职业技术学院 Super-large-scale high-performance database engine with multi-dimensional out-of-order storage function and cloud service platform
CN112579543A (en) * 2020-12-29 2021-03-30 航天科工网络信息发展有限公司 Dynamic metadata management method for distributed file system and distributed file system
CN113032099A (en) * 2021-03-26 2021-06-25 阿里巴巴新加坡控股有限公司 Cloud computing node, file management method and device
CN113032356A (en) * 2021-03-31 2021-06-25 中电科航空电子有限公司 Cabin distributed file storage system and implementation method
CN114328007A (en) * 2021-11-19 2022-04-12 苏州浪潮智能科技有限公司 Container backup and restoration method, device and medium thereof
CN114756509A (en) * 2022-05-19 2022-07-15 北京百度网讯科技有限公司 Operation method, system, device and storage medium of file system

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109213746A (en) * 2018-09-28 2019-01-15 北京赛博贝斯数据科技有限责任公司 The visual modeling method of PB grades of historical datas and online data calculated in real time

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101582841A (en) * 2008-05-15 2009-11-18 北京师范大学 Web data coupling method and Web data coupling system based on mobile storage device and service packet
US20100268764A1 (en) * 2009-04-15 2010-10-21 Wee Sewook Method and system for client-side scaling of web server farm architectures in a cloud data center

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101582841A (en) * 2008-05-15 2009-11-18 北京师范大学 Web data coupling method and Web data coupling system based on mobile storage device and service packet
US20100268764A1 (en) * 2009-04-15 2010-10-21 Wee Sewook Method and system for client-side scaling of web server farm architectures in a cloud data center
CN101883103A (en) * 2009-04-15 2010-11-10 埃森哲环球服务有限公司 The method and system of the client-side extensions of Web server gang fight structure in the cloud data center

Cited By (131)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102244685A (en) * 2011-08-11 2011-11-16 中国科学院软件研究所 Distributed type dynamic cache expanding method and system supporting load balancing
CN102244685B (en) * 2011-08-11 2013-09-18 中国科学院软件研究所 Distributed type dynamic cache expanding method and system for supporting load balancing
CN102355502A (en) * 2011-10-13 2012-02-15 罗立群 Remote access method for remotely accessing storage system into desktop operation system
CN102355502B (en) * 2011-10-13 2015-02-04 罗立群 Remote access method for remotely accessing storage system into desktop operation system
CN103067438A (en) * 2011-10-18 2013-04-24 巴比禄股份有限公司 Network storage system, data manipulation method in network storage system, and storage device
CN102523301A (en) * 2011-12-26 2012-06-27 深圳市创新科信息技术有限公司 Method for caching data on client in cloud storage
CN103186349A (en) * 2011-12-27 2013-07-03 杭州信核数据科技有限公司 Block-level distributed storage system and data reading-writing method thereof
CN102571968A (en) * 2012-01-17 2012-07-11 华中科技大学 Directional data forwarding method in cloud backup system
CN102571968B (en) * 2012-01-17 2014-09-24 华中科技大学 Directional data forwarding method in cloud backup system
WO2013177968A1 (en) * 2012-05-28 2013-12-05 华为技术有限公司 File storage system and device, and file access method
US9405757B2 (en) 2012-05-28 2016-08-02 Huawei Technologies Co., Ltd. File storage system, apparatus, and file access method
CN102724314A (en) * 2012-06-20 2012-10-10 深圳市远行科技有限公司 Memcached client side based on metadata management
CN102724314B (en) * 2012-06-20 2015-08-05 深圳市远行科技有限公司 A kind of distributed caching client based on metadata management
CN102710790B (en) * 2012-06-20 2015-06-10 深圳市远行科技有限公司 Memcached implementation method and system based on metadata management
CN102710790A (en) * 2012-06-20 2012-10-03 深圳市远行科技有限公司 Memcached implementation method and system based on metadata management
CN102724330A (en) * 2012-07-09 2012-10-10 南京理工大学常熟研究院有限公司 Agricultural wireless sensor network system
CN102752146B (en) * 2012-07-13 2015-07-15 中国科学院微电子研究所 Cluster topological graph generation method and server
CN102752146A (en) * 2012-07-13 2012-10-24 中国科学院微电子研究所 Cluster topological graph generation method and server
CN103581243A (en) * 2012-07-27 2014-02-12 永泰软件有限公司 System and method for massive streaming media storage
CN103581243B (en) * 2012-07-27 2017-03-29 程治永 A kind of system and method for magnanimity Streaming Media storage
CN102880531A (en) * 2012-09-27 2013-01-16 新浪网技术(中国)有限公司 Database backup system and backup method and slave database server of database backup system
CN102904948A (en) * 2012-09-29 2013-01-30 南京云创存储科技有限公司 Super-large-scale low-cost storage system
CN102904949B (en) * 2012-10-08 2015-07-01 华中科技大学 Replica-based dynamic metadata cluster system
CN102904949A (en) * 2012-10-08 2013-01-30 华中科技大学 Replica-based dynamic metadata cluster system
CN102904952A (en) * 2012-10-12 2013-01-30 北京锐安科技有限公司 Self-adapting system and method for efficiently processing input of mass data to database
CN102904952B (en) * 2012-10-12 2015-07-01 北京锐安科技有限公司 Self-adapting system and method for efficiently processing input of mass data to database
CN102946429A (en) * 2012-11-07 2013-02-27 浪潮电子信息产业股份有限公司 High-efficiency dynamic resource scheduling method based on cloud storage
CN103001809B (en) * 2012-12-25 2016-12-28 曙光信息产业(北京)有限公司 Service node method for monitoring state for cloud storage system
CN103179185A (en) * 2012-12-25 2013-06-26 中国科学院计算技术研究所 Method and system for creating files in cache of distributed file system client
CN103067485A (en) * 2012-12-25 2013-04-24 曙光信息产业(北京)有限公司 Disk monitoring method for cloud storage system
CN103001809A (en) * 2012-12-25 2013-03-27 曙光信息产业(北京)有限公司 Service node state monitoring method for cloud storage system
CN103179185B (en) * 2012-12-25 2015-07-08 中国科学院计算技术研究所 Method and system for creating files in cache of distributed file system client
WO2014117513A1 (en) * 2013-01-29 2014-08-07 中兴通讯股份有限公司 Method and device for managing recycle bin of cloud storage
CN103236168A (en) * 2013-04-23 2013-08-07 深圳职业技术学院 Traffic data online acquisition system and method
CN103236168B (en) * 2013-04-23 2016-11-02 深圳职业技术学院 Traffic data on-line acquisition system and method
CN103218453A (en) * 2013-04-28 2013-07-24 南京龙渊微电子科技有限公司 Method and device for splitting file
CN103268252A (en) * 2013-05-12 2013-08-28 南京载玄信息科技有限公司 Virtualization platform system based on distributed storage and achieving method thereof
CN104219279A (en) * 2013-06-04 2014-12-17 国际商业机器公司 Modular architecture for extreme-scale distributed processing applications
US10248346B2 (en) 2013-06-04 2019-04-02 International Business Machines Corporation Modular architecture for extreme-scale distributed processing applications
CN104219279B (en) * 2013-06-04 2018-01-23 国际商业机器公司 System and method for the modularization framework of ultra-large distributed treatment application
CN103338252A (en) * 2013-06-27 2013-10-02 南京邮电大学 Distributed database concurrence storage virtual request mechanism
CN103338252B (en) * 2013-06-27 2017-05-24 南京邮电大学 Realizing method of distributed database concurrence storage virtual request mechanism
CN103365979A (en) * 2013-07-03 2013-10-23 交通银行股份有限公司 Long-distance double-center online processing method and system based on open database
CN104424292A (en) * 2013-09-02 2015-03-18 联想(北京)有限公司 Data processing method and electronic device
CN104424292B (en) * 2013-09-02 2020-04-24 联想(北京)有限公司 Data processing method and electronic equipment
CN103546572A (en) * 2013-10-30 2014-01-29 北京荣之联科技股份有限公司 Cloud storage device and multi-cloud storage networking system and method
CN103546572B (en) * 2013-10-30 2017-04-05 北京荣之联科技股份有限公司 A kind of cloudy storing networking system and method
CN103631894A (en) * 2013-11-19 2014-03-12 浪潮电子信息产业股份有限公司 Dynamic copy management method based on HDFS
CN104702634A (en) * 2013-12-04 2015-06-10 腾讯科技（深圳）有限公司 Method, device and system for processing data operation request
CN104702634B (en) * 2013-12-04 2018-10-02 腾讯科技（深圳）有限公司 Data operation request treatment method, apparatus and system
CN103618804A (en) * 2013-12-16 2014-03-05 北京航空航天大学 Performance difference-based load balancing method for distributed key value storage system
CN103744809A (en) * 2013-12-23 2014-04-23 天泽信息产业股份有限公司 Method for dual-computer hot-standby of vehicle information management system on basis of VRRP
CN103744809B (en) * 2013-12-23 2016-10-05 天泽信息产业股份有限公司 Vehicle information management system double hot standby method based on VRRP
CN103701916A (en) * 2013-12-31 2014-04-02 赛凡信息科技（厦门）有限公司 Dynamic load balancing method of distributed storage system
CN103701916B (en) * 2013-12-31 2017-10-27 赛凡信息科技（厦门）有限公司 The dynamic load balancing method of distributed memory system
CN104780192A (en) * 2014-01-15 2015-07-15 携程计算机技术（上海）有限公司 File synchronization method, single-point server and cluster system
CN104780192B (en) * 2014-01-15 2019-05-31 上海携程商务有限公司 File synchronisation method, single-point server and group system
CN103944958A (en) * 2014-03-14 2014-07-23 中国科学院计算技术研究所 Wide area file system and implementation method
CN104469391A (en) * 2014-11-21 2015-03-25 深圳市天威视讯股份有限公司 Cloud-platform-based digital television content distribution system and method
CN104469391B (en) * 2014-11-21 2018-02-27 深圳市天威视讯股份有限公司 A kind of digital TV contents dissemination system and method based on cloud platform
CN104391930A (en) * 2014-11-21 2015-03-04 用友软件股份有限公司 Distributed file storage device and method
CN105718484A (en) * 2014-12-04 2016-06-29 中兴通讯股份有限公司 File writing method, file reading method, file deletion method, file query method and client
CN106294496A (en) * 2015-06-09 2017-01-04 北京京东尚科信息技术有限公司 A kind of data migration method based on hadoop cluster and instrument
CN106294496B (en) * 2015-06-09 2020-02-07 北京京东尚科信息技术有限公司 Data migration method and tool based on hadoop cluster
CN105117166A (en) * 2015-07-30 2015-12-02 苏州绿豆豆软件科技有限公司 Cloud storage card apparatus based on FPGA, and realization method therefor
CN105117166B (en) * 2015-07-30 2017-12-05 苏州绿豆豆软件科技有限公司 A kind of cloud storage card-type device and its implementation based on FPGA
CN105227672A (en) * 2015-10-13 2016-01-06 国家电网公司 The method and system that data store and access
CN105187565A (en) * 2015-10-14 2015-12-23 四川携创信息技术服务有限公司 Method for utilizing network storage data
US10372673B2 (en) 2016-04-29 2019-08-06 Huawei Technologies Co., Ltd. Storage network element discovery method and apparatus
CN107852344A (en) * 2016-04-29 2018-03-27 华为技术有限公司 Store NE Discovery method and device
EP3261302A4 (en) * 2016-04-29 2018-03-21 Huawei Technologies Co., Ltd. Storage network element discovery method and device
CN105956491B (en) * 2016-05-19 2018-12-18 北京春鸿科技有限公司 The data of publicly-owned cloud storage system are carried out with the method and system of more account backups
CN105956491A (en) * 2016-05-19 2016-09-21 北京春鸿企业管理咨询有限公司 Method and system for performing multi-account backup on data of public cloud storage system
CN105871659A (en) * 2016-05-31 2016-08-17 乐视控股（北京）有限公司 Monitoring method and device of newly added server
CN107247714A (en) * 2016-06-01 2017-10-13 国家电网公司 A kind of small documents access system and method based on distributed storage technology
CN106156289A (en) * 2016-06-28 2016-11-23 北京百迈客云科技有限公司 The method of the data in a kind of read-write object storage system and device
CN106060173A (en) * 2016-07-22 2016-10-26 恒业智能信息技术（深圳）有限公司 Cluster type photographing data storage system based on cloud storage
CN106302757A (en) * 2016-08-19 2017-01-04 成都全码特时代科技有限公司 A kind of server cluster system based on cloud storage
CN106791908B (en) * 2016-11-25 2019-11-01 上海熙菱信息技术有限公司 A kind of live video stream storage method for supporting cloud platform to use double buffering
CN106791908A (en) * 2016-11-25 2017-05-31 上海熙菱信息技术有限公司 A kind of live video stream storage method for supporting cloud platform using double buffering
CN106790422A (en) * 2016-12-02 2017-05-31 北京锐安科技有限公司 A kind of data buffer storage cluster and data retrieval method for WEB application
CN108243206A (en) * 2016-12-23 2018-07-03 航天星图科技（北京）有限公司 A kind of server cluster area management method
CN108243206B (en) * 2016-12-23 2019-03-29 中科星图股份有限公司 A kind of server cluster area management method
CN106598503B (en) * 2016-12-23 2019-09-24 郑州云海信息技术有限公司 The method and apparatus that optimization virtual disk occupies memory space under a kind of cloud platform
CN106598503A (en) * 2016-12-23 2017-04-26 郑州云海信息技术有限公司 Method and device for optimizing storage space occupied by virtual disk under cloud platform
CN106792825A (en) * 2016-12-29 2017-05-31 河北远东通信系统工程有限公司 A kind of management method of distributed network equipment
CN106792825B (en) * 2016-12-29 2019-11-22 河北远东通信系统工程有限公司 A kind of management method of distributed network equipment
CN107229425B (en) * 2017-06-02 2020-05-19 浙江宇视科技有限公司 Data storage method and device
CN107229425A (en) * 2017-06-02 2017-10-03 浙江宇视科技有限公司 A kind of date storage method and device
WO2019001280A1 (en) * 2017-06-29 2019-01-03 中兴通讯股份有限公司 Heterogeneous virtual computing resource management method, related device, and storage medium
WO2019020084A1 (en) * 2017-07-27 2019-01-31 杭州海康威视数字技术股份有限公司 Data storage method and system
CN107205049A (en) * 2017-07-28 2017-09-26 郑州云海信息技术有限公司 A kind of network attached storage method and storage service apparatus
CN107205049B (en) * 2017-07-28 2020-06-16 苏州浪潮智能科技有限公司 Network attachment storage method and storage service device
CN107948248A (en) * 2017-11-01 2018-04-20 平安科技（深圳）有限公司 Distributed storage method, control server and computer-readable recording medium
CN107861690A (en) * 2017-11-22 2018-03-30 安徽伊尔思环境科技有限公司 A kind of big data optimizes project environmental impact assessment cloud storage system
CN108132815A (en) * 2017-12-25 2018-06-08 郑州云海信息技术有限公司 A kind of method, apparatus and storage medium of configuration file dynamic load
CN110300131B (en) * 2018-03-21 2022-02-15 北京金风科创风电设备有限公司 Routing method, device, equipment and system for multiple services of wind power plant
CN110300131A (en) * 2018-03-21 2019-10-01 北京金风科创风电设备有限公司 Routing method, device, equipment and system for multiple services of wind power plant
CN112236758A (en) * 2018-05-31 2021-01-15 微软技术许可有限责任公司 Cloud storage distributed file system
CN108809984A (en) * 2018-06-13 2018-11-13 苏州若依玫信息技术有限公司 A kind of cloud computing intelligent security system based on time domain
CN108809984B (en) * 2018-06-13 2020-09-08 广东奥飞数据科技股份有限公司 Time domain-based cloud computing intelligent security system
CN109359100A (en) * 2018-09-28 2019-02-19 北京赛博贝斯数据科技有限责任公司 The visual modeling method of PB grades of historical datas and online data calculated in real time
CN109495540A (en) * 2018-10-15 2019-03-19 深圳市金证科技股份有限公司 A kind of method, apparatus of data processing, terminal device and storage medium
CN109495540B (en) * 2018-10-15 2021-02-26 金证财富南京科技有限公司 Data processing method and device, terminal equipment and storage medium
CN111078120B (en) * 2018-10-18 2023-11-03 深信服科技股份有限公司 Data migration method and system of distributed file system and related components
CN111078120A (en) * 2018-10-18 2020-04-28 深信服科技股份有限公司 Data migration method and system of distributed file system and related components
CN109462642B (en) * 2018-10-30 2021-06-08 新华三技术有限公司成都分公司 Data processing method and device
CN109462642A (en) * 2018-10-30 2019-03-12 新华三技术有限公司成都分公司 Data processing method and device
CN109658284A (en) * 2018-12-19 2019-04-19 国网北京市电力公司 The appraisal procedure and system of equipment state
CN109885552A (en) * 2019-02-18 2019-06-14 天固信息安全系统(深圳)有限责任公司 The metadata dynamic management approach and distributed file system of distributed file system
CN109885552B (en) * 2019-02-18 2023-08-18 天固信息安全系统(深圳)有限责任公司 Metadata dynamic management method of distributed file system and distributed file system
CN110535829B (en) * 2019-07-24 2022-06-07 视联动力信息技术股份有限公司 Data interaction method and video networking system
CN110535829A (en) * 2019-07-24 2019-12-03 视联动力信息技术股份有限公司 A kind of data interactive method and a kind of view networked system
CN111581020A (en) * 2020-04-22 2020-08-25 上海天玑科技股份有限公司 Method and device for data recovery in distributed block storage system
CN111581020B (en) * 2020-04-22 2024-03-19 上海天玑科技股份有限公司 Method and device for recovering data in distributed block storage system
CN111625401A (en) * 2020-05-29 2020-09-04 浪潮电子信息产业股份有限公司 Data backup method and device based on cluster file system and readable storage medium
CN111625401B (en) * 2020-05-29 2023-03-21 浪潮电子信息产业股份有限公司 Data backup method and device based on cluster file system and readable storage medium
CN111767282A (en) * 2020-06-12 2020-10-13 咪咕文化科技有限公司 MongoDB-based storage system, data insertion method and storage medium
CN111901174B (en) * 2020-07-30 2024-04-26 腾讯科技（深圳）有限公司 Service state notification method, related device and storage medium
CN111901174A (en) * 2020-07-30 2020-11-06 腾讯科技（深圳）有限公司 Service state notification method, related device and storage medium
CN112068780A (en) * 2020-09-10 2020-12-11 国家卫星气象中心(国家空间天气监测预警中心) Data unified management system based on heterogeneous storage
CN112445770A (en) * 2020-11-30 2021-03-05 清远职业技术学院 Super-large-scale high-performance database engine with multi-dimensional out-of-order storage function and cloud service platform
CN112398871A (en) * 2020-12-01 2021-02-23 厦门市美亚柏科信息股份有限公司 Mass file transmission method and system
CN112398871B (en) * 2020-12-01 2023-03-24 厦门市美亚柏科信息股份有限公司 Mass file transmission method and system
CN112579543A (en) * 2020-12-29 2021-03-30 航天科工网络信息发展有限公司 Dynamic metadata management method for distributed file system and distributed file system
CN113032099A (en) * 2021-03-26 2021-06-25 阿里巴巴新加坡控股有限公司 Cloud computing node, file management method and device
CN113032099B (en) * 2021-03-26 2023-10-03 阿里巴巴新加坡控股有限公司 Cloud computing node, file management method and device
CN113032356A (en) * 2021-03-31 2021-06-25 中电科航空电子有限公司 Cabin distributed file storage system and implementation method
CN114328007B (en) * 2021-11-19 2024-03-22 苏州浪潮智能科技有限公司 Container backup and restoration method, device and medium thereof
CN114328007A (en) * 2021-11-19 2022-04-12 苏州浪潮智能科技有限公司 Container backup and restoration method, device and medium thereof
CN114756509A (en) * 2022-05-19 2022-07-15 北京百度网讯科技有限公司 Operation method, system, device and storage medium of file system

Also Published As

Publication number Publication date
CN102143215B (en) 2013-04-10

Similar Documents

Publication Publication Date Title
CN102143215B (en) Network-based PB level cloud storage system and processing method thereof
CN109740037B (en) Multi-source heterogeneous flow state big data distributed online real-time processing method and system
CN107547653B (en) Distributed file storage system
Lakshman et al. Cassandra: a decentralized structured storage system
CN106066896B (en) Application-aware big data deduplication storage system and method
EP2784675B1 (en) Method, device and system for data reconstruction
WO2018059032A1 (en) Data migration method for virtual node, and virtual node
CN105320773B (en) A kind of distributed data deduplication system and method based on Hadoop platform
US20200174671A1 (en) Bucket views
Luo et al. A RAMCloud storage system based on HDFS: Architecture, implementation and evaluation
US10929047B2 (en) Storage system with snapshot generation and/or preservation control responsive to monitored replication data
Malensek et al. Galileo: A framework for distributed storage of high-throughput data streams
Noghabi et al. Ambry: Linkedin's scalable geo-distributed object store
CN102420854A (en) Distributed file system facing to cloud storage
CN102880658A (en) Distributed file management system based on seismic data processing
Sun et al. A novel approach to data deduplication over the engineering-oriented cloud systems
CN106775446A (en) Based on the distributed file system small documents access method that solid state hard disc accelerates
Zhang et al. Survey of research on big data storage
CN106648464A (en) Multi-node mixed block cache data read-writing method and system based on cloud storage
CN103491155A (en) Cloud computing method and system for achieving mobile computing and obtaining mobile data
CN105138275B (en) A kind of Lustre memory system datas sharing method
CN102609446A (en) Distributed Bloom filter system and application method thereof
CN103501319A (en) Low-delay distributed storage system for small files
Chang et al. Scalable network file systems with load balancing and fault tolerance for web services
CN104899161A (en) Cache method based on continuous data protection of cloud storage environment

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
C17 Cessation of patent right
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20130410

Termination date: 20140120