CN101944124A - Distributed file system management method, device and corresponding file system - Google Patents
Distributed file system management method, device and corresponding file system Download PDFInfo
- Publication number
- CN101944124A CN101944124A CN 201010290706 CN201010290706A CN101944124A CN 101944124 A CN101944124 A CN 101944124A CN 201010290706 CN201010290706 CN 201010290706 CN 201010290706 A CN201010290706 A CN 201010290706A CN 101944124 A CN101944124 A CN 101944124A
- Authority
- CN
- China
- Prior art keywords
- file
- file system
- distributed
- storage request
- destination
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The embodiment of the invention discloses a distributed file system management method, a device and a corresponding file system. The distributed file system management method comprises the following steps of: receiving a file storage request sent by a client; searching a target distributed file system corresponding to a target file according to preset classification rule information, wherein the target file is a file requested by the file storage request; sending the file storage request to the target distributed file system according to a preset file storage protocol; and responding the file storage request of the client. The invention can solve the problem that the number of files stored by the distributed file system is restricted by the memory size of a metadata server, and ensure that when the memory of the metadata server can not be increased any more, the distributed file system can still be expanded through the mode of adding the metadata server. Meanwhile, the invention has the advantages of good expandability, easy realization and low cost.
Description
Technical field
The present invention relates to the computing machine electronic applications, relate in particular to a kind of distributed file system management method, device and corresponding file system.
Background technology
Distributed file system (DFS, Distributed File System) is meant that the physical store resource of file system management not necessarily directly is connected on the local node, but links to each other with node by computer network.As its name suggests, be exactly distributed+file system.It comprises the intension of these two aspects, and from the angle that the client of file system uses, it is exactly the file system of a standard, and a series of API are provided, carry out thus file or catalogue establishment, move, deletion, and to the operations such as read-write of file.Realize internally, the then no longer the same local disk of being in charge of of distributed systems with the ordinary file system, its file content and bibliographic structure are not to be stored on the local disk, but by Network Transmission to far end system.And same file storage is just on a machine, but on the cluster machine distributed storage, collaboratively provide service.
The inventor finds that there is significant disadvantages in present distributed file system in implementing process of the present invention:
Large-scale website/system is along with function increases, and also can and then increase the demand of file storage, if all files all are stored under the distributed file system, will certainly cause the confusion of file management; The type of various files and function have nothing in common with each other to the demand of distributed system in addition, the storage small documents is arranged, and the big file of storage is arranged, and can't be optimized distributed file system according to functional characteristic if all put together.
In addition, the meta data server of the distributed file system of main flow all is that metadata is placed in the internal memory at present, so the quantity of documents that distributed file system can be preserved is limited by the memory size of meta data server, current hardware advances speed is far away not as good as the growth rate of file to be stored.So the internal memory of metadata of distributed type file system server has become the bottleneck of restriction distributed file system development.
Summary of the invention
Embodiment of the invention technical matters to be solved is, a kind of distributed file system management method, device and corresponding file system are provided, the quantity that can solve present distributed file system storage file is subjected to the restriction of meta data server memory size, when meta data server can't increase internal memory again, still can expand distributed file system by the mode that increases meta data server again.The present invention simultaneously has fine extensibility, implements fairly simplely, and cost is low.
In order to solve the problems of the technologies described above, the embodiment of the invention provides a kind of distributed file system management method, comprising:
Receive the file storage request that client sends;
According to the classifying rules information that presets, search the target distribution formula file system of file destination correspondence; Described file destination is the file that described file storage request is asked;
According to the file storage agreement that presets, described file storage request is sent to described target distribution formula file system; Respond the file storage request of described client.
Wherein, before the file storage request that described reception client sends, also comprise:
Preset file storage agreement, classifying rules information; Described classifying rules information comprises: the sorting technique that file is classified or the corresponding relation of algorithm, Miscellaneous Documents and each distributed file system.
Wherein, the classifying rules information that described basis presets is searched the target distribution formula file system of file destination correspondence, comprising:
According to sorting technique that file is classified or algorithm, determine the classification that described file destination is affiliated;
According to the classification under the described file destination, search described file destination corresponding distributed file system;
With the distributed file system that finds target distribution formula file system as described file destination.
Wherein, described according to the file storage agreement that presets, described file storage request is sent to described target distribution formula file system; Respond the file storage request of described client, comprising:
According to the file storage agreement that presets, described file storage request is sent to described target distribution formula file system;
Target distribution formula file system is carried out corresponding read/write operation according to described file storage request to file destination;
After described target distribution formula file system is finished operation to file destination, respond described client.
Correspondingly, the embodiment of the invention also provides a kind of file system, comprising: distributed file system management devices, client and at least two distributed file systems;
Described client is used for sending corresponding file storage request according to user's demand to described distributed file system management devices;
Described distributed file system management devices is used to receive the file storage request that client sends; According to the classifying rules information that presets, search the target distribution formula file system of file destination correspondence; And, described file storage request is sent to corresponding distributed file system according to the file storage agreement that presets; And respond the file storage request of described client; Described file destination is the file that described file storage request is asked;
Described distributed file system is used to receive the file storage request that described distributed file system management devices sends, and according to described file storage request file destination is carried out corresponding read/write operation.
Wherein, described distributed file system comprises meta data server and storage server;
Described meta data server is used for storing the metadata of the file of described storage server, and the file in the described storage server is carried out read/write operation; The metadata of described file comprises: the memory address of file, size, type;
Described storage server is used for storage file, and accepts described meta data server the file of its storage is carried out read/write operation.
Correspondingly, the embodiment of the invention also provides a kind of distributed file system management devices, comprising:
The client-side interface module is used to receive the file storage request that client sends; And the described file storage request that responds described client;
Goal systems is searched module, is used for searching the target distribution formula file system of file destination correspondence according to the classifying rules information that presets; Described file destination is the file that file storage request that described client-side interface module is received is asked;
The storage request processing module is used for according to the file storage agreement that presets, and described file storage request is sent to described goal systems searches the determined target distribution formula of module file system.
Wherein, described system also comprises:
The protocol information preset module is used to preset file storage agreement, classifying rules information; Described classifying rules information comprises: the sorting technique that file is classified or the corresponding relation of algorithm, Miscellaneous Documents and each distributed file system.
Wherein, described goal systems is searched module and is comprised:
Taxon is used for sorting technique or algorithm according to described protocol information preset module, determines the classification that described file destination is affiliated;
Search the unit, described file destination corresponding distributed file system is searched in the classification under the file destination that is used for determining according to described taxon; And with the distributed file system that finds target distribution formula file system as described file destination.
Wherein, described storage request processing module comprises:
Retransmission unit, be used for the file storage agreement that presets according to the protocol information preset module, described file storage request is sent to described target distribution formula file system, makes described target distribution formula file system, file destination is carried out corresponding read/write operation according to described file storage request;
Processing unit is used for responding described client after described target distribution formula file system is finished operation to file destination.
Implement distributed file system management method provided by the invention, device and corresponding file system, the quantity that can solve present distributed file system storage file is subjected to the restriction of meta data server memory size, when meta data server can't increase internal memory again, still can expand distributed file system by the mode that increases meta data server again.The present invention simultaneously has fine extensibility, implements fairly simplely, and cost is low.
Description of drawings
In order to be illustrated more clearly in the embodiment of the invention or technical scheme of the prior art, to do to introduce simply to the accompanying drawing of required use in embodiment or the description of the Prior Art below, apparently, accompanying drawing in describing below only is some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain other accompanying drawing according to these accompanying drawings.
Fig. 1 is the distributed file system management method first embodiment schematic flow sheet provided by the invention;
Fig. 2 is the distributed file system management method second embodiment schematic flow sheet provided by the invention;
Fig. 3 is a file system example structure synoptic diagram provided by the invention;
Fig. 4 is the distributed file system management devices first example structure synoptic diagram provided by the invention;
Fig. 5 is the distributed file system management devices second example structure synoptic diagram provided by the invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the invention, the technical scheme in the embodiment of the invention is clearly and completely described, obviously, described embodiment only is the present invention's part embodiment, rather than whole embodiment.Based on the embodiment among the present invention, those of ordinary skills belong to the scope of protection of the invention not making the every other embodiment that is obtained under the creative work prerequisite.
Referring to Fig. 1, be the distributed file system management method first embodiment schematic flow sheet provided by the invention, as shown in Figure 1:
At step S100, receive the file storage request that client sends.
At step S101,, search the target distribution formula file system of file destination correspondence according to the classifying rules information that presets; Described file destination is the file that described file storage request is asked.
At step S102,, described file storage request is sent to described target distribution formula file system according to the file storage agreement that presets.
At step S103, respond the file storage request of described client.
Implement the distributed file system management method that the embodiment of the invention provides, the quantity that can solve present distributed file system storage file is subjected to the restriction of meta data server memory size, when meta data server can't increase internal memory again, still can expand distributed file system by the mode that increases meta data server again.The present invention simultaneously has fine extensibility, implements fairly simplely, and cost is low.
Referring to Fig. 2, be the distributed file system management method second embodiment schematic flow sheet provided by the invention, in the present embodiment, with the flow process of this distributed file system management method of more detailed description, as shown in Figure 2:
At step S200, preset file storage agreement, classifying rules information; Described classifying rules information comprises: the sorting technique that file is classified or the corresponding relation of algorithm, Miscellaneous Documents and each distributed file system.The embodiment of the invention provides two kinds of sorting techniques based on vertical and horizontal.Vertical classification is according to function, with the file separate storage of difference in functionality in different distributed file systems, thereby improve the extensibility of system; Horizontal classification is according to certain algorithm (for example multiple algorithm such as user ID segmentation algorithm, hash algorithm) All Files to be stored in a plurality of distributed file systems.
More specifically, the concrete form of classifying rules information can be as follows:
<？xml?version＝″1.0″encoding＝″UTF-8″？>
<storage:rule>
＜!--this example of the configuration rule of a certain application is only to have done vertical classification--〉
<fileSystemRule?name＝″Tmessage″defaultPartition＝″mfs1″/>
＜!--the classifying rules of acquiescence, just use this classifying rules when not specifying when using, be equivalent to the classifying rules of the overall situation, this example is a horizontal classification, so owing to be that overall rule must be provided with defaultRule=true--〉
<fileSystemRule?name＝″default″defaultRule＝″true″>
<rule?name＝″rule1″>
<parameters>uid</parameters>
<expression><！[CDATA[abs(uid)between?1000and?2000]]></expression>
<partition>mfsl</partition>
</rule>
<rule?name＝″rule2″>
<parameters>uid</parameters>
<expression><！[CDATA[abs(uid)between?2001and?3000]]></expression>
<partition>mfs2</partition>
</rule>
</fileSystemRule>
</storage:rule>
Need to prove that the above-mentioned step that presets file storage agreement, classifying rules information is not must needing only and preset file storage agreement, classifying rules information when first enforcement through step of present embodiment.Certainly, in follow-up use, can upgrade and adjust according to actual conditions.
At step S201, the distributed file system management devices receives the file storage request that client sends, and described file storage request comprises: written document request and read file request.
At step S202,, determine the classification that described file destination is affiliated according to sorting technique that file is classified or algorithm.In the present embodiment, be example with the user ID segmentation algorithm, suppose 5 distributed file systems of distributed file system management devices management, its segmentation algorithm is as shown in table 1:
Table 1
| Distributed file system | The ID segmentation |
| Distributed |
1——100000 |
| Distributed file system B | 100001——200000 |
| Distributed file system C | 200001——300000 |
| Distributed file system D | 300001——400000 |
| Distributed file system E | 400001——500000 |
As shown in Table 1,5 distributed file systems of distributed file system management devices management, ID is 1---the file that 100000 client is stored, all in distributed file system A; ID is 100001---the file that 200000 client is stored, all in distributed file system B; Other distributed file system in like manner.For example, client sends a file storage request to the distributed file system management devices, and after the distributed file system management devices received, getting access to its ID was 300570, then, determine that the ID under it is segmented into 300001 according to segmentation algorithm as shown in table 1---400000.
At step S203,, search described file destination corresponding distributed file system according to the classification under the described file destination.More specifically, for example the distributed file system management devices determines that the ID under the described file destination is segmented into 300001---400000, then the distributed file system management devices determines that according to the Miscellaneous Documents as shown in table 1 and the corresponding relation of each distributed file system file destination corresponding distributed file system is distributed file system D.
At step S204, with the distributed file system that finds target distribution formula file system as described file destination.More specifically, present embodiment for example in, the distributed file system management devices will determined file destination corresponding distributed file system be distributed file system D in step S203, as the target distribution formula file system of described file destination.
At step S205,, described file storage request is sent to described target distribution formula file system according to the file storage agreement that presets.More specifically, present embodiment for example in, the distributed file system management devices is forwarded to distributed file system D with the file storage request that client sends.
At step S206, target distribution formula file system is carried out corresponding read/write operation according to described file storage request to file destination.More specifically, the meta data server of target distribution formula file system receives the file storage request, and file destination is carried out corresponding read/write operation.For example, the file storage request is that request writes a document, and then the meta data server of target distribution formula file system distributes corresponding storage server disk zone for it, stores this current file, and the metadata of record this document.
Further, alleged in the present invention meta data server is the metadata that is used for storing the file of described storage server, and the server that the file in the described storage server is carried out read/write operation; The metadata of described file comprises: the memory address of file, size, type.
At step S207, after described target distribution formula file system is finished operation to file destination, respond described client; Be the distributed file system management devices after described target distribution formula file system is finished operation to file destination, the client that sends described file storage request responded that this response mainly is the interface after the client display operation is finished.
Implement the distributed file system management method that the embodiment of the invention provides, the quantity that can solve present distributed file system storage file is subjected to the restriction of meta data server memory size, when meta data server can't increase internal memory again, still can expand distributed file system by the mode that increases meta data server again.The present invention simultaneously has fine extensibility, implements fairly simplely, and cost is low.
Referring to Fig. 3, be file system example structure synoptic diagram provided by the invention, as shown in the figure, this system comprises:
Distributed file system management devices 1, client 2 and at least two distributed file systems 3;
Client 2 is used for sending corresponding file storage request according to user's demand to distributed file system management devices 1.
Distributed file system management devices 1 is used to receive the file storage request that client 2 sends; According to the classifying rules information that presets, search the target distribution formula file system of file destination correspondence; And, the file storage request is sent to corresponding distributed file system according to the file storage agreement that presets; And the file storage request of customer in response end 2; File destination is the file that the file storage request is asked.
Preferably, in various embodiments of the present invention, described distributed file system management devices is the system of a component level, is to be embedded in the third-party system, and performance can be better like this, also the Single Point of Faliure problem can not occur.
Distributed file system 3 is used to receive the file storage request that distributed file system management devices 1 sends, and according to the file storage request file destination is carried out corresponding read/write operation.
More specifically, distributed file system 3 comprises meta data server 31 and storage server 32;
In the prior art, because there is not the distributed file system management devices, client directly connects the meta data server of distributed file system, so and each distributed file system can only independently operate, the file of its storage is also lacked effective management and distribution.The file system that present embodiment provides, can integrate a plurality of distributed file systems, by the unified management of distributed file system management devices, the quantity that solves present distributed file system storage file is subjected to the restriction of meta data server memory size, when meta data server can't increase internal memory again, still can expand distributed file system by the mode that increases meta data server again.The present invention simultaneously has fine extensibility, implements fairly simplely, and cost is low.
Referring to Fig. 4, be the distributed file system management devices first example structure synoptic diagram provided by the invention, as shown in Figure 4, this distributed file system management devices comprises:
Client-side interface module 12 is used to receive the file storage request that client sends; And the file storage request of customer in response end.
Goal systems is searched module 13, is used for searching the target distribution formula file system of file destination correspondence according to the classifying rules information that presets; File destination is the file that file storage request that client-side interface module 12 is received is asked.
Storage request processing module 14 is used for according to the file storage agreement that presets, and the file storage request is sent to goal systems searches module 13 determined target distribution formula file system.
The distributed file system management devices that present embodiment provides, can integrate a plurality of distributed file systems, by the unified management of distributed file system management devices, the quantity that solves present distributed file system storage file is subjected to the restriction of meta data server memory size, when meta data server can't increase internal memory again, still can expand distributed file system by the mode that increases meta data server again.The present invention simultaneously has fine extensibility, implements fairly simplely, and cost is low.
Referring to Fig. 5, be the distributed file system management devices second example structure synoptic diagram provided by the invention, in the present embodiment, with the 26S Proteasome Structure and Function of this distributed file system management devices of more detailed description.As shown in Figure 5, this distributed file system management devices comprises:
Protocol information preset module 11 is used to preset file storage agreement, classifying rules information; Described classifying rules information comprises: the sorting technique that file is classified or the corresponding relation of algorithm, Miscellaneous Documents and each distributed file system.More specifically:
The embodiment of the invention provides two kinds of sorting techniques based on vertical and horizontal.Vertical classification is according to function, with the file separate storage of difference in functionality in different distributed file systems, thereby improve the extensibility of system; Horizontal classification is according to certain algorithm (for example multiple algorithm such as user ID segmentation algorithm, hash algorithm) All Files to be stored in a plurality of distributed file systems.
More specifically, the concrete form of classifying rules information can be as follows:
<？xml?version＝″1.0″encoding＝″UTF-8″？>
<storage:rule>
＜!--this example of the configuration rule of a certain application is only to have done vertical classification--〉
<fileSystemRule?name＝″Tmessage″defaultPartition＝″mfs1″/>
＜!--the classifying rules of acquiescence, just use this classifying rules when not specifying when using, be equivalent to the classifying rules of the overall situation, this example is a horizontal classification, so owing to be that overall rule must be provided with defaultRule=true--〉
<fileSystemRule?name＝″default″defaultRule＝″true″>
<rule?name＝″rule1″>
<parameters>uid</parameters>
<expression><！[CDATA[abs(uid)between?1000and?2000]]></expression>
<partition>mfs1</partition>
</rule>
<rule?name＝″rule2″>
<parameters>uid</parameters>
<expression><！[CDATA[abs(uid)between?2001and?3000]]></expression>
<partition>mfs2</partition>
</rule>
</fileSystemRule>
</storage:rule>
Need to prove that the file storage agreement that presets in the protocol information preset module 11, classifying rules information in the follow-up use of this distributed file system management devices, can be upgraded and adjust according to actual conditions.
Client-side interface module 12 is used to receive the file storage request that client sends; And the file storage request of customer in response end.
Goal systems is searched module 13, is used for searching the target distribution formula file system of file destination correspondence according to the classifying rules information that presets; File destination is the file that file storage request that client-side interface module 12 is received is asked.More specifically, this goal systems is searched module 13 and is comprised:
Taxon 131 is used for sorting technique or algorithm according to described protocol information preset module, determines the classification that described file destination is affiliated.In the present embodiment, be example with the user ID segmentation algorithm, suppose 5 distributed file systems of this distributed apparatus for file system management management, it is as shown in table 1 to preset segmentation algorithm in the protocol information preset module 11.As shown in Table 1, ID is 1---the file that 100000 client is stored, all in distributed file system A; ID is 100001---the file that 200000 client is stored, all in distributed file system B; Other distributed file system in like manner.For example, client sends a file storage request to the distributed file system management devices, and after client-side interface module 12 received, getting access to its ID was 300570, then taxon 131 determines that according to segmentation algorithm as shown in table 1 its described ID is segmented into 300001---400000.
Search unit 132, described file destination corresponding distributed file system is searched in the classification under the file destination that is used for determining according to described taxon; And with the distributed file system that finds target distribution formula file system as described file destination.More specifically, for example taxon 131 determines that the ID under the described file destination is segmented into 300001---400000, then search unit 132 according to the Miscellaneous Documents as shown in table 1 and the corresponding relation of each distributed file system, determine that the target distribution formula file system of file destination correspondence is distributed file system D.
Storage request processing module 14 is used for according to the file storage agreement that presets, and the file storage request is sent to goal systems searches module 13 determined target distribution formula file system.More specifically, storage request processing module 14 comprises:
Retransmission unit 141, be used for the file storage agreement that presets according to protocol information preset module 11, described file storage request is sent to described target distribution formula file system, make described target distribution formula file system according to described file storage request, file destination is carried out corresponding read/write operation.More specifically, present embodiment for example in, retransmission unit 141 is forwarded to distributed file system D with the file storage request that client sends.The meta data server of target distribution formula file system receives the file storage request, and file destination is carried out corresponding read/write operation.For example, the file storage request is that request writes a document, and then the meta data server of target distribution formula file system distributes corresponding storage server disk zone for it, stores this current file, and the metadata of record this document.
Further, alleged in the present invention meta data server is the metadata that is used for storing the file of described storage server, and the server that the file in the described storage server is carried out read/write operation; The metadata of described file comprises: the memory address of file, size, type.
Processing unit 142 is used for responding described client after described target distribution formula file system is finished operation to file destination.Be processing unit 142 after described target distribution formula file system is finished operation to file destination, the client that sends described file storage request responded that this response mainly is the interface after the client display operation is finished.
The distributed file system management devices that present embodiment provides, can integrate a plurality of distributed file systems, by the unified management of distributed file system management devices, the quantity that solves present distributed file system storage file is subjected to the restriction of meta data server memory size, when meta data server can't increase internal memory again, still can expand distributed file system by the mode that increases meta data server again.The present invention simultaneously has fine extensibility, implements fairly simplely, and cost is low.
One of ordinary skill in the art will appreciate that all or part of flow process that realizes in the foregoing description method, be to instruct relevant hardware to finish by computer program, program can be stored in the computer read/write memory medium, this program can comprise the flow process as the embodiment of above-mentioned each side method when carrying out.Wherein, storage medium can be magnetic disc, CD, read-only storage memory body (Read-Only Memory, ROM) or at random store memory body (Random Access Memory, RAM) etc.
Above disclosed is a kind of preferred embodiment of the present invention only, can not limit the present invention's interest field certainly with this, and therefore the equivalent variations of doing according to claim of the present invention still belongs to the scope that the present invention is contained.
Claims (10)
1. a distributed file system management method is characterized in that, comprising:
Receive the file storage request that client sends;
According to the classifying rules information that presets, search the target distribution formula file system of file destination correspondence; Described file destination is the file that described file storage request is asked;
According to the file storage agreement that presets, described file storage request is sent to described target distribution formula file system; Respond the file storage request of described client.
2. distributed file system management method as claimed in claim 1 is characterized in that, before the file storage request that described reception client sends, also comprises:
Preset file storage agreement, classifying rules information; Described classifying rules information comprises: the sorting technique that file is classified or the corresponding relation of algorithm, Miscellaneous Documents and each distributed file system.
3. distributed file system management method as claimed in claim 1 is characterized in that, the classifying rules information that described basis presets is searched the target distribution formula file system of file destination correspondence, comprising:
According to sorting technique that file is classified or algorithm, determine the classification that described file destination is affiliated;
According to the classification under the described file destination, search described file destination corresponding distributed file system;
With the distributed file system that finds target distribution formula file system as described file destination.
4. distributed file system management method as claimed in claim 1 is characterized in that, and is described according to the file storage agreement that presets, and described file storage request is sent to described target distribution formula file system; Respond the file storage request of described client, comprising:
According to the file storage agreement that presets, described file storage request is sent to described target distribution formula file system;
Target distribution formula file system is carried out corresponding read/write operation according to described file storage request to file destination;
After described target distribution formula file system is finished operation to file destination, respond described client.
5. a file system is characterized in that, comprising: distributed file system management devices, client and at least two distributed file systems;
Described client is used for sending corresponding file storage request according to user's demand to described distributed file system management devices;
Described distributed file system management devices is used to receive the file storage request that client sends; According to the classifying rules information that presets, search the target distribution formula file system of file destination correspondence; And, described file storage request is sent to corresponding distributed file system according to the file storage agreement that presets; And respond the file storage request of described client; Described file destination is the file that described file storage request is asked;
Described distributed file system is used to receive the file storage request that described distributed file system management devices sends, and according to described file storage request file destination is carried out corresponding read/write operation.
6. file system as claimed in claim 5 is characterized in that, comprising: described distributed file system comprises meta data server and storage server;
Described meta data server is used for storing the metadata of the file of described storage server, and the file in the described storage server is carried out read/write operation; The metadata of described file comprises: the memory address of file, size, type;
Described storage server is used for storage file, and accepts described meta data server the file of its storage is carried out read/write operation.
7. a distributed file system management devices is characterized in that, comprising:
The client-side interface module is used to receive the file storage request that client sends; And the described file storage request that responds described client;
Goal systems is searched module, is used for searching the target distribution formula file system of file destination correspondence according to the classifying rules information that presets; Described file destination is the file that file storage request that described client-side interface module is received is asked;
The storage request processing module is used for according to the file storage agreement that presets, and described file storage request is sent to described goal systems searches the determined target distribution formula of module file system.
8. distributed file system management devices as claimed in claim 7 is characterized in that, described system also comprises:
The protocol information preset module is used to preset file storage agreement, classifying rules information; Described classifying rules information comprises: the sorting technique that file is classified or the corresponding relation of algorithm, Miscellaneous Documents and each distributed file system.
9. distributed file system management devices as claimed in claim 7 is characterized in that, described goal systems is searched module and comprised:
Taxon is used for sorting technique or algorithm according to described protocol information preset module, determines the classification that described file destination is affiliated;
Search the unit, described file destination corresponding distributed file system is searched in the classification under the file destination that is used for determining according to described taxon; And with the distributed file system that finds target distribution formula file system as described file destination.
10. distributed file system management devices as claimed in claim 7 is characterized in that, described storage request processing module comprises:
Retransmission unit, be used for the file storage agreement that presets according to the protocol information preset module, described file storage request is sent to described target distribution formula file system, makes described target distribution formula file system, file destination is carried out corresponding read/write operation according to described file storage request;
Processing unit is used for responding described client after described target distribution formula file system is finished operation to file destination.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010290706 CN101944124B (en) | 2010-09-21 | 2010-09-21 | Distributed file system management method, device and corresponding file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010290706 CN101944124B (en) | 2010-09-21 | 2010-09-21 | Distributed file system management method, device and corresponding file system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101944124A true CN101944124A (en) | 2011-01-12 |
| CN101944124B CN101944124B (en) | 2012-07-04 |
Family
ID=43436115
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201010290706 Expired - Fee Related CN101944124B (en) | 2010-09-21 | 2010-09-21 | Distributed file system management method, device and corresponding file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101944124B (en) |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102082830A (en) * | 2011-01-18 | 2011-06-01 | 浙江大学 | Unstable network-oriented distributed file storage method based on quality perception |
| CN102147809A (en) * | 2011-03-22 | 2011-08-10 | 曙光信息产业股份有限公司 | Parallel file system and management method thereof |
| CN102306151A (en) * | 2011-06-28 | 2012-01-04 | 携程旅游网络技术（上海）有限公司 | Distributed fax receiving method, system and corresponding equipment |
| CN102938784A (en) * | 2012-11-06 | 2013-02-20 | 无锡江南计算技术研究所 | Method and system used for data storage and used in distributed storage system |
| CN103037004A (en) * | 2012-12-21 | 2013-04-10 | 曙光信息产业(北京)有限公司 | Implement method and device of cloud storage system operation |
| CN103067419A (en) * | 2011-10-19 | 2013-04-24 | 北大方正集团有限公司 | Distributed type file system and method of controlling file storage in distributed type file system |
| CN103095758A (en) * | 2011-11-01 | 2013-05-08 | 北大方正集团有限公司 | Distributed file system and file data processing method thereof |
| CN104090891A (en) * | 2013-12-12 | 2014-10-08 | 深圳市腾讯计算机系统有限公司 | Method and device for data processing and server and system for data processing |
| CN104468482A (en) * | 2013-09-18 | 2015-03-25 | 华为技术有限公司 | Document sorting method and document sorting device |
| CN105912612A (en) * | 2016-04-06 | 2016-08-31 | 中广天择传媒股份有限公司 | Distributed file system and data equilibrium distribution method orienting same |
| CN106021506A (en) * | 2016-05-23 | 2016-10-12 | 深圳市中博科创信息技术有限公司 | File storage method and apparatus for cluster system |
| CN106557509A (en) * | 2015-09-29 | 2017-04-05 | 镇江雅迅软件有限责任公司 | A kind of distributed file system |
| CN106899693A (en) * | 2017-03-20 | 2017-06-27 | 聚好看科技股份有限公司 | Distributed file management method and apparatus |
| CN107844542A (en) * | 2017-10-26 | 2018-03-27 | 山东浪潮通软信息科技有限公司 | A kind of distributed document storage method and device |
| CN108306946A (en) * | 2012-10-08 | 2018-07-20 | P·孙雄 | Distributed memory system and method |
| CN109542910A (en) * | 2018-12-03 | 2019-03-29 | 中国电波传播研究所（中国电子科技集团公司第二十二研究所） | A kind of magnanimity sea clutter data classification storage method |
| CN110765075A (en) * | 2019-10-24 | 2020-02-07 | 北京百度网讯科技有限公司 | Storage method and equipment of automatic driving data |
| CN111095233A (en) * | 2017-09-28 | 2020-05-01 | 深圳清华大学研究院 | Hybrid file system architecture, file storage, dynamic migration and applications thereof |
| CN112988062A (en) * | 2021-01-28 | 2021-06-18 | 腾讯科技（深圳）有限公司 | Metadata reading limiting method and device, electronic equipment and medium |
| CN113032405A (en) * | 2021-05-25 | 2021-06-25 | 湖南省第二测绘院 | Spatio-temporal data management method, system, host and computer readable storage medium |
| CN114500485A (en) * | 2022-01-28 | 2022-05-13 | 北京沃东天骏信息技术有限公司 | Data processing method and device |
| CN115834713A (en) * | 2023-02-07 | 2023-03-21 | 北京大道云行科技有限公司 | Interaction method and system for network file system and distributed file system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060064536A1 (en) * | 2004-07-21 | 2006-03-23 | Tinker Jeffrey L | Distributed storage architecture based on block map caching and VFS stackable file system modules |
| WO2006115594A2 (en) * | 2005-04-20 | 2006-11-02 | Microsoft Corporation | Systems and methods for providing distributed, decentralized data storage and retrieval |
| CN101446976A (en) * | 2008-12-26 | 2009-06-03 | 中兴通讯股份有限公司 | File storage method in distributed file system |
| CN101587476A (en) * | 2008-05-22 | 2009-11-25 | 中兴通讯股份有限公司 | Memory database system for realizing distributed storage and application thereof |
-
2010
- 2010-09-21 CN CN 201010290706 patent/CN101944124B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060064536A1 (en) * | 2004-07-21 | 2006-03-23 | Tinker Jeffrey L | Distributed storage architecture based on block map caching and VFS stackable file system modules |
| WO2006115594A2 (en) * | 2005-04-20 | 2006-11-02 | Microsoft Corporation | Systems and methods for providing distributed, decentralized data storage and retrieval |
| CN101587476A (en) * | 2008-05-22 | 2009-11-25 | 中兴通讯股份有限公司 | Memory database system for realizing distributed storage and application thereof |
| CN101446976A (en) * | 2008-12-26 | 2009-06-03 | 中兴通讯股份有限公司 | File storage method in distributed file system |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102082830A (en) * | 2011-01-18 | 2011-06-01 | 浙江大学 | Unstable network-oriented distributed file storage method based on quality perception |
| CN102147809A (en) * | 2011-03-22 | 2011-08-10 | 曙光信息产业股份有限公司 | Parallel file system and management method thereof |
| CN102306151A (en) * | 2011-06-28 | 2012-01-04 | 携程旅游网络技术（上海）有限公司 | Distributed fax receiving method, system and corresponding equipment |
| CN103067419A (en) * | 2011-10-19 | 2013-04-24 | 北大方正集团有限公司 | Distributed type file system and method of controlling file storage in distributed type file system |
| CN103067419B (en) * | 2011-10-19 | 2016-08-24 | 新奥特（北京）视频技术有限公司 | A kind of distributed file system and the method controlling file storage within the system |
| CN103095758A (en) * | 2011-11-01 | 2013-05-08 | 北大方正集团有限公司 | Distributed file system and file data processing method thereof |
| CN108306946A (en) * | 2012-10-08 | 2018-07-20 | P·孙雄 | Distributed memory system and method |
| CN108306946B (en) * | 2012-10-08 | 2021-03-16 | P·孙雄 | Distributed storage system and method |
| CN102938784A (en) * | 2012-11-06 | 2013-02-20 | 无锡江南计算技术研究所 | Method and system used for data storage and used in distributed storage system |
| CN103037004A (en) * | 2012-12-21 | 2013-04-10 | 曙光信息产业(北京)有限公司 | Implement method and device of cloud storage system operation |
| CN104468482A (en) * | 2013-09-18 | 2015-03-25 | 华为技术有限公司 | Document sorting method and document sorting device |
| CN104090891A (en) * | 2013-12-12 | 2014-10-08 | 深圳市腾讯计算机系统有限公司 | Method and device for data processing and server and system for data processing |
| CN104090891B (en) * | 2013-12-12 | 2016-05-04 | 深圳市腾讯计算机系统有限公司 | Data processing method, Apparatus and system |
| CN106557509A (en) * | 2015-09-29 | 2017-04-05 | 镇江雅迅软件有限责任公司 | A kind of distributed file system |
| CN105912612B (en) * | 2016-04-06 | 2019-04-05 | 中广天择传媒股份有限公司 | A kind of distributed file system and the data balancing location mode towards the system |
| CN105912612A (en) * | 2016-04-06 | 2016-08-31 | 中广天择传媒股份有限公司 | Distributed file system and data equilibrium distribution method orienting same |
| CN106021506A (en) * | 2016-05-23 | 2016-10-12 | 深圳市中博科创信息技术有限公司 | File storage method and apparatus for cluster system |
| CN106899693A (en) * | 2017-03-20 | 2017-06-27 | 聚好看科技股份有限公司 | Distributed file management method and apparatus |
| CN111095233A (en) * | 2017-09-28 | 2020-05-01 | 深圳清华大学研究院 | Hybrid file system architecture, file storage, dynamic migration and applications thereof |
| CN111095233B (en) * | 2017-09-28 | 2023-09-26 | 深圳清华大学研究院 | Hybrid file system architecture, file storage, live migration and applications thereof |
| CN107844542A (en) * | 2017-10-26 | 2018-03-27 | 山东浪潮通软信息科技有限公司 | A kind of distributed document storage method and device |
| CN109542910A (en) * | 2018-12-03 | 2019-03-29 | 中国电波传播研究所（中国电子科技集团公司第二十二研究所） | A kind of magnanimity sea clutter data classification storage method |
| CN109542910B (en) * | 2018-12-03 | 2022-10-28 | 中国电波传播研究所（中国电子科技集团公司第二十二研究所） | Mass sea clutter data classification storage method |
| CN110765075A (en) * | 2019-10-24 | 2020-02-07 | 北京百度网讯科技有限公司 | Storage method and equipment of automatic driving data |
| CN112988062A (en) * | 2021-01-28 | 2021-06-18 | 腾讯科技（深圳）有限公司 | Metadata reading limiting method and device, electronic equipment and medium |
| CN112988062B (en) * | 2021-01-28 | 2023-02-14 | 腾讯科技（深圳）有限公司 | Metadata reading limiting method and device, electronic equipment and medium |
| CN113032405A (en) * | 2021-05-25 | 2021-06-25 | 湖南省第二测绘院 | Spatio-temporal data management method, system, host and computer readable storage medium |
| CN114500485A (en) * | 2022-01-28 | 2022-05-13 | 北京沃东天骏信息技术有限公司 | Data processing method and device |
| CN115834713A (en) * | 2023-02-07 | 2023-03-21 | 北京大道云行科技有限公司 | Interaction method and system for network file system and distributed file system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101944124B (en) | 2012-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101944124B (en) | Distributed file system management method, device and corresponding file system | |
| US9933979B2 (en) | Device and method for storing data in distributed storage system | |
| US20100312749A1 (en) | Scalable lookup service for distributed database | |
| CN102725755B (en) | Method and system of file access | |
| CN102930060B (en) | A kind of method of database quick indexing and device | |
| CN104021161A (en) | Cluster storage method and device | |
| CN105354255A (en) | Data query method and apparatus | |
| CN101996217A (en) | Method for storing data and memory device thereof | |
| CN101741907A (en) | Method and system for balancing server load and main server | |
| CN107026876B (en) | A kind of file data access system and method | |
| CN107315533A (en) | A kind of date storage method and device | |
| CN102708165A (en) | Method and device for processing files in distributed file system | |
| CN101741730A (en) | Method and equipment for downloading file and method and system for providing file downloading service | |
| CN101539950A (en) | Data storage method and device | |
| CN109033360B (en) | Data query method, device, server and storage medium | |
| CN104881466A (en) | Method and device for processing data fragments and deleting garbage files | |
| CN104731896A (en) | Data processing method and system | |
| CN103324533A (en) | distributed data processing method, device and system | |
| CN103186622A (en) | Updating method of index information in full text retrieval system and device thereof | |
| CN104408044A (en) | File access method and system | |
| CN106951179A (en) | A kind of data migration method and device | |
| CN106713250A (en) | Data access method and device based on distributed system | |
| CN103279489A (en) | Method and device for storing metadata | |
| CN103049574A (en) | Key value system and key value method for implementation of dynamic duplicates of documents | |
| CN104021088A (en) | Log storage method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120704 Termination date: 20130921 |