CN101599994B - Distributed file system (DFS), access node (AN) and method of transmitting file data among nodes - Google Patents
Distributed file system (DFS), access node (AN) and method of transmitting file data among nodes Download PDFInfo
- Publication number
- CN101599994B CN101599994B CN2009100858083A CN200910085808A CN101599994B CN 101599994 B CN101599994 B CN 101599994B CN 2009100858083 A CN2009100858083 A CN 2009100858083A CN 200910085808 A CN200910085808 A CN 200910085808A CN 101599994 B CN101599994 B CN 101599994B
- Authority
- CN
- China
- Prior art keywords
- access node
- node
- file
- access
- service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The invention provides a distributed file system (DFS), an access node (AN) and a method of transmitting file data among nodes; wherein, the access node is applied to a distributed file system containing at least one service node, and comprises a first processing module and a second processing module; the first processing module is used for receiving file service request from a user and transmitting an inquiry request to the first service node(s) of at least one service node after judging that the file data requested by the file service request does not exist locally; the second processing module is used for receiving a second access node list returned by the first service node, each second access node of the second access node list is stored with the data block of the file data requested by the file service request, establishing connection with each the second access node and obtaining the file data requested by the file service request from the second access node to reverse back to the user. The invention improves the network transmission efficiency.
Description
Technical field
The present invention relates to a kind of distributed file system field, be meant the method for transfer files data between a kind of distributed file system, access node and node of cross-node store file data especially.
Background technology
As shown in Figure 1, the networking mode of the distributed file system of existing network operation at present generally adopts the structure of layering; For example: Centroid; The regional center node, fringe node etc., a Centroid is in charge of several regional center nodes; Each regional center node is in charge of a plurality of fringe nodes, and the user generally inserts distributed file system through fringe node.
Because memory limitations, fringe node can't be preserved all files of the whole network.If can not find certain file that oneself needs on the fringe node of certain user under him, generally to adopt dual mode be that the user provides service to fringe node so:
First kind of mode is: user's tandem is had on the regional center node or Centroid of this document to upper level, by even higher level of node service is provided;
The second way is: the fringe node under the user is responsible for having the node of this document to be transferred to this node from upper level this document, for the user service is provided then.
If adopt above-mentioned first kind of mode; It is tandem has the node of this document to upper level mode; Since file content can not be in time nodal cache on the edge of, a large amount of users will be scheduled for regional center node or Centroid, can cause the Centroid overload overweight.
If adopt the above-mentioned second way, the file that is about to user's request is transferred to the mode of this fringe node from file source node (being regional center node or Centroid), and the generic-document transmission all to adopt complete file be unit; This will produce following problem: if when file transfer will be accomplished, the file source node breaks down suddenly, just can cause this bust this; The data of having transmitted will be dropped; If continue as the user service is provided, must seeks new file source node, proceed file transfer; This makes between node the file transfer scheduling waste the network bandwidth and node processing ability, service second-rate.
In addition; In the existing distributed file system, though some system adopts: complete file is divided into some little data blocks carries out storage transmission, reduced the transmission granularity to a certain extent; Improve network bandwidth utilance; But the scheduled transmission method only is simply to carry out scheduled transmission based on the visit temperature of individual data piece, when data block does not reach heat degree threshold, can trigger data not dispatch, and causes network transmission efficiency lower like this; The real-time of scheduling is relatively poor, can not well for the user service be provided.
Summary of the invention
The technical problem that the present invention will solve provides the method for transfer files data between a kind of distributed file system, access node and node, the file data transmission technical problem that real-time is relatively poor, network transmission efficiency is low between the Centroid overload that exists in the solution prior art, node.
For solving the problems of the technologies described above, the present invention provides a kind of access node, is applied to comprise the distributed file system of at least one service node, comprising:
First processing module is used to receive user's file service request, judges locally when not having file data that said file service request asked, sends first service node in a query requests to said at least one service node;
Second processing module; Be used to receive second access node tabulation that said first service node returns; Store the data block of the file data that said file service request asked in each second access node in the tabulation of said second access node; And connect with said each second access node, from said second access node, obtain the file data that said file service request is asked, return to the user.
Wherein, said second processing module comprises:
Link block is used to receive second access node tabulation that said service node returns, and sets up parallel the connection with each second access node in said second access node tabulation;
Acquisition module is used for from said each second access node, obtains the different pieces of information piece of the file data that said file service request asked respectively, and said different pieces of information piece is merged into the complete file data, returns to the user.
Wherein, above-mentioned access node also comprises:
Memory module is used for the data block of store file data, said file data and the tabulation of said at least one service node;
Said first processing module comprises:
Receiver module is used to receive user's file service request;
First processing sub is used for when said memory module has the file data that said file service request asked, directly said file data being returned to the user;
Second processing sub is used for when said memory module does not have file data that said file service request asked, sending first service node in a query requests to the tabulation of said at least one service node.
Wherein, above-mentioned access node also comprises:
Switching Module is used for and two said access nodes at least, and other access node except that local access node exchanges said file data and the tabulation of said at least one service node in the said memory module.
Wherein, above-mentioned access node also comprises:
Handover module is used for when said first service node is unavailable, from the tabulation of said at least one service node, reselecting a service node, as the first new service node;
Said second processing sub is sent said query requests to the said first new service node.
Wherein, above-mentioned access node also comprises:
First maintenance module; Be used in said second processing module when said second access node obtains data block and accomplishes; Calculate the Hash of said data block; If the Hash of said data block is different with the Hash that said second access node sends, then obtain said data block again, identical up to the Hash of said data block with the Hash that said second access node sends.
Wherein, above-mentioned access node also comprises:
When second maintenance module, the visit temperature of the file data that is used for obtaining from said second access node in said second processing module are lower than a preset heat degree threshold, said file data is carried out burin-in process.
Wherein, above-mentioned access node also comprises:
The third dimension is protected module; Be used to safeguard that is downloaded a formation; When local access node obtains data block, adopt the principle of FIFO at a plurality of other access nodes, the data block of the file data that said other access node is asked returns to said other access node.
Wherein, above-mentioned access node also comprises:
Fourth dimension protects module; The same data block of the file data that is used for being asked in the said file service request of local access node; In the time of in the download formation of the second different access nodes; If said data block is downloaded from one of them second access node and accomplished, then when said query requests arrived the top of download formation of other second access node, refusal was carried out the download of this data block.
The present invention also provides a kind of distributed file system, comprising: at least two access nodes and at least one service node;
First access node in said at least two access nodes comprises:
First processing module is used to receive user's file service request, judges locally when not having file data that said file service request asked, sends first service node in a query requests to said at least one service node;
Said first service node comprises:
Receiver module is used to receive the query requests that said first processing module is sent;
Select module; Be used for according to said query requests, from said at least two access nodes, obtain at least one second access node; Form second access node tabulation, said second access node stores the data block of the file data that said file service request asked;
Sending module is used for said second access node tabulation is returned to said first access node;
Said first access node also comprises:
Second processing module is used for connecting with each second access node of said second access node tabulation, from said second access node, obtains the file data that said file service request is asked, and returns to the user.
Wherein, said second access node is during at least two, and said second processing module comprises:
Link block is used to receive second access node tabulation that said sending module returns, and sets up parallel the connection with each second access node in said second access node tabulation;
Acquisition module is used for from said each second access node, obtains the different pieces of information piece of the file data that said file service request asked respectively, and said different pieces of information piece is merged into the complete file data, returns to the user.
The present invention also provides a kind of service node, is applied to include the distributed file system of at least two access nodes, comprising:
Receiver module is used for receiving the query requests that first access node of said at least two access nodes sends;
Acquisition module; Be used for according to said query requests, from said at least two access nodes, obtain at least one second access node; Form second access node tabulation, said second access node stores the data block of the file data that said file service request asked;
Sending module is used for said second access node tabulation is returned to said first access node.
Embodiments of the invention also provide the method for transfer files data between a kind of node, are applied to comprise the distributed file system of at least two access nodes and at least one service node, and said method comprises:
First access node in said at least two access nodes receives user's file service request; Judge locally when not having file data that said file service request asked, send first service node in a query requests to said at least one service node;
Said first service node is according to said query requests; From said at least two access nodes; Obtain at least one second access node; Form second access node tabulation, and will said second access node tabulate and return to said first access node, said second access node stores the data block of the file data that said file service request asked;
Each second access node in said first access node and the tabulation of said second access node connects, and from said second access node, obtains the file data that said file service request is asked, and returns to the user.
Wherein, Said second access node is during at least two; Each second access node in said first access node and the tabulation of said second access node connects; From said second access node, obtain the file data that said file service request is asked, the step that returns to the user specifically comprises:
Said first access node receives second access node tabulation that said service node returns, and sets up parallel the connection with each second access node in said second access node tabulation;
Said first access node obtains the different pieces of information piece of the file data that said file service request asked respectively from said each second access node, and said different pieces of information piece is merged into the complete file data, returns to the user.
Wherein, when said first access node obtains the data block completion from said second access node, also comprise:
Calculate the Hash of said data block,, then obtain said data block again if the Hash of said data block is different with the Hash that said second access node sends, identical up to the Hash of said data block with the Hash that said second access node sends.
The beneficial effect of technique scheme of the present invention is following:
In the such scheme; First access node sends a query requests to service node, after this service node receives this query requests searching this locality when not having file data that the user asks; Can be with in this distributed file system; The tabulation of second access node of a plurality of data blocks that store the file data that this document services request asked returns to this first access node, and this first access node can be set up parallel the connection with these second access nodes; Obtain the different pieces of information piece of the file data that this document services request asked simultaneously from these second access nodes; Parallel transmission between a plurality of nodes of realization data block improves network transmission efficiency, improves the real-time of data dispatch.
Description of drawings
Fig. 1 is the networking structure sketch map of existing distributed file system;
Fig. 2 is the networking structure sketch map of embodiments of the invention distributed file system;
Fig. 3 is the concrete networking structure sketch map of distributed file system shown in Figure 2;
Fig. 4 is in the distributed file system shown in Figure 3, a concrete structure sketch map of first access node;
Fig. 5 is another concrete structure sketch map of first access node;
Fig. 6 is the another concrete structure sketch map of first access node;
Fig. 7 is a concrete structure sketch map again of first access node;
Fig. 8 is a concrete structure sketch map again of first access node;
Fig. 9 is a concrete structure sketch map again of first access node;
Figure 10 is the concrete structure sketch map that first access node has the download formation maintenance function of second access node;
Figure 11 is a concrete structure sketch map again of first access node;
Figure 12 is in the above-mentioned distributed file system, a concrete file transfer sketch map;
Figure 13 is the method flow sketch map of transfer files data between node of the present invention.
Embodiment
For technical problem, technical scheme and advantage that the present invention will be solved is clearer, will combine accompanying drawing and specific embodiment to be described in detail below.
The present invention is directed in the existing distributed file system; The network transmission efficiency of data is lower between each node; The real-time of scheduling is relatively poor; Can not provide the network transmission efficiency of data between a kind of each node higher well for the user provides service problem, the real-time of scheduling is the method for transfer files data between distributed file system, access node and node preferably.
Like Fig. 2, shown in Figure 3, the embodiments of the invention distributed file system comprises: at least two access nodes 30 and at least one service node 40, not communication between these service nodes 40, the correctness of transfer of data between node in the assurance system;
Wherein, for describing conveniently, will at least two in the access nodes, when this locality does not have file data that user's file service request asked, be called first access node 31 to the access node of service node initiation request; Will at least two in the access nodes, store the data block of the file data that user's file service request asked, and, be called second access node 32 for this first access node 31 provides the access node of file data service;
In the actual distributed file system, this first access node 31 is same user's access nodes with this second access node 32, and this user's access node is a subscription access server, and the user obtains the network file service through this access node; In this system, any access node all might be to service node 40 initiation request, and any access node also all might provide the file data service for other access node in the system.
As shown in Figure 3, first access node 31 in above-mentioned at least two access nodes can specifically comprise:
This first service node 40 is servers, uses an internal database to preserve the metadata information of access node and file data; Service node is not preserved file data, and it is the center index of file location information, and concentrated index service is provided;
This first service node 40 can specifically comprise:
Sending module 403 is used for said second access node tabulation is returned to this first access node 31; Specifically, this sending module 403 is specially: the address sending module is used for the address of this each second access node of second access node tabulation is returned to this first access node 31; If that storage is the title or the ID of at least one second access node 32 in certainly above-mentioned second access node tabulation; Then this sending module 403 is that the title or the ID of second access node 32 during this second access node is tabulated returns to first access node when first access node 31 returns the tabulation of second access node;
This first access node 31 also can comprise:
As shown in Figure 4, in this distributed file system, second access node is during at least 2, and this second processing module 312 can specifically comprise:
Among this embodiment; First access node 31 is being searched this locality when not having file data that the user asks; To sending a query requests for service node 40, this service node 40 can be with in this distributed file system according to this query requests, the tabulation of second access node 32 of a plurality of data blocks that store the file data that this document services request asked; Return to this first access node 31; This first access node 31 can be set up parallel connections with these second access nodes 32, obtains the different pieces of information piece of the file data that this document services request asked simultaneously from these second access nodes 32, parallel transmission between a plurality of nodes of realization data block; Improve network transmission efficiency, improve the real-time of data dispatch.
As shown in Figure 5, in this distributed file system, this first access node 31 also can comprise:
This document data can be divided into data block; Stored in form with data block; This first access node 31 can data block be that granularity is transmitted when for other access node in the system file data service being provided, and conveniently for a plurality of other access nodes the file data service is provided simultaneously; Certainly the files stored data can be for a plurality of in this memory module 313, and also should store the listed files of these file datas this moment in this memory module 313, so that service node is searched the needed file data of other access node on this access node;
Above-mentioned first processing module 311 comprises:
As shown in Figure 6, in above-mentioned distributed file system, this first access node 31 also can comprise:
As shown in Figure 7, in the above-mentioned distributed file system, this first access node 31 also can comprise:
That is to say; When above-mentioned first access node 31 starts, be connected to one of them user's service node according to the address in the tabulation of the service node of storage in the memory module 313 and obtain the network service, this connection will keep always; This first access node 31 can not be connected to a plurality of servers simultaneously; When having only current service node (i.e. this first service node 40) unavailable, just can switch to other service nodes in the tabulation of service node, guarantee the reliability of file data transmission like this.
As shown in Figure 8, in the above-mentioned distributed file system, this first access node 31 also can comprise:
As the data block of the damage that will download (for example: data block B), be divided into again plurality of small blocks (for example: b1, b2, b3...bn), then:
1. download b1 again, cover original b1, after the covering, data block B is carried out the Hash verification,, then repair successfully, otherwise carry out next step if verification is errorless with the b1 that downloads;
2. download b2 again, cover original b2, after the covering, data block B is carried out the Hash verification,, then repair successfully, otherwise continue to carry out if verification is errorless with the b2 that downloads;
3. up to data block B verification succeeds, the data block reparation of the damage of soon having downloaded well.
As shown in Figure 9, in the above-mentioned distributed file system, this first access node 31 also can comprise:
Shown in figure 10, in above-mentioned distributed file system, above-mentioned second access node 32 can comprise:
The third dimension is protected module 318; Be used to safeguard that is downloaded a formation; At a plurality of first access nodes 31 when this second access node 32 obtains data block; Adopt the principle of FIFO, the data block of the file data that said other first access node is asked returns to said other first access node; Specifically; At a plurality of first access nodes 31 when this second access node 32 obtains data block; Other first access node is afterwards added the bottom that this downloads formation; When other first access node arrived this top of downloading formation, the data block of the file data of again other first access node being asked returned to this other first access node.
Same; Among the distributed file system embodiment of the present invention, first access node 31 too can be for other access node provides the file data service, and promptly this first access node 31 has the above-mentioned third dimension too and protects module 318; For its data block of shared file data safeguards that is downloaded a formation; The access node downloaded adds the bottom of this formation, advancing gradually then, and the top that reaches formation up to this access node begins file in download; When the access node of downloading has arrived the top of download formation of this access node 31, this first access node 31 set up one with being connected of the access node of downloading, the data block that is used for having it sends user's access node of needs download to.
Shown in figure 11, in the above-mentioned distributed file system, this first access node 31 also can comprise:
Fourth dimension protects module 319; The same data block of the file data that is used for being asked in the file service request of first access node 31 is when the download formation of the second different access nodes 32; If this data block is downloaded from one of them second access node 32 and is accomplished; Then when this query requests arrived the top of download formation of other second access node 32, refusal was carried out the download of this data block.
That is to say; User's access node (like first access node 31) possibly downloaded same file data simultaneously from a plurality of other user's access nodes (like second access node 32); Obtain the different pieces of information piece of file from different user's access node (like second access node 32), the file data different portions just can be downloaded simultaneously like this;
User's access node (like first access node 31) also may be in a plurality of other download waiting lists of user's access node (like second access node 32); Other user's access node at each (like second access node 32) is gone up and is downloaded same file data blocks; In fact having accomplished this data block when user's access node (like first access node 31) of this wait downloads; It can not notify those other user's access node (like second access node 32) to delete its download request, and only is that refusal is carried out and downloaded when in the download formation of those other user's access node (like second access node 32) at its place, being discharged to the formation top; Certainly this user's access node also can be notified other its download request of user's access node deletion.
Below in conjunction with Figure 12, specify in the above-mentioned distributed file system browsing process of file data between node:
When (1) user's access node (being first access node 31 in the foregoing description) starts, from the tabulation of storage service node within it, select a service node (being above-mentioned first service node 40), connect;
(2) when the user needs access file, user's access node is at first searched this document data in local memory module, if find, then directly to the user file data is provided, otherwise user's access node is initiated query requests to the service node that connects;
(3) user's service node is user's access node tabulation at required file data place, i.e. file source node listing (being above-mentioned second access node tabulation), and for example node A, B issue the requesting users access node;
(4) user's access node simultaneously all nodes in the file source node listing initiate connection requests; Promptly initiate connection request to file source access node A and file source access node B simultaneously; Exchange then about information such as the tabulation of user's service node and files stored tabulations, and the request file in download.
(5) each file source node joins download formation bottom with the download request of other user's access nodes, waits for downloads, and when other user's access nodes reach the top of formation, just begins file in download;
(6) user's access node is from a plurality of file source nodes; For example node A, B; Download same file simultaneously, obtain the different pieces of information piece of file, for example obtain data block a from file source access node A from different file source nodes; B obtains data block b from the file source access node, and the file different portions just has been downloaded simultaneously like this;
(7) after user's access node is accomplished the download of data block; The Hash of calculated data piece; The blocks of files Hash that this Hash and file source node send is done comparison,, damaged with regard to the supporting paper piece if different; User's access node covers the blocks of files of damaging (for example 128kb) with the download of block-by-block, till hash calculation shows that blocks of files has been repaired;
The data block of (8) downloading successfully or asking when file has been filled up the required buffering area of user, and user's access node just can provide file service to the user.
To sum up; Among the above-mentioned distributed file system embodiment of the present invention; First access node 31 sends a query requests searching this locality when not having file data that the user asks to service node 40, and this service node 40 can be with in this distributed file system according to this query requests; The tabulation of second access node 32 of a plurality of data blocks that store the file data that this document services request asked; Return to this first access node 31, this first access node 31 can be set up parallel connections with these second access nodes 32, obtains the different pieces of information piece of the file data that this document services request asked simultaneously from these second access nodes 32; Problem such as overcome between Centroid overload that prior art exists, node the file transfer scheduling waste network bandwidth and disposal ability and network transmission efficiency is lower and the real-time of scheduling is relatively poor; Parallel transmission between a plurality of nodes of realization data block improves network transmission efficiency, improves the real-time of data dispatch.
Embodiments of the invention also provide a kind of access node; This access node is applied to comprise the distributed file system of at least one service node; This access node is a subscription access server, becomes data block to store file division, and the user obtains the network file service through user's access node; The concrete structure of this access node such as above-mentioned Fig. 3 are to first access node 31 shown in Figure 10; This access node is identical with first access node 31 in the above-mentioned distributed file system, and above-mentioned description all about first access node 31 is equally applicable to this access node, repeats no more at this.
Embodiments of the invention also provide a kind of service node; Be applied to include the distributed file system of at least two access nodes; This service node is a client server, uses an internal database to preserve the metadata information of user's access node and file, and it is the center index of file location information; Concentrated index service is provided, does not have communication between the different servers.As shown in Figure 3, this service node 40 comprises:
Sending module 403; Be used for this second access node tabulation is returned to first access node; Specifically, this sending module is specially: the address sending module is used for the address of this each second access node of second access node tabulation is returned to this first access node.Like this; First access node just can be set up parallel the connection with each second access node, from these second access nodes, obtains different data blocks simultaneously, parallel transmission between a plurality of nodes of realization data block; Improve network transmission efficiency, improve the real-time of data dispatch.
Shown in figure 13, embodiments of the invention also provide the method for transfer files data between a kind of node, are applied to comprise the distributed file system of at least two access nodes and at least one service node, and this method comprises:
Wherein, above-mentioned steps 133 can specifically comprise:
First access node receives second access node tabulation that said service node returns, and sets up parallel the connection with each second access node in said second access node tabulation;
First access node obtains the different pieces of information piece of the file data that said file service request asked respectively from said each second access node, and said different pieces of information piece is merged into the complete file data, returns to the user.
When first access node obtains the data block completion from said second access node, also comprise:
Calculate the Hash of said data block; If the Hash of said data block is different with the Hash that said second access node sends; Then notify said second access node to retransmit said data block, identical up to the Hash of said data block with the Hash that said second access node sends.
After the above-mentioned steps 133, when the visit temperature of complete file data is lower than a preset heat degree threshold, said complete file data are carried out burin-in process.
In addition, in the above-mentioned steps 133, when first access node obtains data block from second access node, also comprise:
Second access node safeguards that is downloaded a formation, when a plurality of first access nodes and second access node obtain data block, and the principle of employing FIFO, the data block of the file data that other first access node is asked returns to other first access node; Specifically; Other first access node is afterwards added the bottom that this downloads formation; When other first access node arrived this top of downloading formation, the data block of the file data of again other first access node being asked returned to other first access node.
In the above-mentioned steps 133; Also can comprise: the same data block of the file data that the file service request of first access node is asked; In the time of in the download formation of the second different access nodes; If this data block is downloaded from one of them second access node and accomplished, then when this query requests arrived the top of download formation of other second access node, refusal was carried out the download of this data block.
This method embodiment; First access node is being searched this locality when not having file data that the user asks; Send a query requests to service node; This service node can be with in this distributed file system according to this query requests, and the tabulation of second access node of a plurality of data blocks that store the file data that this document services request asked returns to this first access node; This first access node can be set up parallel the connection with these second access nodes; Obtain the different pieces of information piece of the file data that this document services request asked simultaneously from these second access nodes, problem such as overcome between the Centroid overload that prior art exists, node the file transfer scheduling waste network bandwidth and disposal ability and network transmission efficiency is lower and the real-time of scheduling is relatively poor realizes parallel transmission between a plurality of nodes of data block; Improve network transmission efficiency, improve the real-time of data dispatch.
The above is a preferred implementation of the present invention; Should be pointed out that for those skilled in the art, under the prerequisite that does not break away from principle according to the invention; Can also make some improvement and retouching, these improvement and retouching also should be regarded as protection scope of the present invention.
Claims (15)
1. access node is applied to comprise it is characterized in that the distributed file system of at least one service node, comprising:
First processing module is used to receive user's file service request, judges locally when not having file data that said file service request asked, sends first service node in a query requests to said at least one service node;
Second processing module; Be used to receive second access node tabulation that said first service node returns; Store the data block of the file data that said file service request asked in each second access node in the tabulation of said second access node; And connect with said each second access node, from said second access node, obtain the file data that said file service request is asked, return to the user.
2. access node according to claim 1 is characterized in that, said second processing module comprises:
Link block is used to receive second access node tabulation that said service node returns, and sets up parallel the connection with each second access node in said second access node tabulation;
Acquisition module is used for from said each second access node, obtains the different pieces of information piece of the file data that said file service request asked respectively, and said different pieces of information piece is merged into the complete file data, returns to the user.
3. access node according to claim 1 is characterized in that, also comprises:
Memory module is used for the data block of store file data, said file data and the tabulation of said at least one service node;
Said first processing module comprises:
Receiver module is used to receive user's file service request;
First processing sub is used for when said memory module has the file data that said file service request asked, directly said file data being returned to the user;
Second processing sub is used for when said memory module does not have file data that said file service request asked, sending first service node in a query requests to the tabulation of said at least one service node.
4. access node according to claim 3 is characterized in that, also comprises:
Switching Module is used for and two said access nodes at least, and other access node except that local access node exchanges said file data and the tabulation of said at least one service node in the said memory module.
5. access node according to claim 3 is characterized in that, also comprises:
Handover module is used for when said first service node is unavailable, from the tabulation of said at least one service node, reselecting a service node, as the first new service node;
Said second processing sub is sent said query requests to the said first new service node.
6. according to each described access node of claim 1-5, it is characterized in that, also comprise:
First maintenance module; Be used in said second processing module when said second access node obtains data block and accomplishes; Calculate the Hash of said data block; If the Hash of said data block is different with the Hash that said second access node sends, then obtain said data block again, identical up to the Hash of said data block with the Hash that said second access node sends.
7. according to each described access node of claim 1-5, it is characterized in that, also comprise:
When second maintenance module, the visit temperature of the file data that is used for obtaining from said second access node in said second processing module are lower than a preset heat degree threshold, said file data is carried out burin-in process.
8. according to each described access node of claim 1-5, it is characterized in that, also comprise:
The third dimension is protected module; Be used to safeguard that is downloaded a formation; When local access node obtains data block, adopt the principle of FIFO at a plurality of other access nodes, the data block of the file data that said other access node is asked returns to said other access node.
9. according to each described access node of claim 1-5, it is characterized in that, also comprise:
Fourth dimension protects module; The same data block of the file data that is used for being asked in the said file service request of local access node; In the time of in the download formation of the second different access nodes; If said data block is downloaded from one of them second access node and accomplished, then when said query requests arrived the top of download formation of other second access node, refusal was carried out the download of this data block.
10. a distributed file system comprises: two access nodes and at least one service node at least; It is characterized in that,
First access node in said at least two access nodes comprises:
First processing module is used to receive user's file service request, judges locally when not having file data that said file service request asked, sends first service node in a query requests to said at least one service node;
Said first service node comprises:
Receiver module is used to receive the query requests that said first processing module is sent;
Select module; Be used for according to said query requests, from said at least two access nodes, obtain at least one second access node; Form second access node tabulation, said second access node stores the data block of the file data that said file service request asked;
Sending module is used for said second access node tabulation is returned to said first access node;
Said first access node also comprises:
Second processing module is used for connecting with each second access node of said second access node tabulation, from said second access node, obtains the file data that said file service request is asked, and returns to the user.
11. distributed file system according to claim 10 is characterized in that, said second access node is during at least two, and said second processing module comprises:
Link block is used to receive second access node tabulation that said sending module returns, and sets up parallel the connection with each second access node in said second access node tabulation;
Acquisition module is used for from said each second access node, obtains the different pieces of information piece of the file data that said file service request asked respectively, and said different pieces of information piece is merged into the complete file data, returns to the user.
12. a service node is applied to include the distributed file system of at least two access nodes, it is characterized in that, comprising:
Receiver module is used for receiving the query requests that first access node of said at least two access nodes sends;
Acquisition module; Be used for according to said query requests, from said at least two access nodes, obtain at least one second access node; Form second access node tabulation, said second access node stores the data block of the file data that said file service request asked;
Sending module is used for said second access node tabulation is returned to said first access node.
13. the method for transfer files data between a node is applied to comprise the distributed file system of at least two access nodes and at least one service node it is characterized in that said method comprises:
First access node in said at least two access nodes receives user's file service request; Judge locally when not having file data that said file service request asked, send first service node in a query requests to said at least one service node;
Said first service node is according to said query requests; From said at least two access nodes; Obtain at least one second access node; Form second access node tabulation, and will said second access node tabulate and return to said first access node, said second access node stores the data block of the file data that said file service request asked;
Each second access node in said first access node and the tabulation of said second access node connects, and from said second access node, obtains the file data that said file service request is asked, and returns to the user.
14. method according to claim 13; It is characterized in that; Said second access node is during at least two; Each second access node in said first access node and the tabulation of said second access node connects, and from said second access node, obtains the file data that said file service request is asked, and the step that returns to the user specifically comprises:
Said first access node receives second access node tabulation that said service node returns, and sets up parallel the connection with each second access node in said second access node tabulation;
Said first access node obtains the different pieces of information piece of the file data that said file service request asked respectively from said each second access node, and said different pieces of information piece is merged into the complete file data, returns to the user.
15. according to claim 13 or 14 described methods, it is characterized in that, when said first access node obtains the data block completion from said second access node, also comprise:
Calculate the Hash of said data block,, then obtain said data block again if the Hash of said data block is different with the Hash that said second access node sends, identical up to the Hash of said data block with the Hash that said second access node sends.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009100858083A CN101599994B (en) | 2009-06-01 | 2009-06-01 | Distributed file system (DFS), access node (AN) and method of transmitting file data among nodes |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009100858083A CN101599994B (en) | 2009-06-01 | 2009-06-01 | Distributed file system (DFS), access node (AN) and method of transmitting file data among nodes |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101599994A CN101599994A (en) | 2009-12-09 |
| CN101599994B true CN101599994B (en) | 2012-07-18 |
Family
ID=41421238
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009100858083A Expired - Fee Related CN101599994B (en) | 2009-06-01 | 2009-06-01 | Distributed file system (DFS), access node (AN) and method of transmitting file data among nodes |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101599994B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110554919A (en) * | 2019-08-19 | 2019-12-10 | 武汉世纪楚林科技有限公司 | Edge device data processing method and device, terminal device and storage medium |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103200242B (en) * | 2013-03-20 | 2016-04-06 | 成都康赛信息技术有限公司 | The method of cross-layer face data analysis hinge is built based on Internet of Things |
| CN104580268A (en) * | 2013-10-09 | 2015-04-29 | 南京中兴新软件有限责任公司 | Method and device for transmitting file data |
| CN104679665A (en) * | 2013-12-02 | 2015-06-03 | 中兴通讯股份有限公司 | Method and system for achieving block storage of distributed file system |
| CN110023944B (en) * | 2017-01-03 | 2021-12-28 | 华为技术有限公司 | Communication method, terminal equipment and core network equipment |
| CN106888261A (en) * | 2017-02-27 | 2017-06-23 | 深圳市先河系统技术有限公司 | A kind of distributed document transmission method, server and private clound equipment |
| CN108737470B (en) * | 2017-04-19 | 2020-03-13 | 贵州白山云科技股份有限公司 | Access request source returning method and device |
| CN107229692B (en) * | 2017-05-19 | 2018-05-01 | 哈工大大数据产业有限公司 | A kind of distributed multi-table connecting method and system based on assembly line |
| CN108710681B (en) * | 2018-05-18 | 2022-02-22 | 腾讯科技（深圳）有限公司 | File acquisition method, device, equipment and storage medium |
| CN112491939B (en) * | 2019-09-12 | 2022-12-27 | 上海哔哩哔哩科技有限公司 | Multimedia resource scheduling method and system |
| CN112749004B (en) * | 2019-10-30 | 2023-09-05 | 中国移动通信集团安徽有限公司 | Data storage method and device based on node access heat |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101404649A (en) * | 2008-11-11 | 2009-04-08 | 阿里巴巴集团控股有限公司 | Data processing system based on CACHE and its method |
| CN101426193A (en) * | 2007-11-01 | 2009-05-06 | 华为技术有限公司 | Network resource information management method, communication apparatus and communication system |
| CN101442475A (en) * | 2007-11-24 | 2009-05-27 | 华为技术有限公司 | Distributed business acting method, network system and network appliance |
-
2009
- 2009-06-01 CN CN2009100858083A patent/CN101599994B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101426193A (en) * | 2007-11-01 | 2009-05-06 | 华为技术有限公司 | Network resource information management method, communication apparatus and communication system |
| CN101442475A (en) * | 2007-11-24 | 2009-05-27 | 华为技术有限公司 | Distributed business acting method, network system and network appliance |
| CN101404649A (en) * | 2008-11-11 | 2009-04-08 | 阿里巴巴集团控股有限公司 | Data processing system based on CACHE and its method |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110554919A (en) * | 2019-08-19 | 2019-12-10 | 武汉世纪楚林科技有限公司 | Edge device data processing method and device, terminal device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101599994A (en) | 2009-12-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101599994B (en) | Distributed file system (DFS), access node (AN) and method of transmitting file data among nodes | |
| CN101741986B (en) | Page cache method for mobile communication equipment terminal | |
| JP5745169B2 (en) | Content processing method, content processing device, and content processing system | |
| US8068512B2 (en) | Efficient utilization of cache servers in mobile communication system | |
| CN102375837B (en) | Data acquiring system and method | |
| CN102523256A (en) | Content management method, device and system | |
| CN105005611B (en) | A kind of file management system and file management method | |
| WO2008104131A1 (en) | Method, apparatus and system of searching and downloading mobile telephone file | |
| CN105868333A (en) | File processing method and device | |
| US8086629B2 (en) | Content delivery apparatus, content delivery method, and content delivery program | |
| CN102333130A (en) | Method and system for accessing cache server and intelligent cache scheduler | |
| CN103685476B (en) | The method and web page template server that webpage shows are realized using web page template | |
| CN102833352A (en) | Distributed cache management system and method for implementing distributed cache management | |
| CN105721538A (en) | Data access method and apparatus | |
| CN105068755A (en) | Data duplicate storage method facing cloud computing content distribution network | |
| CN103179148A (en) | Processing method and system for sharing enclosures in internet | |
| CN109558378A (en) | File management method, device, equipment and storage medium | |
| CN104239509A (en) | Multi-version GIS slicing service system | |
| CN102546674A (en) | Directory tree caching system and method based on network storage device | |
| CN106331160A (en) | Data migration method and system | |
| CN102404372A (en) | Method, system and node device for storing content in WEB cache in distributed mode | |
| CN109905479A (en) | Document transmission method and device | |
| US11606415B2 (en) | Method, apparatus and system for processing an access request in a content delivery system | |
| CN102075584A (en) | Distributed file system and access method thereof | |
| CN103024018A (en) | Method and device for operating multiple content distribution network (CDN) service processes in single device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120718 Termination date: 20130601 |