Embodiment
The following embodiment of the present invention provides a kind of formula document handling method, system and client based on distributed file system.
In order to make those skilled in the art person understand the technical scheme in the embodiment of the invention better; And make the above-mentioned purpose of the embodiment of the invention, feature and advantage can be more obviously understandable, below in conjunction with accompanying drawing technical scheme in the embodiment of the invention done further detailed explanation.
Distributed file system among the application embodiment comprises metadata node; The plurality of client end of communicating by letter with metadata node; And the plurality of data memory node of communicating by letter with metadata node; Wherein each data memory node is made up of at least one memory device, and the memory property of each memory device maybe be different.
Referring to Fig. 2, be the first embodiment flow chart of the application based on the document handling method of distributed file system:
Step 201: preserve consistency Hash ring in the client based on dummy node.
For the data memory node in the distributed file system; Each memory device on it corresponds at least one dummy node on the consistency Hash ring according to the size of its memory capacity; Each dummy node is preserved the device identification and the user mode of the memory device under the dummy node, according to the value that each dummy node distributes dummy node that adds up to of dummy node.Further, per two adjacent dummy nodes do not belong to same memory device, so that when write operation, reduce the possibility of choosing the dummy node that belongs to same memory device.
Step 202: client is sent file to metadata node and is write request.
When the data in the client terminal local buffer memory surpass predetermined number, need these data be write with document form in the memory device of data memory node, therefore send file and write request to metadata node.
Step 203: receive metadata node and write the file identification that request is returned for this document distribution according to file; And form the data block sign of each data block of this document, the copy amount and the selectivity factor of data block, the selectivity factor initialization value is zero.
Wherein, can represent with File inode number for the file identification of file allocation; The sign of each data block can represent that stripe id distributes according to data block order hereof with stripe id, and in distributed file system, file generally carries out piecemeal according to preset size; The copy amount of data block can be represented with replication count; Usually each data block needs to be preserved many parts; Be copy amount, this copy amount can be set by the keeper, and the different copies of common same data block are kept on the different storage device of different nodes; When guaranteeing certain memory device fault, do not influence from other storage and read this data block; Selectivity factor representes that with choice factor its initial value is 0, each data block being carried out cryptographic hash when calculating, can add an adjustment as required, so as on consistency Hash ring corresponding dummy node.
Step 204: form each data block of this document for order, the data block sign of the file identification of this document, data block and selectivity factor are carried out Hash operation obtain cryptographic hash.
Step 205: search consistency Hash ring according to cryptographic hash, obtain value, begin on consistency Hash ring, to select clockwise the dummy node consistent with copy amount from this dummy node greater than the minimum dummy node of cryptographic hash from consistency Hash ring.
Step 206: when selected dummy node belongs to different storage devices respectively, and the user mode of different storage devices is when being upstate, data block sent on the different storage devices store.
Further; The user mode that at least two dummy nodes in the dummy node of said selection belong to same data memory node or certain memory device is not upstate; Then can the value of selectivity factor be added one, return step 204 then, calculate cryptographic hash with the value of current selectivity factor; Belong to different storage devices respectively until selected dummy node, and the user mode of different storage devices is a upstate.At this moment, client can send to metadata node with the value of current selectivity factor, the value of the selectivity factor of recorded data piece is updated to the value of this current selectivity factor by metadata node.
Referring to Fig. 3 A, be the second embodiment flow chart of the application based on the document handling method of distributed file system, this embodiment shows the process that generates consistency Hash ring in the distributed file system:
Step 301: safeguard the chained list of memory device in the metadata node, the device identification of this memory device is added in this chained list according to the time backward that adds distributed file system.
In distributed file system; Comprised the plurality of data memory node, comprised some memory devices on each memory node, each memory device is assigned device identification; Safeguard the chained list of a memory device in the metadata node; Write down the equipment mark of all memory devices in this chained list, these device identifications are added in the chained list according to the time backward that equipment adds system, shown in Fig. 3 B; Chained list tail place is the memory device 1 that adds system at first, and the linked list head place is the memory device N of up-to-date adding system.
Step 302: when this distributed file system starts, and reach Preset Time after starting at interval the time, metadata node receives the storaging state information that this data memory node reports at every turn.
When the distributed file system initial start, need the storaging state information of the memory device of all data memory nodes in the system be reported metadata node; After system start-up, the state of the memory device in the system in the memory node possibly change, and therefore the storaging state information that is reported memory device according to the preset time interval by data memory node to metadata node can be set in advance.
Wherein, the storaging state information that reports of data memory node can comprise: device identification, place capacity, the equipment of forming each memory device of this memory node have been used the capacity and the possibility factor; Wherein, device identification is used for a memory device of unique identification system; Place capacity is represented the total storage capacity size of this memory device; Equipment representes that with capacity this memory device used the size of memory space; Can confirm the user mode of memory device with capacity according to this; For example, if the ratio that has accounted for total storage capacity with capacity greater than preset ratio, then the user mode of memory device is " expiring state "; If less than preset ratio, then user mode is " upstate "; The possibility factor is used to represent the performance of this memory device, can use the numeric representation between 0 to 1 usually, for example; High speed storing equipment such as solid state hard disc etc., its memory property is higher, and then the possibility factor can be set to " 1 "; Low speed storage device such as SATA hard disk etc., its memory property is lower, and then the possibility factor can be set to " 0.1 "; When carrying out write operation, can select the higher memory device of memory property according to the possibility factor, to improve writing speed.
Step 303: metadata node compares according to the device identification of memory device in the storaging state information and the device identification in the chained list.
Step 304: the device identification backward of the memory device that increases newly is added in this chained list according to comparative result.
When having increased memory device newly in the distributed system, then can newly-increased memory device backward be added in the chained list according to its order that reports that belongs to data memory node.
Step 305: this chained list of metadata node traversal generates the consistency Hash ring based on dummy node according to the place capacity of each memory device in the chained list, and be this consistency Hash ring distribution version number.
Wherein, each memory device is according at least one dummy node on the corresponding said consistency Hash ring of its memory capacity size, according to the value that each dummy node distributes dummy node that adds up to of said dummy node.
Suppose to comprise in the distributed file system three data memory nodes, be respectively NodeA, NodeB and NodeC, the memory device (Storage Device) and the capacity of each memory device that comprise in above-mentioned three memory nodes are as shown in table 1 below:
Table 1
Based on the shown data memory node of above-mentioned table 1, suppose that the maximum that hash function shines upon is 1000 on consistency Hash ring, then each memory device is mapped on this consistency Hash ring according to the size of its memory capacity form with dummy node; Corresponding at least one dummy node (Vnode) of each memory device, each dummy node is according to the same intervals assign a value, and any two adjacent virtual nodes do not belong to same memory device; Referring to Fig. 3 B, be the structural representation of the consistency Hash ring that generates according to the listed memory device of last table 1: the value that wherein Vnode1 distributed is 100, and it belongs to memory device A1; Clockwise, the value that Vnode2 distributed is 200, and it belongs to memory device A2; Until the value that Vnode10 distributed is 1000; It belongs to memory device C1, can be known by Fig. 3 C, and the capacity of memory device is big more; The quantity of dummy node that then belongs to this memory device is many more; For example, the only corresponding dummy node of memory device A1, then corresponding three dummy storage node of memory device B1.
Each dummy node on the consistency Hash ring is safeguarded an information list; Comprised the device identification of the memory device under this dummy node in this information list; The place capacity of memory device, the equipment of memory device has been used capacity, the user mode and the possibility factor.
For user mode, can be divided into upstate, expire state and down state.Wherein, Upstate and expired state and can be provided with the ratio that capacity accounts for place capacity according to equipment; The information of the memory device that down state can report according to data memory node, perhaps the state of data memory node carries out equipment in the system, such as the disk failures of certain memory device; Perhaps the data memory node at its place rolls off the production line, and the user mode that this memory device then can be set is a down state.When the user mode of dummy node when expiring state, the disk space of the memory device that it is corresponding can not distribute again, but the data on it can read; When the user mode of dummy node is a down state, then the disk space of corresponding memory device can not distribute again, and the data on it also cannot read.
Referring to Fig. 4, be the three embodiment flow chart of the application based on the document handling method of distributed file system, this embodiment shows the process that writes file in the distributed file system:
Step 401: client is sent the version number of the current consistency Hash ring of preserving to metadata node.
After the memory device in the system increased, metadata node generated new consistency Hash ring, and was that this new consistency Hash ring distributes new version number.When client write file operation in execution, needs need to abide by new consistency Hash ring to carry out, and therefore in order to know whether upgraded consistency Hash ring in the metadata node, client is sent the version number of current saved to metadata node.
Step 402: whether metadata node relatively this version number is consistent with the current version number that has preserved.
After metadata node receives the version number of client transmission, compare,, explain that then consistency Hash ring does not upgrade,, explain that then consistency Hash ring upgrades if inconsistent if consistent with the version number that self preserves.
Step 403: when inconsistent, consistency Hash ring and the version number corresponding with the current version number that has preserved are sent to client.
Step 404: preserve the consistency Hash ring based on dummy node that receives in the client, each dummy node is preserved device identification, user mode and the selectivity factor of the affiliated memory device of this dummy node.
Common each virtual information list of node maintenance that connects; Like the description among aforementioned second embodiment; The device identification of the memory device in this information list under comprising this dummy node, user mode and the selectivity factor, the place capacity that can also comprise memory device has been used capacity with the equipment of the memory device that regularly upgrades.
Step 405: client is sent file to metadata node and is write request.
Step 406: receive metadata node and write the file identification that request is returned for this document distribution according to file; And form the data block sign of each data block of this document, the copy amount and the selectivity factor of data block, the selectivity factor initialization value is zero.
Wherein, can represent with File inode number for the file identification of file allocation; The sign of each data block can represent that stripe id distributes according to data block order hereof with stripe id, and in distributed file system, file generally carries out piecemeal according to preset size; The copy amount of data block can be represented with replication count; Usually each data block needs to be preserved many parts; Be copy amount, this copy amount can be set by the keeper, and the different copies of common same data block are kept on the different storage device of different nodes; When guaranteeing certain memory device fault, do not influence from other storage and read this data block; Selectivity factor representes that with choice factor its initial value is 0, each data block being carried out cryptographic hash when calculating, can add an adjustment as required, so as on consistency Hash ring corresponding dummy node.
Step 407: form each data block of this document for order, the data block sign of the file identification of this document, data block and selectivity factor are carried out Hash operation obtain cryptographic hash.
Step 408: search consistency Hash ring according to cryptographic hash, obtain value, begin on consistency Hash ring, to select clockwise the dummy node consistent with copy amount from this dummy node greater than the minimum dummy node of cryptographic hash from consistency Hash ring.
After obtaining cryptographic hash; Can specifically adopt dichotomy on consistency Hash ring, to search value greater than the minimum dummy node of this cryptographic hash; Because the dummy node on the consistency Hash ring is arranged according to clockwise direction; Therefore begin to obtain clockwise the dummy node consistent from this minimum value corresponding virtual node,, can know memory device, user mode and the possibility factor that each dummy node is affiliated according to the information list of safeguarding on the dummy node with copy amount.
Step 409: judge whether the corresponding memory device of selected dummy node satisfies three preset conditions, if then execution in step 411; Otherwise, execution in step 410.
Wherein, three preset conditions comprise:
1, the pairing memory device of selected dummy node is all inequality; Have only the corresponding memory device of dummy node all inequality, could guarantee that the copy of data block can not be kept on the same memory device purpose of redundancy backup in the assurance system thus.
The user mode of the memory device that 2, selected dummy node is corresponding is upstate; The user mode of having only memory device is that upstate could support to write data, has expired state and down state and has not all supported to write data.
3, the selectivity factor of selected dummy node is 1 according to the random function result calculated; Because each memory device is provided with selectivity factor according to the height of its memory property; The application embodiment can adopt following random algorithm: all provide random function random in the programming language usually, utilize this function can produce the random number between 0~1, the hypothesis possibility factor is C in the present embodiment; Then C is imported above-mentioned random function random; Export a result of calculation R, if R less than C then return value 1, otherwise return value 0.
Step 410: the value of current selectivity factor is added one, return step 407.
When at least one condition in above-mentioned three conditions does not satisfy, need selectivity factor be added one, and recomputate cryptographic hash, till satisfying above-mentioned three conditions according to this new selectivity factor.
Step 411: data block sent on the corresponding different storage devices of the dummy node of selection store.
When above-mentioned three conditions all satisfy; Be that selected dummy node belongs to different storage devices respectively; And the user mode of different storage devices is a upstate; And the stochastic arithmetic value of the possibility factor of different storage devices is to be 1 o'clock, can data block be dealt on the pairing memory device of the dummy node consistent with copy amount to store.
Step 412: client sends to metadata node with the value of the current selectivity factor of the data block of composing document.
Step 413: metadata node is updated to the initialization value of the selectivity factor of data block the value of the current selectivity factor that receives.
Referring to Fig. 5, be the four embodiment flow chart of the application based on the document handling method of distributed file system, this embodiment shows the process that reads file in the distributed file system:
Step 501: client is sent file to metadata node and is read request, and this document reads include file sign in the request.
Step 502: metadata node is obtained the data block sign of each data block of forming this document, the copy amount of data block and the value of selectivity factor according to file identification.
Step 503: metadata node sends to client with the data block sign and the selectivity factor of file identification, each data block.
Step 504: for each data block of forming this document in proper order, client is according to file identification, and the value of the data block of each data block sign and selectivity factor is carried out Hash operation and obtained cryptographic hash.
Wherein, the Hash operation that the Hash operation that client adopted is adopted when writing data is consistent, to guarantee that according to file identification the cryptographic hash that the value of the data block of each data block sign and selectivity factor calculates is constant.
Step 505: client is searched consistency Hash ring according to this cryptographic hash; Obtain value from consistency Hash ring, begin on consistency Hash ring, to select clockwise the dummy node consistent with copy amount from this dummy node greater than the minimum dummy node of this cryptographic hash.
Step 506: client is according to the dummy node that finds, read block to the corresponding memory device of one of them dummy node.
For each data block; When carrying out write operation; All preserve, so after finding the memory device consistent, can therefrom select a memory device read block to get final product with copy amount through the memory device consistent with copy amount; For example, can select greater than the corresponding memory device of the value of the minimum dummy node of cryptographic hash.
Step 507: client is formed said file according to the data block sign of the data block that reads with all data blocks in order.
Visible by the foregoing description; Owing to preserved consistency Hash ring in the client; Therefore when execution writes file operation, need not metadata node and preserve the distribution situation of file on the memory device of each memory node through the file layout allocation table in advance, file allocation and the positioning action originally on metadata node, carried out are moved to client executing; Metadata node only needs the selectivity factor of transaction file data block, has therefore reduced taking the metadata node memory space; Because each client only need be when self carries out write operation be carried out the location of memory device according to consistency Hash ring; Rather than position by the request that writes of metadata node to all clients in the system, therefore can improve the locating speed when writing data; And, thereby through the user mode of the corresponding memory device of each dummy node on the consistency Hash ring being judged self adaptation carries out the selection of memory device, realized thus making system reach load balancing according to the residual capacity of disk.
Corresponding based on the embodiment of the document handling method of distributed file system with the application, the application also provides the embodiment of distributed file system and client.
Referring to Fig. 6, be the embodiment block diagram of the application's distributed file system:
This distributed file system comprises: metadata node 610, with client 620 and data memory node 630 that metadata node 610 is communicated by letter respectively, each memory node 630 is made up of at least one memory device 631.Convenient for example, only show a client 610 and three memory nodes 630 among Fig. 6, and show a memory device, two memory devices and three memory devices in each memory node respectively.Need to prove that in actual application, the quantity of the memory device on client, data memory node and each data memory node can be provided with as required, the setting that the embodiment among Fig. 6 is not used in system limits.
Wherein, Said client 620; Be used to preserve consistency Hash ring based on dummy node; Wherein each memory device corresponds at least one dummy node on the said consistency Hash ring according to the size of its memory capacity, and each dummy node is preserved the device identification and the user mode of the memory device under the said dummy node, according to the value that each dummy node distributes dummy node that adds up to of said dummy node; And send file to metadata node and write request;
Said metadata node 610; Be used for writing request and be the file identification of said file allocation to what said client was returned according to said file; And form the data block sign of each data block of said file, the copy amount and the selectivity factor of data block, said selectivity factor initialization value is zero;
Said client 620 also is used for forming each data block of said file for order, the data block sign of the file identification of said file, said data block and selectivity factor is carried out Hash operation obtain cryptographic hash; Search said consistency Hash ring according to said cryptographic hash; Obtain value from said consistency Hash ring, begin on said consistency Hash ring, to select clockwise the dummy node consistent with said copy amount from said dummy node greater than the minimum dummy node of said cryptographic hash; When selected dummy node belongs to different storage devices respectively, and the user mode of said different storage devices sends to data block on the said different storage devices when being upstate;
Said memory device 631 is used for the data block that said client is sent is preserved.
Further, said metadata node 610 also is used to safeguard the chained list of memory device, and the device identification of said memory device is added in the said chained list according to the time backward that adds distributed file system;
Said data memory node 630; Be used for when said distributed file system starts; And reach Preset Time after starting at interval the time at every turn; Report the storaging state information of said data memory node, said storaging state information to comprise that device identification, place capacity, the equipment of each memory device of forming said memory node used the capacity and the possibility factor to said metadata node;
Said metadata node 610 also is used for comparing according to the device identification of storaging state information memory device and the device identification in the chained list, and the device identification backward of the memory device that increases newly is added in the said chained list; Travel through said chained list, and generate said consistency Hash ring, and be that said consistency Hash ring distributes version number based on dummy node according to the place capacity of said each memory device; Said consistency Hash ring and said version number are sent to said client.
Further, said client 620 also was used for before metadata node transmission file writes request the version number that sends the current consistency Hash ring of preserving to said metadata node;
Said metadata node 610, whether with the current version number that preserved consistent, when inconsistent, consistency Hash ring and the version number corresponding with the said current version number that has preserved are sent to said client if also being used for more said version number.
Further; Said client 620; Also be used for also having preserved the possibility factor of the memory device under the said dummy node, then belong to different storage devices respectively, and the user mode of said memory device is when being upstate at dummy node when said selection when each dummy node; Whether the stochastic arithmetic value of judging the possibility factor of said memory device is 1, if be 1 execution data block sent on the said different storage devices store.
Further; Said client 620; Also be used for user mode that at least two dummy nodes when the dummy node of said selection belong to same data memory node or said memory device for the stochastic arithmetic value of the possibility factor of upstate or said memory device is not 1, then the value with said selectivity factor adds one; Returning execution carries out Hash operation to the data block of the file identification of said file, said data block sign and selectivity factor and obtains cryptographic hash; Dummy node until said selection belongs to different storage devices respectively; And the user mode of said different storage devices is a upstate, and the stochastic arithmetic value of the said possibility factor is 1.
Further, said client 620, also be used for when said data block is sent to store on the said different storage devices after, the value of current selectivity factor is sent to said metadata node;
Said metadata node 610 also is used for the initialization value of the selectivity factor of said data block is updated to the value of said current selectivity factor.
Further, said client 620 also is used for sending file to said metadata node and reads request, and said file reads include file sign in the request;
Said metadata node 610 also is used for obtaining the data block sign of each data block of forming said file, the copy amount of data block and the value of selectivity factor according to said file identification; The data block sign and the selectivity factor of said file identification, each data block are sent to client;
Said client 620 also is used for for each data block of forming said file in proper order, and according to said file identification, the value of the data block of each data block sign and selectivity factor is carried out Hash operation and obtained cryptographic hash; Search said consistency Hash ring according to said cryptographic hash; Obtain value from said consistency Hash ring, begin on said consistency Hash ring, to select clockwise the dummy node consistent with said copy amount from said dummy node greater than the minimum dummy node of said cryptographic hash; According to the dummy node that finds, to the corresponding memory device of one of them dummy node, read said data block; Data block sign according to the data block that reads is formed said file in order with all data blocks.
Referring to Fig. 7, be the embodiment block diagram of the application's client:
This client application is in the distributed file system shown in the application embodiment; The framework of this system can be referring to system embodiment shown in Figure 6; Comprise metadata node; Said metadata node is communicated by letter with data memory node with said client respectively, and each back end is made up of at least one memory device.This client comprises: preserve unit 710, transmitting element 720, receiving element 730, arithmetic element 740 and acquiring unit 750.
Wherein, Preserve unit 710; Be used to preserve consistency Hash ring based on dummy node; Wherein each memory device corresponds at least one dummy node on the said consistency Hash ring according to the size of its memory capacity, and each dummy node is preserved the device identification and the user mode of the memory device under the said dummy node, according to the value that each dummy node distributes dummy node that adds up to of said dummy node;
Transmitting element 720 is used for sending file to metadata node and writes request;
Receiving element 730; Be used to receive metadata node and be the file identification of said file allocation according to what said file write that request returns; And form the data block sign of each data block of said file, the copy amount and the selectivity factor of data block, said selectivity factor initialization value is zero;
Arithmetic element 740 is used for forming each data block of said file for order, the data block sign of the file identification of said file, said data block and selectivity factor is carried out Hash operation obtain cryptographic hash;
Acquiring unit 750; Be used for searching said consistency Hash ring according to said cryptographic hash; Obtain value from said consistency Hash ring, begin on said consistency Hash ring, to select clockwise the dummy node consistent with said copy amount from said dummy node greater than the minimum dummy node of said cryptographic hash;
Transmitting element 720 also is used for belonging to different storage devices respectively when selected dummy node, and the user mode of said different storage devices is when being upstate, data block is sent on the said different storage devices store.
Further, said receiving element 730 also is used for when said distributed file system starts, and reaches Preset Time after starting at interval the time at every turn, receives the consistency Hash ring that said metadata node sends and the version number of said consistency Hash ring; Wherein, Said metadata node be used to safeguard memory device chained list; The device identification of said memory device is added in the said chained list according to the time backward that adds distributed file system; When said distributed file system starts; Receive the storaging state information that said data memory node reports, said storaging state information comprises that device identification, place capacity, the equipment of each memory device of forming said memory node used the capacity and the possibility factor, compares according to the device identification of memory device in the storaging state information and the device identification in the chained list; The device identification backward of the memory device that increases newly is added in the said chained list; Travel through said chained list, and generate said consistency Hash ring, and be that said consistency Hash ring distributes version number based on dummy node according to the place capacity of said each memory device;
Said transmitting element 720 also was used for before metadata node transmission file writes request the version number that sends the current consistency Hash ring of preserving to said metadata node;
Said receiving element 730 also is used in the more said version number of said metadata node and the current version number that has preserved when inconsistent, receive that said metadata node sends with corresponding consistency Hash ring and version number of the said current version number that has preserved.
Further; This client can also comprise (not shown among Fig. 7): judging unit; The possibility factor that is used for the memory device under said each dummy node is also preserved said dummy node; And when the dummy node of said selection belongs to different storage devices respectively, and the user mode of said memory device judges whether the stochastic arithmetic value of the possibility factor of said memory device is 1 when being upstate;
Said transmitting element 720 also is used for when the said stochastic arithmetic value of said judgment unit judges is 1, data block is sent on the said different storage devices store.
Further; Said arithmetic element 740; Also being used for user mode that at least two dummy nodes when the dummy node of said selection belong to same data memory node or said memory device is not 1 for the stochastic arithmetic value of the possibility factor of upstate or said memory device; Then the value with said selectivity factor adds one; And return execution and the data block of the file identification of said file, said data block sign and selectivity factor are carried out Hash operation obtain cryptographic hash; Dummy node until said selection belongs to different storage devices respectively, and the user mode of said different storage devices is upstate, and the stochastic arithmetic value of the said possibility factor is 1.
Further; Said transmitting element 720; Also be used for when said transmitting element said data block is sent to store on the said different storage devices after; The value of current selectivity factor is sent to said metadata node, so that said metadata node is updated to the initialization value of the selectivity factor of said data block the value of said current selectivity factor.
Further, said transmitting element 720 also is used for sending file to said metadata node and reads request, and said file reads include file sign in the request;
Said receiving element 730 also is used to receive data block sign, the copy amount of data block and the value of selectivity factor of each data block of the said file of composition that said metadata node sends obtains according to said file identification;
Said arithmetic element 740 also is used for for each data block of forming said file in proper order, and client is according to said file identification, and the value of the data block of each data block sign and selectivity factor is carried out Hash operation and obtained cryptographic hash;
Said acquiring unit 750; Also be used for searching said consistency Hash ring according to said cryptographic hash; Obtain value from said consistency Hash ring, begin on said consistency Hash ring, to select clockwise the dummy node consistent with said copy amount from said dummy node greater than the minimum dummy node of said cryptographic hash;
Said client also comprises (not shown among Fig. 7): reading unit; According to the dummy node that finds; To the corresponding memory device of one of them dummy node, read said data block, and all data blocks are formed said file in order according to the data block sign of the data block that reads.
Description through to above execution mode can be known; Send file by client to metadata node among the application embodiment and write request; Receive metadata node and write the file identification that request is returned for this document distribution according to file; And copy amount and the initialization value of forming data block sign, the data block of each data block of said file be zero selectivity factor, forms each data block of said file for order, the data block sign of the file identification of file, data block and selectivity factor carried out Hash operation obtain cryptographic hash; Search the consistency Hash ring of preserving on the client according to cryptographic hash; Obtain value, begin on consistency Hash ring, to select the dummy node consistent clockwise from this dummy node, when selected dummy node belongs to different storage devices respectively with copy amount greater than the minimum dummy node of this cryptographic hash; And when the user mode of different storage devices is upstate, data block is sent on the different storage devices store.In the distributed file system of the application embodiment; Owing to preserved consistency Hash ring in the client; Therefore when execution writes file operation; Need not metadata node and preserve the distribution situation of file on the memory device of each memory node through the file layout allocation table in advance; File allocation and the positioning action originally on metadata node, carried out are moved to client executing, and metadata node only needs the selectivity factor of transaction file data block, has therefore reduced taking the metadata node memory space; Because each client only need be when self carries out write operation be carried out the location of memory device according to consistency Hash ring; Rather than position by the request that writes of metadata node to all clients in the system, therefore can improve the locating speed when writing data; And, thereby through the user mode of the corresponding memory device of each dummy node on the consistency Hash ring being judged self adaptation carries out the selection of memory device, realized thus making system reach load balancing according to the residual capacity of disk.
The technology that those skilled in the art can be well understood in the embodiment of the invention can realize by the mode that software adds essential general hardware platform.Based on such understanding; The part that technical scheme in the embodiment of the invention contributes to prior art in essence in other words can be come out with the embodied of software product; This computer software product can be stored in the storage medium, like ROM/RAM, magnetic disc, CD etc., comprises that some instructions are with so that a computer equipment (can be a personal computer; Server, the perhaps network equipment etc.) carry out the described method of some part of each embodiment of the present invention or embodiment.
Each embodiment in this specification all adopts the mode of going forward one by one to describe, and identical similar part is mutually referring to getting final product between each embodiment, and each embodiment stresses all is the difference with other embodiment.Especially, for system embodiment, because it is basically similar in appearance to method embodiment, so description is fairly simple, relevant part gets final product referring to the part explanation of method embodiment.
Above-described embodiment of the present invention does not constitute the qualification to protection range of the present invention.Any modification of within spirit of the present invention and principle, being done, be equal to replacement and improvement etc., all should be included within protection scope of the present invention.