CN107145302A

CN107145302A - A kind of method and apparatus for being used to perform file write-in in distributed memory system

Info

Publication number: CN107145302A
Application number: CN201710115370.3A
Authority: CN
Inventors: 刘善阳; 张海勇; 石超
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2016-03-01
Filing date: 2017-03-01
Publication date: 2017-09-08
Anticipated expiration: 2037-03-01
Also published as: CN107145302B

Abstract

The purpose of the application is to provide a kind of method and apparatus for being used to perform file write-in in distributed memory system.Specifically, the block message for submitting the writeable data block on file destination to metadata management equipment is asked；Receive the metadata management equipment and the corresponding multiple writeable data blocks of user equipment return, described are asked based on the block message；File write-in is performed to one of the multiple writeable data block according to the write request on the file destination.Compared with prior art, the multiple processes of the application write to the different pieces of information block of file so that write-in handling capacity increase, reduction write-in focus probability of happening simultaneously；Run into after file write-in failure, can not write state by data management apparatus, metadata management renewal of the equipment data storage block, user equipment, which deletes not writeable data block, selects other data blocks to re-write, increase so as to avoid the occurrence of the write-in delay caused by write-in failure, the probability that small data block occurs is reduced, mitigates user management burden.

Description

A kind of method and apparatus for being used to perform file write-in in distributed memory system

Technical field

The application is related to computer realm, more particularly to a kind of for performing file write-in in distributed memory system Technology.

Background technology

With the popularization of internet, data service is stepped up, therefore the requirement to distributed memory system is also progressively carried Height, especially has certain requirement to the handling capacity that file writes, in current distributed memory system, generally requires support one Individual file simultaneously by it is multiple enter degree read and write, that is, allow multiple processes simultaneously to a file write data, its data it is consistent Property be to give one of data equipment management end node to be responsible for, the Azure systems of such as Microsoft are to employ such arrange Apply.

However, prior art can because each file only one of which correspondence can write data data block so that handling capacity by Limit, while when multiple client writes to a file simultaneously, if write-in delay can be caused to increase when running into write-in failure, The availability of reduction system, and running into can cause the length of data block to be less than desired value, and the number of small data block after write-in failure Amount increase causes management cost increase.

The content of the invention

The purpose of the application be to provide it is a kind of be used in distributed memory system perform file write-in method with Equipment, to solve, monofile multi-process write-in handling capacity is low in distributed memory system and the caused system of write-in failure is unstable The problem of determining.

According to the one side of the application, it is used to deposit in distribution in metadata management equipment end this application provides one kind The method that file write-in is performed in storage system, this method solve monofile multi-process in distributed memory system and writes handling capacity The problem of system caused by low and write-in failure is unstable, this method includes：

Obtain the block message request for the writeable data block on file destination that user equipment is submitted；

The corresponding multiple writeable numbers of the user equipment are determined according to the corresponding writeable number of blocks threshold value of the file destination According to block, wherein, the multiple writeable data block includes the writeable data for having distributed to the other users equipment of the file destination Block or newly assigned writeable data block；

The multiple writeable data block is back to the user equipment.

According to further aspect of the application, it is used for this application provides one kind at user equipment end in distributed storage system The method that file write-in is performed in system, this method solve in distributed memory system monofile multi-process write-in handling capacity it is low with And the problem of caused system of failure is unstable is write, this method includes：

The block message request of the writeable data block on file destination is submitted to metadata management equipment；

Receive the metadata management equipment is returned based on the block message request, user equipment corresponding multiple Writeable data block；

The write request on the file destination is sent with to the multiple writeable number to correspondence master data management equipment File write-in is performed according to one of block.

According to further aspect of the application, it is used for this application provides one kind in master data management equipment end in distribution The method that file write-in is performed in storage system, this method solve monofile multi-process write-in in distributed memory system and handles up Amount is low and writes the problem of caused system of failure is unstable, and this method includes：

The request that file write-in is performed to data block that user equipment is sent is received, and performs the file to the data block Write-in；

If occurring the failure that file write-in is performed to the data block, failure information is reported to the user equipment, and refuse The other users equipment of identical file corresponding with the user equipment absolutely performs the request of file write-in to the data block；

The state for updating the data block is not writeable, and reports that the data block updates to corresponding element data management apparatus State afterwards.

According to the another aspect of the application, it is used to perform file in distributed memory system this application provides one kind The metadata management equipment of write-in, the equipment solve in distributed memory system monofile multi-process write-in handling capacity it is low and The problem of system caused by write-in failure is unstable, the equipment includes：

Block message acquisition request device, the block of the writeable data block on file destination for obtaining user equipment submission Information request；

Writeable data block determining device, for determining the use according to the corresponding writeable number of blocks threshold value of the file destination The corresponding multiple writeable data blocks of family equipment, wherein, the multiple writeable data block includes having distributed to the file destination The writeable data block of other users equipment or newly assigned writeable data block；

Block message feedback device, for the multiple writeable data block to be back into the user equipment.

According to another aspect of the application, it is used to perform file in distributed memory system this application provides one kind The user equipment of write-in, the equipment solves the problem of monofile write-in handling capacity is low in distributed memory system, the equipment Including：

Block message send-request unit, for submitting the writeable data block on file destination to metadata management equipment Block message is asked；

Block message receiving device, for receiving the metadata management equipment is returned based on block message request, institute State the corresponding multiple writeable data blocks of file destination；

Data block write request dispensing device, for being sent to correspondence master data management equipment on the file destination Write request is write with performing file to one of the multiple writeable data block.

According to another aspect of the application, it is used to deposit in distribution in master data management equipment this application provides one kind The method that file write-in is performed in storage system, this method solve monofile multi-process in distributed memory system and writes handling capacity The problem of system caused by low and write-in failure is unstable, this method includes：

File writes performs device, the request that file write-in is performed to data block for receiving user equipment transmission, and Perform and the file of the data block is write；

Failure handling device is write, if for the failure for occurring to perform the data block file write-in, to the user Device report failure information, and refuse the other users equipment of corresponding with the user equipment identical file data block is held The request that part of composing a piece of writing writes；

Data block status updating device, for updating the state of the data block to be not writeable, and to corresponding element data pipe Manage the state after data block updates described in device report.

According to another aspect of the application, it is used to perform file in distributed memory system this application provides one kind The metadata management equipment of write-in, wherein, the equipment includes：

Processor；

And it is arranged to store the memory of computer executable instructions, the executable instruction makes institute when executed State processor：

The multiple writeable data block is back to the user equipment.

According to another aspect of the application, it is used to perform file in distributed memory system this application provides one kind The user equipment of write-in, wherein, the equipment includes：

Processor；

According to another aspect of the application, it is used to perform file in distributed memory system this application provides one kind The master data management equipment of write-in, wherein, the equipment includes：Processor；

According to the another further aspect of the application, additionally provide a kind of realize and file write-in is performed in distributed memory system System, solve in distributed memory system that monofile multi-process write-in handling capacity is low and write-in failure caused by system it is unstable The problem of, wherein, the system includes according to one kind of one aspect of the invention being used in distributed memory system as previously described The metadata management equipment of file write-in is performed, one kind according to a further aspect of the present invention is used to deposit in distribution as previously described The user equipment of file write-in is performed in storage system, and one kind according to a further aspect of the present invention is used to divide as previously described The master data management equipment of file write-in is performed in cloth storage system.

Compared with prior art, the application submits the block of the writeable data block on file destination to metadata management equipment Information request, receives the metadata management equipment is returned based on the block message request, user equipment corresponding many Individual writeable data block, and text is performed to one of the multiple writeable data block according to the write request on the file destination Part write so that allow single file can correspond to write multiple data blocks so that multiple processes can simultaneously it is corresponding to file not Write with data block so that writing handling capacity increase, reduction writes the probability that focus occurs.Master data management equipment After occurring to perform data block file write-in failure, failure information is reported to user equipment, and refuse and the user equipment The other users equipment of correspondence identical file performs the request of file write-in to the data block, and updates the shape of the data block State is not writeable, and reports the state after the data block renewal to corresponding element data management apparatus, and user equipment end is being met Not writeable data block is directly deleted after to write-in failure, the corresponding other data blocks of selection target file re-write, it is to avoid Write-in delay caused by there is write-in failure increases, and reduces the probability that small data block occurs, and then mitigate user management burden.

Brief description of the drawings

By reading the detailed description made to non-limiting example made with reference to the following drawings, the application's is other Feature, objects and advantages will become more apparent upon：

Fig. 1 is shown according to the metadata management equipment end of the application one side, user equipment end and master data management Equipment end coordinates a kind of realize realized to perform the method flow diagram that file writes in distributed memory system；

Fig. 2 shows to perform file write-in in distributed memory system according to a kind of realize of one preferred embodiment of the application Method in step S22 flow charts；

Fig. 3 shows to perform file write-in in distributed memory system according to a kind of realize of one preferred embodiment of the application Method in step S13 flow charts；

Fig. 4 is shown according to the metadata management equipment end of the application other side, user equipment end and master data pipe Reason equipment end coordinates a kind of realize in the method for performing file write-in in distributed memory system realized to change writeable block number Measure the flow chart of threshold value；

Fig. 5 shows to be set according to metadata management equipment, user equipment and the master data management of the application other side It is standby to coordinate a kind of realize realized to perform the equipment schematic diagram that file writes in distributed memory system；

Fig. 6 shows to realize that performing file in distributed memory system writes according to a kind of of the application another preferred embodiment Writeable data block determining device schematic diagram in the equipment entered；

Fig. 7 shows to realize that performing file in distributed memory system writes according to a kind of of the application another preferred embodiment Data block write request dispensing device schematic diagram in the equipment entered；

Fig. 8 shows to coordinate the one kind realized according to the metadata management equipment and user equipment of the application other side Realize the schematic diagram that the writeable number of blocks threshold function of modification is realized in the equipment that file write-in is performed in distributed memory system.

Same or analogous reference represents same or analogous part in accompanying drawing.

Embodiment

The application is described in further detail below in conjunction with the accompanying drawings.

In one typical configuration of the application, terminal, the equipment of service network and trusted party include one or more Processor (CPU), input/output interface, network interface and internal memory.

Internal memory potentially includes the volatile memory in computer-readable medium, random access memory (RAM) and/or The forms such as Nonvolatile memory, such as read-only storage (ROM) or flash memory (flash RAM).Internal memory is computer-readable medium Example.

Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method Or technology come realize information store.Information can be computer-readable instruction, data structure, the module of program or other data. The example of the storage medium of computer includes, but are not limited to phase transition internal memory (PRAM), static RAM (SRAM), moved State random access memory (DRAM), other kinds of random access memory (RAM), read-only storage (ROM), electric erasable Programmable read only memory (EEPROM), fast flash memory bank or other memory techniques, read-only optical disc read-only storage (CD-ROM), Digital versatile disc (DVD) or other optical storages, magnetic cassette tape, magnetic disk storage or other magnetic storage apparatus or Any other non-transmission medium, the information that can be accessed by a computing device available for storage.Defined according to herein, computer Computer-readable recording medium does not include the data-signal and carrier wave of non-temporary computer readable media (transitory media), such as modulation.

Fig. 1 is shown according to the metadata management equipment end of the application one side, user equipment end and master data management Equipment end coordinates a kind of realize realized to perform the method flow diagram that file writes in distributed memory system.Wherein, it is described User equipment end includes step S11, step S12, step S13；The metadata management equipment include step S21, step S22, Step S23；The master data management equipment end includes step S31, step S32 and step S33.

Wherein, the writeable data on file destination are submitted at user equipment end to metadata management equipment in step s 11 The block message request of block；In the step s 21 metadata management equipment end obtain user equipment submit on file destination can Write the block message request of data block；Metadata management equipment end is corresponding writeable piece according to the file destination in step S22 Amount threshold determines the corresponding multiple writeable data blocks of the user equipment, wherein, the multiple writeable data block includes having divided The writeable data block of the other users equipment of file destination described in dispensing or newly assigned writeable data block；It is first in step S23 The multiple writeable data block is back to the user equipment by data management apparatus end；User equipment is terminated in step s 12 Receive the metadata management equipment and the corresponding multiple writeable data of user equipment return, described are asked based on the block message Block；In step s 13 user equipment end to correspondence master data management equipment send on the file destination write request with File write-in is performed to one of the multiple writeable data block；Master data management equipment end receives user and set in step S31 The request that file write-in is performed to data block that preparation is sent, and perform the file write-in to the data block；Lead in step s 32 If the failure that file write-in is performed to the data block occurs for data management apparatus end, unsuccessfully believe to user equipment report Cease, and refuse the other users equipment of identical file corresponding with the user equipment and data block execution file write-in is asked Ask；Master data management equipment end updates the state of the data block to be not writeable in step S33, and to correspondence metadata management State after data block renewal described in device report.

Specifically, the writeable number on file destination is submitted at user equipment end to metadata management equipment in step s 11 Asked according to the block message of block；In the step s 21 metadata management equipment end obtain that user equipment submits on file destination The block message request of writeable data block.Wherein, the user equipment end refers to be used to provide the user in distributed memory system The port module of various interfaces, by taking the distributed memory system of three end patterns as an example, the user equipment end be client i.e. Client, hereafter these three generation claim all refer to user equipment end.The metadata management equipment end refers to distributed memory system The module of metadata in middle management distributed memory system, by taking the distributed memory system of three end patterns as an example, the metadata I.e. management equipment end is Master NameSpace management modules.The file destination is the text for needing to perform write-in Part, can be pre-created but not limited to this generally in distributed memory system, the file in need for performing write-in can be into For file destination.The data block refers to one section of user logic data, is stored in the form of byte stream, the number for file destination According to write-in, by taking the distributed memory system of three end patterns as an example, data block is alleged chunk in the system.It is described on The block message of the writeable data block of file destination refer to can be used for the write-in of performance objective file current writeable data block and Its position, the information such as quantity.The method of submission and the acquisition request includes but is not limited to own by distributed memory system Interface carry out, by taking the distributed memory system of three end patterns as an example, Client client terminal start-ups, herein because be multi-process simultaneously One file is write, so being that multiple client can be write to Master transmission acquisition file destinations F is all The request of Chunk information.By above-mentioned action, metadata management equipment end can be caused to initiate to the writeable data of file destination The lookup of the block message of block, so that file writes flow startup.

Those skilled in the art will be understood that above-mentioned transmission and obtain the block message of the writeable data block on file destination The mode of request is only for example, and other existing or transmissions for being likely to occur from now on and obtains the writeable data on file destination The mode of the block message request of block is such as applicable to the application, should also be included within the application protection domain, and herein to draw It is incorporated herein with mode.

Then, in step S22 metadata management equipment end according to the corresponding writeable number of blocks threshold value of the file destination The corresponding multiple writeable data blocks of the user equipment are determined, wherein, the multiple writeable data block is described including having distributed to The writeable data block of the other users equipment of file destination or newly assigned writeable data block.I.e. will according to writeable number of blocks threshold value The writeable data block of the file destination for meeting preset number compared with the threshold value is distributed to the user equipment and write, when Some user equipmenies write to file destination simultaneously, the writeable data block list for distributing to user equipment be it is consistent, For example, the writeable number of blocks threshold value is M, then by taking the distributed memory system of three end patterns as an example, meta data block management equipment End Master receives what is found all Chunk that can be write of file destination F information T after request in internal memory and can write Chunk quantity M, it is writeable corresponding to file destination F that this T writeable data block Chunk are determined if T is more than or equal to M Data block.The writeable number of blocks threshold value refers to the threshold value of the set writeable data number of blocks of control, here, block is to refer to number According to block, this is referred to, and context is general, and the writeable number of blocks threshold value can be set according to the requirement of write-in handling capacity or speed Put, the presence of this threshold value causes user oneself need not manage the file of multichannel write-in, it is only necessary to adjusted according to the demand of write-in The quantity of writeable data block, in addition, the writeable number of blocks threshold value can also set system voluntarily to adjust according to the requirement of system, Or initially set.The corresponding multiple writeable data blocks of the user equipment refer to that each write-in file can have more than one Writeable data block, therefore the user equipment write to it also corresponds to multiple writeable data blocks, in the distribution of three end patterns In formula storage system, prior art only supports each file destination to have a writeable number in meta data block management equipment end Master , can be by the agreement the interface user equipment end and metadata management equipment end in embodiments herein according to block It is configured so that each file destination there are multiple writeable data blocks at meta data block management equipment end, so as to reach list Multiple data blocks are supported when writing for file while the purpose write so that file write-in is not only restricted to the physics money of individual machine Source, and then improve the efficiency of monofile write-in, the formation probability of reduction write-in focus.

Wherein, by the interface user equipment end and metadata management equipment end in embodiments herein The method that agreement is configured, by taking the distributed memory system of three end patterns as an example, in meta data block management equipment end Master It is middle to set as follows：In terms of user oriented equipment end Client service and agreement, for obtaining the writeable data block Chunk of file The service of information：Existing protocol is extended, increase there are multiple supports to writeable data block Chunk；For writeable data block The service of Chunk quantity：Increase sets agreement and the service of the writeable data block Chunk quantity of file, and it is writeable that increase obtains file The agreement of data block Chunk quantity and service.Set in the Client of user equipment end as follows：Set towards meta data block management In terms of standby end Master agreements：Agreement for obtaining the writeable data block Chunk information of file：Extend existing protocol, increase pair Writeable data block Chunk has multiple supports；For the interface of writeable data block Chunk quantity：Increase sets file writeable The agreement of data block Chunk quantity, increase obtains the agreement of the writeable data block Chunk quantity of file；For in user oriented Interface in terms of：For the interface of writeable data block Chunk quantity：Increase sets connecing for the writeable data block Chunk quantity of file Mouthful, increase obtains the interface of the writeable data block Chunk quantity of file.Arrangement above causes each file destination in meta data block pipe Reason equipment end has multiple writeable data blocks, and multiple data blocks are supported during monofile write-in while the mesh write so as to reach , user's request is met, while the internal memory for only consuming single file in meta data block management equipment end Master, is not reduced The quantity of cluster service user.

Those skilled in the art will be understood that above-mentioned the interface of user equipment end and metadata management equipment end Agreement is placed so that each file destination is only to lift in the mode that there are multiple writeable data blocks at meta data block management equipment end Example, other are existing or being likely to occur from now on to enter the agreement the interface of user equipment end and metadata management equipment end Row, which is set, causes each file destination to be such as applicable to this in the mode that there are multiple writeable data blocks at meta data block management equipment end Application, should also be included within the application protection domain, and be incorporated herein by reference herein.

Then, the multiple writeable data block is back to the user and set by metadata management equipment end in step S23 It is standby；User equipment end receives the metadata management equipment and asks returning, described use based on the block message in step s 12 The corresponding multiple writeable data blocks of family equipment.Determine as described above after the corresponding multiple writeable data blocks of the file destination The data block and its relevant information are back to user equipment end by metadata management equipment end, the user equipment is corresponding Multiple writeable data blocks are the writeable data block corresponding to each user equipment returned the application file destination to be write, with Exemplified by the distributed memory system of three end patterns, connect exemplified above, metadata management equipment end Master owns file destination F The quantity T and its relevant information for the data block Chunk that can be write find and fed back from internal memory, the mode of feedback include but It is not limited by distributed memory system and has interface progress by oneself.

Wherein, the quantity of the corresponding writeable data block of the file destination in the metadata management equipment end memory with And the acquisition modes of relevant information include but is not limited to obtaining each in distributed memory system after file, metadata management Equipment end be storage its it is all write and allow the data block information write, using the distributed memory system of three end patterns as Example, meta data block management equipment end Master is in terms of the metadata storage of file：It is responsible for each file of storage currently all just In the data block Chunk information write, such as locations of copies of data block；It is responsible for the currently permitted writeable number of each file of storage According to block Chunk quantity.Information above is required for carrying out the preservation of persistence at meta data block management equipment end Master ends, delays Machine will not be lost after restarting, so that needing file destination to be to find in time at user equipment end.

Those skilled in the art will be understood that the mode that above-mentioned metadata management equipment end is stored to each file data blocks It is only for example, the mode that other metadata management equipment ends that are existing or being likely to occur from now on are stored to each file data blocks The application is such as applicable to, should be also included within the application protection domain, and be incorporated herein by reference herein.

Then, user equipment end is sent on the file destination to correspondence master data management equipment in step s 13 Write request is write with performing file to one of the multiple writeable data block；The master data management equipment end in step S31 The request that file write-in is performed to data block that user equipment is sent is received, and execution writes to the file of the data block.I.e. Because the file destination there are corresponding multiple writeable data blocks, therefore, the writeable data are obtained at the user equipment end The multiple writeable data block is ranked up after block relevant information, or file destination is being received successively according to default sequence Corresponding master data tube apparatus request write-in after write request.By taking the distributed memory system of three end patterns as an example, user equipment Client is held by after corresponding writeable data block Chunk all the file destination F being connected to, according to acquiescence or set When order writes to a certain Chunk, find its corresponding i.e. Chunksever of data block server and carry out respective copies Write-in, wherein, because in the present embodiment being multi-process write-in, therefore, write request is sent to the main data block clothes dominated Business device is PrimaryChunksever, and then master data server is after write request is received, by the client received Data are ranked up, and are turned while data are locally stored to from data block server i.e. SecondaryChunksever Hair, the mode of operation of principal and subordinate's data block server may be referred to data in the bottom distributed memory system in Microsoft Azure and deposit The mode of operation of node is stored up, principal and subordinate's data block server performs the write-in to file after write request is received.Here, being lifted The data block server of example, and Chunksever each mean the master data management equipment or from data management apparatus, upper Hereafter in three kinds of generations, claim general.The acquisition modes of Chunksever information include but is not limited to by metadata management equipment end All Chunkserver where the current writeable Chunk of Master records, this partial information exists in the form of caching, is not required to Want persistence, the Information recovering that the machine of delaying can be recorded after restarting before comes out, that is, from be responsible for storage it is each Recovered in the current all data block Chunk information write of file；The current institute of user equipment end Client storage files Have writeable Chunk information, this partial information exists in the form of caching, it is not necessary to persistence, the machine of delaying restart after from Master End is reacquired.

Those skilled in the art will be understood that above-mentioned preservation and obtain the mode of data block corresponding data block server information It is only for example, other existing or the preservation being likely to occur from now on and modes of acquisition data block corresponding data block server information The application is such as applicable to, should be also included within the application protection domain, and be incorporated herein by reference herein.

Then, if the failure that file write-in is performed to the data block occurs for master data management equipment end in step s 32, Failure information is reported to the user equipment, and refuses the other users equipment of identical file corresponding with the user equipment to institute State the request that data block performs file write-in.Wherein, said write, which unsuccessfully refers to, causes the scene for writing failure, such as network delay High or network is obstructed, and disk delay is high or can not write.Because multiple user equipmenies enter row write to identical file simultaneously Enter operation, therefore, after a certain user equipment sends write request to master data management equipment, the master data management equipment logarithm The situation of failure is write during being write according to block, then master data management equipment feeds back user equipment write-in failure, simultaneously Refuse the write request that all user equipmenies are newly initiated, so as to avoid that the data block for writing failure is initiated to write again and then caused Failure.By taking the distributed memory system of three end patterns as an example, i.e., when PrimaryChunkserver runs into failover, On the one hand, for it has been accepted that request, returns to Client failures, the write request that all Client of refusal are newly initiated.

Then, the state that master data management equipment end updates the data block in step S33 is not writeable, and to correspondingly State after data block renewal described in metadata management device report.Will currently write failure data block be set to it is not writeable State, and by this feedback of status to metadata management equipment end, so that the data block that it is stored is updated, make The data block that extended meeting causes write-in to fail after obtaining will not be again written.By taking the distributed memory system of three end patterns as an example, i.e., The Chunk states for writing failure are set to not writeable, and persistent storage by PrimaryChunkserver, while this is write The Chunk of failure state is reported to Master, it is desirable to which Master carries out persistent storage.Here, the not writeable shape of data block The setting of state is responsible for by master data management equipment end and metadata management equipment end, it is to avoid unnecessary data block switching, drop The growth for writing delay under low non-global write-in failure scene, improves system availability.Meanwhile, because avoid it is unnecessary Data block switching, reduce the ratio of the few small data block i.e. produced by switching of write-in, improve the service scale of system.

Preferably, methods described includes step S24 (not shown), in step s 24 the reception pair of metadata management equipment end The data block more new state of one of the multiple writeable data block of master data management equipment transmission is answered, wherein, the data block More new state is not writeable.The not writeable state of some data block of above-mentioned master data management equipment end transmission is received And in local persistent storage, for example, connecing exemplified above, Master receives some that PrimaryChunkserver is reported Data block is arranged to after not writeable information, local persistent storage is carried out to this state according to its requirement, so as to avoid Not writeable data block is again acquired in user equipment end so as to causing unnecessary to write failure.

Those skilled in the art will be understood that mistake is write in above-mentioned master data management equipment end and the processing of metadata management equipment end The mode for losing data block is only for example, other master data management equipment ends and metadata management existing or be likely to occur from now on The mode that miss data block is write in equipment end processing is such as applicable to the application, should also be included within the application protection domain, and It is incorporated herein by reference herein.

Preferably, methods described also includes step S14 (not shown), and user equipment end is when the multiple in step S14 When writeable data block is not writeable, the writeable data on the file destination are resubmited to the metadata management equipment The block message request of block.Refer to, when writing the data block corresponding to file destination, set if running into from metadata management The writeable data block that standby end is obtained writes or not writeable situation, then needs to resubmit acquisition to metadata management equipment The writeable data of same file destination and its request of relevant information, now, as described in hereinbefore embodiment, the step of inciting somebody to action before Re-start one time.By taking the distributed memory system of three end patterns as an example, when Client has found do not have writeable Chunk in caching When, the request for the information for obtaining all Chunk that can be write of file F is sent to Master, redistributes and obtains writeable Chunk's Quantity so that the user equipment end obtains the new writeable data block corresponding to file destination, so as to proceed file Write-in.

Preferably, methods described also includes step S15 (not shown), if user equipment end occurs to institute in step S15 The failure that data block performs file write-in is stated, the corresponding data block information of the corresponding file destination will be write unsuccessfully and deleted.Its In, the situation for writing failure includes master data management equipment and failure occurred is write to data block, also including user equipment The failure scenarios occurred between end and master data management equipment, for example, event occurs in the network for transmission information between the two Barrier.Got at user equipment end write-in failure information after, this data block information is directly deleted, so directly from Select another data block to re-start in the corresponding writeable data block of file destination acquired in metadata management equipment end to write Enter.By taking the distributed memory system of three end patterns as an example, when Client runs into write-in failure, by the corresponding Chunk of write-in failure Information is removed from the cache, but does not change this Chunk state, and then is pressed in current writeable Chunk all from caching The next Chunk of sequential selection is retried, so as to avoid, and reduces the delay of writing under non-global write-in failure scene Increase.

Fig. 2 shows to perform file write-in in distributed memory system according to a kind of realize of one preferred embodiment of the application Method in step S22 flow charts.Wherein, the step S22 includes step S221, step S222, step S223.

Wherein, metadata management equipment end determines the corresponding all writeable data of the file destination in step S221 Block；If the quantity of metadata management equipment end all writeable data blocks is equal to or more than the target in step S222 The corresponding writeable number of blocks threshold value of file, then using all writeable data blocks as the user equipment it is corresponding it is multiple can Write data block；Otherwise, metadata management equipment end is that the file destination distributes other writeable data blocks in step S223, And other described writeable data blocks and all writeable data blocks are corresponding multiple writeable as the user equipment Data block, wherein, the quantity of the multiple writeable data block is equal to writeable number of blocks threshold value.

Specifically, metadata management equipment end determines the corresponding all writeable numbers of the file destination in step S221 According to block.Refer to all writeable data block informations for determining the file destination, the method for acquisition is included as described above from described The quantity of the currently permitted writeable data block of each file in the metadata information that metadata management equipment end is stored, i.e., By taking the distributed memory system of three end patterns as an example, because meta data block management equipment end Master is stored in the metadata of file Aspect：It is responsible for the current all data block Chunk information write of each file of storage, the locations of copies of such as data block；It is negative The currently permitted writeable data block Chunk of each file of duty storage quantity, therefore target can be found from Master ends File F all writeable data blocks, it is assumed that the file destination F found all writeable data blocks are T.

Then, if the quantity of metadata management equipment end all writeable data blocks is equal to or greatly in step S222 In the corresponding writeable number of blocks threshold value of the file destination, then all writeable data blocks are regard as the user equipment pair The multiple writeable data blocks answered.Will according to the writeable number of blocks threshold value of system default or user according to set by being required write-in with All writeable data numbers of blocks of the file destination obtained are compared, if all writeable data block numbers of the file destination Amount not less than the writeable number of blocks threshold value then by all writeable data blocks and its relevant information of the file destination directly by Metadata management equipment end feeds back to user equipment end, is allowed to need the file destination write corresponding as user equipment Writeable data block is write.By taking the distributed memory system of three end patterns as an example, if current goal file F is existing Chunk quantity is not less than writeable number of blocks threshold value M, then these Chunk information is returned into Client.

Then, if the quantity of all writeable data blocks is less than the corresponding writeable number of blocks threshold of the file destination Value, metadata management equipment end is that the file destination distributes other writeable data blocks in step S223, and will it is described its His writeable data block and all writeable data blocks as the corresponding multiple writeable data blocks of the user equipment, its In, the quantity of the multiple writeable data block is equal to writeable number of blocks threshold value.Will be according to system default or user according to write-in It is required that all writeable data numbers of blocks of file destination of the set writeable number of blocks threshold value with being obtained are compared, if institute All writeable data numbers of blocks for stating file destination are less than the writeable number of blocks threshold value, then again by metadata management equipment end Distribute the data block that difference quantity is distributed from the data block for storing or being newly released so that the writeable number of the file destination It is equal with the writeable number of blocks threshold value according to number of blocks, so be sent to each user equipment as user equipment it is corresponding it is some can Write data block.By taking the distributed memory system of three end patterns as an example, if the existing Chunk quantity of current file F is less than writeable Number of blocks threshold value M, then redistribute sufficient amount of Chunk, and newly assigned Chunk quantity adds existing Chunk quantity to be equal to All Chunk information are returned to Client by writeable number of blocks threshold value M.Here, it is each because monofile is multi-process write-in, because This, each Client retains the corresponding writeable Chunk of relatively independent file destination of a completion information, and the later stage is each Client each safeguards the Chunk information in each caching, and it is each between renewal time it is different, over time can There is little bit different.By the above-mentioned means, can be controlled by writeable number of blocks threshold value so that writeable data block is maintained necessarily Value, so that user need not be manually controlled file write-in, reduces the management cost of user.

Those skilled in the art will be understood that the mode of the writeable data block of above-mentioned determination file destination correspondence is only for example, its He is such as applicable to the application at the mode of the corresponding writeable data block of file destination that is existing or being likely to occur from now on, should also include Within the application protection domain, and it is incorporated herein by reference herein.

Preferably, it is described to distribute other writeable data blocks for the file destination and include：It is currently right according to each file That answers is writing data block and is each writing the corresponding block server of data block, is that the file destination distributes other writeable data Block, wherein, other each corresponding block servers of writeable data block are with being each less than writing the corresponding block server of data block Or repeat threshold value equal to predetermined block server.I.e. in the data block server corresponding to the new writeable data block of distribution, to the greatest extent Amount avoids its allocated data block server from being recorded in metadata management equipment end, so as to reduce data block service Device receives the write request of multiple data blocks to cause the situation of write-in focus simultaneously.The block server refers to Chunksever, i.e. data management module are also referred to as data block server or block server herein, for safeguarding data block pair There is provided the interface of the access of data block copy for this module.The block server repeats threshold value and refers to allow block server simultaneously The quantity that different data blocks are write is provided, for example, it is 3 that the block server, which repeats threshold value, i.e., each block server is only Service can be provided to three data blocks.The specific method of salary distribution of block server corresponding to data block, with the distribution of three end patterns Exemplified by formula storage system, in terms of distribution of the metadata management equipment end to Chunk positions：Where the current writeable Chunk of record All Chunkserver, this partial information exists in the form of caching, it is not necessary to which persistence, the machine of delaying can basis after restarting The metadata information of above-noted is recovered.When distributing new Chunk, threshold value is repeated according to block server and avoids new point The Chunkserver matched somebody with somebody is repeated with the Chunkserver that above-mentioned steps are recorded.Or this single stepping use as far as possible and for plan Slightly, when the writeable Chunkserver quantity of cluster very little, it is impossible to when meeting unduplicated require, also allow Chunk point With acting successfully.The data segment information of file internal is visible, effectively avoids multiple Chunk from being distributed in identical On Chunkserver, focus is eliminated, it is ensured that the actual effect of write-in handling capacity.

Fig. 3 shows to perform file write-in in distributed memory system according to a kind of realize of one preferred embodiment of the application Method in step S13 flow charts.The step S13 includes step S131, step S132, step S133.

Wherein, user equipment end receives the write request on the file destination in step S131；In step S132 Said write request is forwarded to current first writeable data block correspondence in the multiple writeable data block by middle user equipment end Master data management equipment, to perform text to the corresponding one or more data block copies of current first writeable data block Part writes；User equipment end updates current first writeable data block in the multiple writeable data block in step S133.

Specifically, user equipment end receives the write request on the file destination in step S131.I.e. user sets Standby termination is by the write request on the file destination, because the writable data number of blocks of the file destination is more than described Writeable number of blocks threshold value, therefore support user to initiate the write request of respective numbers simultaneously.With the distributed storage of three end patterns Exemplified by system, user is supported to initiate M write request simultaneously, M is equal to current writeable Chunk quantity.Some user equipment ends Client receives some write requests on single file simultaneously so that the writing rate lifting of single file, and multi-process is write Multiple client writes to same data block during entering, so as to retry caused delay after reducing write-in failure.

Then, in step S132 user equipment end by said write request be forwarded in the multiple writeable data block when The corresponding master data management equipment of preceding first writeable data block, with corresponding one to current first writeable data block Or multiple data block copies perform file write-in.I.e. user equipment end is receiving the corresponding multiple writeable numbers of the file destination The master data management equipment of the writeable data block is sought according to according to default sort after block or according to certain ranking criteria Look for, and pass on corresponding write request, so as to enter in data block and its enterprising row write of copy.With the distributed storage of three end patterns Exemplified by system, Client is cached in order after receiving writeable Chunk information, while next writeable Chunk sequence number I is set to First writeable Chunk sequence number, Client receives the write request of user, chooses serial number I Chunk, writes data into Request is sent to this Chunk all PrimaryChunkserver, while I sequence numbers are revised as into next writeable Chunk's Sequence number.

Then, user equipment end updates current first writeable data in the multiple writeable data block in step S133 Block.I.e. user equipment end is responded to all write requests in order, i.e., perform write-in to the data block in order, works as institute First writeable data block where stating block sequencing is performed after write-in, by the follow-up writeable data block more new sort Information, such as exemplified above, Client receives the write request of user, chooses serial number I Chunk, writes data into request This Chunk all PrimaryChunkserver are sent to, while I sequence numbers are revised as to next writeable Chunk sequence number, I.e. when user initiates write request, corresponding Chunk is selected according to sequence number I, while I is pointed into next Chunk, this Sample accesses all Chunk in the way of poll, it is to avoid some Chunk turns into the focus of write-in.

Those skilled in the art will be understood that it is above-mentioned the mode that writeable data block is polled write-in is only for example, other It is existing or be likely to occur from now on the application is such as applicable to the mode that writeable data block is polled write-in, should also include Within the application protection domain, and it is incorporated herein by reference herein.

Preferably, step S13 also includes：Step S134 (not shown), user equipment end is worked as and received in step S134 The file write-in successful information that the master data management equipment is returned, reclaims current first writeable data block to receive pass In the new write request of the file destination.Refer to after some writeable data block and its copy are write, when master data pipe After reason equipment feedback writes successfully, the write request corresponding to the user equipment end return user data block writes successfully, and then Because presently described writeable data block does not write the full situation for triggering and redistributing writeable data block, therefore can receive user again Write request on the file destination.It is corresponding as some Chunk by taking the distributed memory system of three end patterns as an example When PrimaryChunkserver returns to Client successes, Client returns to user's success, writes flow return and receives user The stage of new request, so as to make full use of writeable data block simultaneously so that the write-in of monofile is maintained at certain amount data block While write-in in, lifted monofile writing rate.

Fig. 4 is shown according to the metadata management equipment end of the application other side, user equipment end and master data pipe Reason equipment end coordinates a kind of realize in the method for performing file write-in in distributed memory system realized to change writeable block number Measure the flow chart of threshold value.To realize the writeable number of blocks threshold function of modification, the user equipment end includes step S16.Institute Stating metadata management equipment end includes step S25, step S26.

Wherein, the writeable number of blocks threshold on the file destination that user is submitted at user equipment end in step s 16 Value sets request to send to the metadata management equipment, wherein, the writeable number of blocks threshold value sets request to include on institute State the new writeable number of blocks threshold value of file destination.First master data management equipment end receives the user equipment in step s 25 The writeable number of blocks threshold value on the file destination sent sets request, wherein, the writeable number of blocks threshold value is set please Ask including the new writeable number of blocks threshold value on the file destination；Metadata management equipment end is according to institute in step S26 State new writeable number of blocks threshold value and update the writeable number of blocks threshold value.

Specifically, the writeable number of blocks on the file destination that user is submitted at user equipment end in step s 16 Threshold value sets request to send to the metadata management equipment, wherein, the writeable number of blocks threshold value set request include on The new writeable number of blocks threshold value of the file destination.Then, metadata management equipment end receives the user in step s 25 The writeable number of blocks threshold value on the file destination that equipment is sent sets request, wherein, the writeable number of blocks threshold value is set Putting request includes the new writeable number of blocks threshold value on the file destination；Referring to that the writeable number of blocks threshold value is can root It is adjusted according to the write-in demand of user, the specific data of adjustment and request is sent to meta data block pipe by user's equipment end Manage equipment end.Current writeable data number of blocks is may compare in specific execution with needing the writeable number of blocks threshold of setting Whether value is equal, without modification if equal, if, by numerical value of the writeable number of blocks threshold modifying for needed for, But not limited to this.By taking the distributed memory system of three end patterns as an example, the requirement that user handles up according to write-in, decision will be writeable Chunk quantity is set to M；User calls Client interfaces, obtains current writeable Chunk quantity；Client is to Master Request is initiated, current writeable Chunk quantity is obtained；Master receives after Client requests that file is found in internal memory is writeable Chunk quantity N, Client is returned by N, and current writeable Chunk quantity N is returned to user by Cilent；User is according to N's and M Value is judged, if N==M, without modification；If N！=M, then call Client interfaces, sets current writeable Chunk Quantity be M；Client is initiated to ask to Master, and writeable Chunk quantity is set into M.

Then, metadata management equipment end can according to updating the new writeable number of blocks threshold value in step S26 Write number of blocks threshold value.Refer to new writeable number of blocks of the meta data block management equipment termination by request in the request Threshold value is updated or stored to history value.Specifically, by taking the distributed memory system of three end patterns as an example, Master is received please Writeable Chunk quantity is set to M after asking, while this is changed into persistence.Preferably, Client is can return to change successfully； Client returns to user and changed successfully.It is that distributed file system with the addition of a kind of new data write-in side by above method Formula, it is allowed to which single file can write multiple Chunk simultaneously, while writeable Chunk number supports dynamic adjustment, so that user is not Need oneself to manage the file of multichannel write-in, it is only necessary to adjust the quantity of writeable data block according to the demand of write-in, save user Management cost.

Fig. 5 shows to be set according to metadata management equipment, user equipment and the master data management of the application other side It is standby to coordinate a kind of realize realized to perform the equipment schematic diagram that file writes in distributed memory system.Wherein, the user Equipment includes block message send-request unit 111, block message receiving device 112, data block write-in performs device 113；The member Data management apparatus includes block message acquisition request device 121, writeable data block determining device 122, block message feedback device 123；The master data management equipment end includes file write-in performs device 131, write-in failure handling device 132 and data block State updating device 133.

Wherein, block message send-request unit 111 submits the writeable data on file destination to metadata management equipment The block message request of block；Block message acquisition request device 121 obtains the writeable data on file destination that user equipment is submitted The block message request of block；Writeable data block determining device 122 is determined according to the corresponding writeable number of blocks threshold value of the file destination The corresponding multiple writeable data blocks of the user equipment, wherein, the multiple writeable data block includes having distributed to the target The writeable data block of the other users equipment of file or newly assigned writeable data block；Block message feedback device 123 will be described more Individual writeable data block is back to the user equipment；Block message receiving device 112 receives the metadata management equipment and is based on institute State the corresponding multiple writeable data blocks of file destination that block message request is returned, described；Data block writes performs device 113 to right Master data management equipment is answered to send on the write request of the file destination to hold one of the multiple writeable data block Part of composing a piece of writing writes；File write-in performs device 131 receives the request that file write-in is performed to data block that user equipment is sent, and Perform and the file of the data block is write；If writing failure handling device 132 to occur to perform the data block file write-in Failure, report failure information to the user equipment, and refuse the other users of corresponding with user equipment identical file Equipment performs the request of file write-in to the data block；Data block status updating device 133 updates the state of the data block To be not writeable, and to corresponding element data management apparatus report the data block update after state.

Specifically, block message send-request unit 111 is submitted on mesh to metadata management equipment in the user equipment Mark the block message request of the writeable data block of file；Block message acquisition request device 121 is obtained in the metadata management equipment The block message request for the writeable data block on file destination that user equipment is submitted.Wherein, the user equipment end refers to point Be used to provide the user the port modules of various interfaces in cloth storage system, using the distributed memory system of three end patterns as Example, the user equipment end is client i.e. Client, and hereafter these three generation titles all refer to user equipment end.The metadata Management equipment end refers to the module that metadata in distributed memory system is managed in distributed memory system, with point of three end patterns Exemplified by cloth storage system, i.e. the metadata management equipment end is Master NameSpace management modules.The mesh Mark file is the file for needing to perform write-in, can be pre-created but not limited to this generally in distributed memory system, institute The file in need for performing write-in can turn into file destination.The data block refers to one section of user logic data, with byte stream Form storage, for file destination data write, by taking the distributed memory system of three end patterns as an example, data block is this Alleged chunk in system.The block message of the writeable data block on file destination refers to can be used for performance objective text The current writeable data block of part write-in and its position, the information such as quantity.The submission and obtain the method for request and include but not It is limited by distributed memory system and has interface progress, by taking the distributed memory system of three end patterns as an example, Client client by oneself End starts, and is that multiple client is sent to Master herein because be that multi-process writes to a file simultaneously Obtain the request of all Chunk that can be write of file destination F information.By above-mentioned action, metadata management can be set The lookup to the block message of the writeable data block of file destination is initiated at standby end, so that file writes flow startup.

Then, writeable data block determining device 122 is corresponding according to the file destination in the metadata management equipment Writeable number of blocks threshold value determines the corresponding multiple writeable data blocks of the user equipment, wherein, the multiple writeable data block bag Include the writeable data block or newly assigned writeable data block for the other users equipment for having distributed to the file destination.I.e. basis can Write number of blocks threshold value and the writeable data block of the file destination for meeting preset number compared with the threshold value is distributed into the user Equipment is write, and when some user equipmenies write to file destination simultaneously, distributes to the writeable data of user equipment Block list is consistent, for example, the writeable number of blocks threshold value is M, then by taking the distributed memory system of three end patterns as an example, member Block management data equipment end Master receives the information T for finding all Chunk that can be write of file destination F after request in internal memory With the quantity M for the Chunk that can be write, determine that this T writeable data block Chunk are file destination F institutes if T is more than or equal to M Corresponding writeable data block.The writeable number of blocks threshold value refers to the threshold value of the set writeable data number of blocks of control, here, Block is to refer to data block, and this is referred to, and context is general, and the writeable number of blocks threshold value can be according to write-in handling capacity or speed It is required that being configured, the presence of this threshold value causes user oneself need not manage the file of multichannel write-in, it is only necessary to according to write-in Demand adjust the quantity of writeable data block, in addition, the writeable number of blocks threshold value can also be set according to the requirement of system is System is voluntarily adjusted, or is initially set.The corresponding multiple writeable data blocks of the user equipment refer to that each write-in file can To there is more than one writeable data block, therefore the user equipment write to it also corresponds to multiple writeable data blocks, three In the distributed memory system of end pattern, prior art only supports each file destination in meta data block management equipment end Master There is a writeable data block, can be by user equipment end and metadata management equipment end in embodiments herein Agreement between interface is configured so that each file destination has multiple writeable data blocks at meta data block management equipment end, from And multiple data blocks are supported when can reach monofile write-in while the purpose write so that file write-in is not only restricted to single machine The physical resource of device, and then improve the efficiency of monofile write-in, the formation probability of reduction write-in focus.

Then, the multiple writeable data block is back to by block message feedback device 123 in the metadata management equipment The user equipment；Block message receiving device 112 receives the metadata management equipment and is based on described piece in the user equipment Information request is returned, the corresponding multiple writeable data blocks of the user equipment.The file destination is determined as described above The data block and its relevant information are back into user by metadata management equipment end after corresponding multiple writeable data blocks to set Standby end, the corresponding multiple writeable data blocks of the user equipment are that each user equipment returned the application target to be write is literary Writeable data block corresponding to part, by taking the distributed memory system of three end patterns as an example, connects exemplified above, metadata management equipment All data block Chunk that can be write of file destination F quantity T and its relevant information are found and gone forward side by side by end Master from internal memory Row feedback, the mode of feedback includes but is not limited to carry out by the own interface of distributed memory system.

Then, in the user equipment data block write performs device 113 to correspondence master data management equipment send on The write request of the file destination is write with performing file to one of the multiple writeable data block；The master data management File write-in performs device 131 receives the request to data block execution file write-in that user equipment is sent in equipment, and performs File write-in to the data block.I.e. because the file destination there are corresponding multiple writeable data blocks, therefore, used described The multiple writeable data block is ranked up by family equipment end after obtaining the writeable data block relevant information, or suitable according to giving tacit consent to The sequence corresponding master data tube apparatus request write-in after the write request of file destination is received successively.With the distribution of three end patterns Exemplified by formula storage system, user equipment end Client is by corresponding writeable data block Chunk all the file destination F being connected to Afterwards, when being write according to acquiescence or the order set to a certain Chunk, its corresponding data block server is found i.e. Chunksever carries out the write-in of respective copies, wherein, because being multi-process write-in in the present embodiment, therefore, by write request The main data block server i.e. PrimaryChunksever dominated is sent to, and then master data server is receiving write-in After request, the client data received is ranked up, while data are locally stored to from data block server i.e. SecondaryChunksever is forwarded, and the mode of operation of principal and subordinate's data block server may be referred to the bottom in Microsoft Azure The mode of operation of data memory node in layer distributed memory system, principal and subordinate's data block server is held after write request is received Write-in of the row to file.Here, the data block server illustrated, and Chunksever each mean that the master data management is set For or from data management apparatus, claim in three kinds of generations of context general.The acquisition modes of Chunksever information include but not limited All Chunkserver where current writeable Chunk is recorded as metadata management equipment end Master, this partial information with The form of caching is present, it is not necessary to which persistence, the Information recovering that the machine of delaying can be recorded after restarting before comes out, that is, Recovered from the current all data block Chunk information write of each file for being responsible for storage；User equipment end The current all writeable Chunk information of Client storage files, this partial information exists in the form of caching, it is not necessary to persistently Change, the machine of delaying is reacquired after restarting from Master ends.

Then, if the failure that file write-in is performed to the data block occurs for write-in failure handling device 132, used to described Family device report failure information, and refuse the other users equipment of identical file corresponding with the user equipment to the data block Perform the request of file write-in.Wherein, said write, which unsuccessfully refers to, causes the scene for writing failure, for example network delay height or network It is obstructed, and disk delay is high or can not write.Because multiple user equipmenies carry out write operation to identical file simultaneously, because This, after a certain user equipment sends write request to master data management equipment, the master data management equipment writes to data block During write the situation of failure, then master data management equipment feeds back user equipment write-in failure, while refusal is all The write request that user equipment is newly initiated, so as to avoid initiating to write and then lead to the failure again to the data block for writing failure.With Exemplified by the distributed memory system of three end patterns, i.e., when PrimaryChunkserver runs into failover, on the one hand, For it has been accepted that request, returns to Client failures, the write request that all Client of refusal are newly initiated.

Then, data block status updating device 133 updates the state of the data block to be not writeable, and to correspondence metadata Management equipment reports the state after the data block renewal.The data block that failure will currently be write is set to not writeable state, And by this feedback of status to metadata management equipment end, so that the data block that it is stored is updated so that after The data block that extended meeting causes write-in to fail will not be again written.By taking the distributed memory system of three end patterns as an example, i.e., The Chunk states for writing failure are set to not writeable, and persistent storage by PrimaryChunkserver, while this is write The Chunk of failure state is reported to Master, it is desirable to which Master carries out persistent storage.Here, the not writeable shape of data block The setting of state is responsible for by master data management equipment end and metadata management equipment end, it is to avoid unnecessary data block switching, drop The growth for writing delay under low non-global write-in failure scene, improves system availability.Meanwhile, because avoid it is unnecessary Data block switching, reduce the ratio of the few small data block i.e. produced by switching of write-in, improve the service scale of system.

Preferably, the equipment includes the (not shown) of data block status receiving device 124, the metadata management equipment end Described in data block status receiving device 124 receive one of the multiple writeable data block that correspondence master data management equipment is sent Data block more new state, wherein, the data block more new state is not writeable.Certain that above-mentioned master data management equipment is sent The not writeable state of individual data block is received and in local persistent storage, for example, connecing exemplified above, Master is received Some data block that PrimaryChunkserver is reported is arranged to after not writeable information, according to its requirement to this shape State carries out local persistent storage, so as to avoid not writeable data block again acquired in user equipment end so as to which cause need not That wants writes failure.

Preferably, the equipment also writes data in the most (not shown) of processing unit 114, the user equipment including data block Block writes most processing unit 114 when the multiple writeable data block is not writeable, is carried again to the metadata management equipment Hand over the block message request of the writeable data block on the file destination.Refer to carry out to the data block corresponding to file destination During write-in, write or not writeable situation, need if running into the writeable data block obtained from metadata management equipment end The request of the writeable data and its relevant information that obtain same file destination is resubmited to metadata management equipment, now, such as Hereinbefore described in embodiment, the step of inciting somebody to action before re-starts one time.By taking the distributed memory system of three end patterns as an example, when When Client has found not having writeable Chunk in caching, the letter for obtaining all Chunk that can be write of file F is sent to Master The request of breath, redistributes the quantity for obtaining writeable Chunk so that the user equipment end obtains new corresponding to file destination Writeable data block, so as to proceed the write-in of file.

Preferably, the equipment also includes writing in the write-in unsuccessfully deletion (not shown) of device 115, the user equipment and lost Occur the failure that file write-in is performed to the data block if losing and deleting device 115, the corresponding file destination will be write unsuccessfully Corresponding data block information is deleted.Wherein, the situation for writing failure is sent out data block write-in including master data management equipment Raw failure, also including the failure scenarios occurred between user equipment end and master data management equipment, for example, using between the two Broken down in the network of transmission information.Got at user equipment end after write-in failure information, this data block information is straight Capable deletion is tapped into, and then is directly being selected from the corresponding writeable data block of the file destination acquired in metadata management equipment end Another data block re-starts write-in.By taking the distributed memory system of three end patterns as an example, fail when Client runs into write-in, The corresponding Chunk information of write-in failure is removed from the cache, but does not change this Chunk state, and then the institute from caching Select next Chunk to be retried in order in the current writeable Chunk having, so as to avoid, reduce non-global write Enter the growth for writing delay under failure scene.

Fig. 6 shows to realize that performing file in distributed memory system writes according to a kind of of the application another preferred embodiment Writeable data block determining device schematic diagram in the equipment entered.Wherein, the writeable data block determining device 122 includes candidate data Block determining unit 1221, direct feedback unit 1222, reassignment feedback unit 1223.

Wherein, candidate data block determining unit 1221 determines the corresponding all writeable data blocks of the file destination；Directly If it is corresponding writeable piece that the quantity for connecing all writeable data blocks of feedback unit 1222 is equal to or more than the file destination Amount threshold, then regard all writeable data blocks as the corresponding multiple writeable data blocks of the user equipment；Otherwise, weight Distribution feedback unit 1223 is that the file destination distributes other writeable data blocks, and by other described writeable data blocks and All writeable data blocks as the corresponding multiple writeable data blocks of the user equipment, wherein, the multiple writeable number It is equal to writeable number of blocks threshold value according to the quantity of block.

Specifically, candidate data block determining unit 1221 determines the corresponding all writeable data blocks of the file destination. Refer to all writeable data block informations for determining the file destination, the method for acquisition is included as described above from the metadata The quantity of the currently permitted writeable data block of each file in the metadata information that management equipment end is stored, i.e., with three ends Exemplified by the distributed memory system of pattern, because meta data block management equipment end Master is in terms of the metadata storage of file： It is responsible for the current all data block Chunk information write of each file of storage, the locations of copies of such as data block；It is responsible for storage The currently permitted writeable data block Chunk of each file quantity, therefore can find file destination F's from Master ends All writeable data blocks, it is assumed that the file destination F found all writeable data blocks are T.

Then, if the quantity of all writeable data blocks of direct feedback unit 1222 is equal to or more than target text The corresponding writeable number of blocks threshold value of part, then it is all writeable data blocks are corresponding multiple writeable as the user equipment Data block.Will be according to the writeable number of blocks threshold value of system default or user according to set by being required write-in and the target obtained All writeable data numbers of blocks of file are compared, if all writeable data numbers of blocks of the file destination are not less than described Writeable number of blocks threshold value is then directly set all writeable data blocks and its relevant information of the file destination by metadata management Standby end feeds back to user equipment end, is allowed to need the corresponding writeable data block of file destination write to enter row write as user equipment Enter.By taking the distributed memory system of three end patterns as an example, if the existing Chunk quantity of current goal file F is not less than writeable Number of blocks threshold value M, then return to Client by these Chunk information.

Then, if the quantity of all writeable data blocks is less than the corresponding writeable number of blocks threshold of the file destination Value, it is that the file destination distributes other writeable data blocks to reassign feedback unit 1223, and by other described writeable numbers According to block and all writeable data blocks as the corresponding multiple writeable data blocks of the user equipment, wherein, it is the multiple The quantity of writeable data block is equal to writeable number of blocks threshold value.Will be according to system default or user according to set by being required write-in All writeable data numbers of blocks of file destination of the writeable number of blocks threshold value with being obtained are compared, if the file destination All writeable data numbers of blocks are less than the writeable number of blocks threshold value, then again by the distribution of metadata management equipment end from being stored Or the data block of difference quantity is distributed in the data block being newly released so that the writeable data number of blocks of the file destination and institute State writeable number of blocks threshold value equal, and then be sent to each user equipment as the corresponding some writeable data blocks of user equipment.With Exemplified by the distributed memory system of three end patterns, if the existing Chunk quantity of current file F is less than writeable number of blocks threshold value M, Sufficient amount of Chunk is then redistributed, newly assigned Chunk quantity adds existing Chunk quantity to be equal to writeable number of blocks threshold All Chunk information are returned to Client by value M.Here, each because monofile is multi-process write-in, therefore, each Client Retain a corresponding writeable Chunk of relatively independent file destination completed information, the later stage, each Client was each tieed up The Chunk information of shield each in caching, and it is each between renewal time it is different, little bit different is had over time. By the above-mentioned means, can be controlled by writeable number of blocks threshold value so that writeable data block maintains certain value, so that with Family need not be manually controlled file write-in, reduce the management cost of user.

Fig. 7 shows to realize that performing file in distributed memory system writes according to a kind of of the application another preferred embodiment Data block write request dispensing device schematic diagram in the equipment entered.The data block write-in performs device 113 includes write request Receiving unit 1131, data block write request transmitting element 1132, data block updating block 1133.

Wherein, write request receiving unit 1131 receives the write request on the file destination；Data block write-in please Ask transmitting element 1132 that said write request is forwarded into current first writeable data block pair in the multiple writeable data block The master data management equipment answered, to be performed to the corresponding one or more data block copies of current first writeable data block File writes；Data block updating block 1133 updates current first writeable data block in the multiple writeable data block.

Specifically, write request receiving unit 1131 receives the write request on the file destination.That is user equipment Termination is by the write request on the file destination, because the writable data number of blocks of the file destination can more than described in Number of blocks threshold value is write, therefore supports user to initiate the write request of respective numbers simultaneously.With the distributed storage system of three end patterns Exemplified by system, user is supported to initiate M write request simultaneously, M is equal to current writeable Chunk quantity.Some user equipment ends Client receives some write requests on single file simultaneously so that the writing rate lifting of single file, and multi-process is write Multiple client writes to same data block during entering, so as to retry caused delay after reducing write-in failure.

Then, said write request is forwarded to the multiple writeable data block by data block write request transmitting element 1132 In the corresponding master data management equipment of current first writeable data block, with corresponding to current first writeable data block One or more data block copies perform file write-in.I.e. user equipment end receive the file destination it is corresponding it is multiple can Write according to default sort after data block or the master data management equipment of the writeable data block is entered according to certain ranking criteria Row is found, and passes on corresponding write request, so as to enter in data block and its enterprising row write of copy.With the distribution of three end patterns Exemplified by storage system, Client is cached in order after receiving writeable Chunk information, while next writeable Chunk sequence number I is set First writeable Chunk sequence number is set to, Client receives the write request of user, serial number I Chunk is chosen, by data Write request is sent to this Chunk all PrimaryChunkserver, while I sequence numbers are revised as next writeable Chunk sequence number.

Then, data block updating block 1133 updates current first writeable data block in the multiple writeable data block. I.e. user equipment end is responded to all write requests in order, i.e., write-in is performed to the data block in order, when described First writeable data block where block sequencing is performed after write-in, and the follow-up writeable data block more new sort is believed Breath, such as exemplified above, Client receives the write request of user, chooses serial number I Chunk, writes data into request hair Toward this Chunk all PrimaryChunkserver, while I sequence numbers to be revised as to next writeable Chunk sequence number, i.e., When user initiates write request, corresponding Chunk is selected according to sequence number I, while I is pointed into next Chunk, so All Chunk are accessed in the way of poll, it is to avoid some Chunk turns into the focus of write-in.

Preferably, the data block write-in performs device 113 also includes：New write request receiving unit 1134 (is not shown Go out), the new write request receiving unit 1134 is successfully believed when the file for receiving the master data management equipment return writes Breath, reclaims current first writeable data block to receive the new write request on the file destination.Refer at some After writeable data block and its copy are write, after master data management equipment feedback writes successfully, user equipment end, which is returned, to be used Write request corresponding to the data block of family writes successfully, and then because presently described writeable data block does not write full triggering again The situation of writeable data block is distributed, therefore write request of the user on the file destination can be received again.With three end patterns Distributed memory system exemplified by, when some Chunk corresponding PrimaryChunkserver return Client it is successful when Wait, Client returns to user's success, write flow and return to the stage for receiving the request of user newly, so as to make full use of writeable data block Cause simultaneously while the write-in of monofile is maintained at certain amount data block in write-in, lifting monofile writing rate.

Fig. 8 shows to coordinate the one kind realized according to the metadata management equipment and user equipment of the application other side Realize the schematic diagram that the writeable number of blocks threshold function of modification is realized in the equipment that file write-in is performed in distributed memory system. To realize the writeable number of blocks threshold function of modification, the user equipment includes writeable number of blocks threshold value and sets request to send dress Put 116.The metadata management equipment includes writeable number of blocks threshold value and sets request receiving device 125, writeable number of blocks threshold value Updating device 126.

Wherein, writeable number of blocks threshold value set send-request unit 116 by user submit on the file destination Writeable number of blocks threshold value sets request to send to the metadata management equipment, wherein, the writeable number of blocks threshold value is set please Ask including the new writeable number of blocks threshold value on the file destination.Writeable number of blocks threshold value sets request receiving device 125 The writeable number of blocks threshold value setting request on the file destination that the user equipment is sent is received, wherein, it is described writeable Number of blocks threshold value sets request to include the new writeable number of blocks threshold value on the file destination；Writeable number of blocks threshold value updates Device 126 updates the writeable number of blocks threshold value according to the new writeable number of blocks threshold value.

Specifically, writeable number of blocks threshold value set send-request unit 116 by user submit on the file destination Writeable number of blocks threshold value set request to send to the metadata management equipment, wherein, the writeable number of blocks threshold value is set Request includes the new writeable number of blocks threshold value on the file destination.Then, writeable number of blocks threshold value sets request to receive Device 125 receives the writeable number of blocks threshold value setting request on the file destination that the user equipment is sent, wherein, institute Stating writeable number of blocks threshold value sets request to include the new writeable number of blocks threshold value on the file destination；Refer to described writeable Number of blocks threshold value is can be adjusted according to the write-in demand of user, and the specific data of adjustment and request are passed through into user equipment End is sent to meta data block management equipment end.Current writeable data number of blocks is may compare in specific execution and needs to set The writeable number of blocks threshold value it is whether equal, if equal without modification, if, by the writeable number of blocks threshold value It is revised as required numerical value, but not limited to this.By taking the distributed memory system of three end patterns as an example, user handles up according to write-in It is required that, writeable Chunk quantity is set to M by decision；User calls Client interfaces, obtains current writeable Chunk quantity； Client initiates to ask to Master, obtains current writeable Chunk quantity；Master is received after Client requests in internal memory The writeable Chunk of file quantity N is found, N is returned into Client, current writeable Chunk quantity N is returned to user by Cilent；With Family is judged according to N and M value, if N==M, without modification；If N！=M, then call Client interfaces, sets Current writeable Chunk quantity is M；Client is initiated to ask to Master, and writeable Chunk quantity is set into M.

Then, writeable number of blocks threshold value updating device 126 updates described writeable according to the new writeable number of blocks threshold value Number of blocks threshold value.Refer to new writeable number of blocks threshold of the meta data block management equipment termination by request in the request Value is updated or stored to history value.Specifically, by taking the distributed memory system of three end patterns as an example, Master receives request Writeable Chunk quantity is set to M afterwards, while this is changed into persistence.Preferably, Client is can return to change successfully； Client returns to user and changed successfully.It is that distributed file system with the addition of a kind of new data write-in side by above method Formula, it is allowed to which single file can write multiple Chunk simultaneously, while writeable Chunk number supports dynamic adjustment, so that user is not Need oneself to manage the file of multichannel write-in, it is only necessary to adjust the quantity of writeable data block according to the demand of write-in, save user Management cost.

In addition, being set this application provides a kind of metadata management for being used to perform file write-in in distributed memory system It is standby, wherein, the equipment includes：

Processor；

The multiple writeable data block is back to the user equipment.

In addition, be used to perform the user equipment that file writes in distributed memory system this application provides a kind of, its In, the equipment includes：

Processor；

In addition, being set this application provides a kind of master data management for being used to perform file write-in in distributed memory system It is standby, wherein, the equipment includes：Processor；

Obviously, those skilled in the art can carry out the essence of various changes and modification without departing from the application to the application God and scope.So, if these modifications and variations of the application belong to the scope of the application claim and its equivalent technologies Within, then the application is also intended to comprising including these changes and modification.

It should be noted that the application can be carried out in the assembly of software and/or software and hardware, for example, can adopt Realized with application specific integrated circuit (ASIC), general purpose computer or any other similar hardware device.In one embodiment In, the software program of the application can realize steps described above or function by computing device.Similarly, the application Software program (including related data structure) can be stored in computer readable recording medium storing program for performing, for example, RAM memory, Magnetically or optically driver or floppy disc and similar devices.In addition, some steps or function of the application can employ hardware to realize, example Such as, as coordinating with processor so as to performing the circuit of each step or function.

In addition, the part of the application can be applied to computer program product, such as computer program instructions, when its quilt When computer is performed, by the operation of the computer, it can call or provide according to the present processes and/or technical scheme. And the programmed instruction of the present processes is called, it is possibly stored in fixed or moveable recording medium, and/or pass through Broadcast or the data flow in other signal bearing medias and be transmitted, and/or be stored according to described program instruction operation In the working storage of computer equipment.Here, including a device according to one embodiment of the application, the device includes using In the memory and processor for execute program instructions of storage computer program instructions, wherein, when the computer program refers to When order is by the computing device, method and/or skill of the plant running based on foregoing multiple embodiments according to the application are triggered Art scheme.

It is obvious to a person skilled in the art that the application is not limited to the details of above-mentioned one exemplary embodiment, Er Qie In the case of without departing substantially from spirit herein or essential characteristic, the application can be realized in other specific forms.Therefore, no matter From the point of view of which point, embodiment all should be regarded as exemplary, and be nonrestrictive, scope of the present application is by appended power Profit is required rather than described above is limited, it is intended that all in the implication and scope of the equivalency of claim by falling Change is included in the application.Any reference in claim should not be considered as to the claim involved by limitation.This Outside, it is clear that the word of " comprising " one is not excluded for other units or step, and odd number is not excluded for plural number.That is stated in device claim is multiple Unit or device can also be realized by a unit or device by software or hardware.The first, the second grade word is used for table Show title, and be not offered as any specific order.

Claims

1. a kind of method for being used to perform file write-in in distributed memory system in metadata management equipment end, wherein, should Method includes：

The corresponding multiple writeable data blocks of the user equipment are determined according to the corresponding writeable number of blocks threshold value of the file destination, Wherein, the multiple writeable data block includes the writeable data block or new for having distributed to the other users equipment of the file destination The writeable data block of distribution；

The multiple writeable data block is back to the user equipment.

2. according to the method described in claim 1, wherein, it is described true according to the corresponding writeable number of blocks threshold value of the file destination Determining the corresponding multiple writeable data blocks of the user equipment includes：

Determine the corresponding all writeable data blocks of the file destination；

If the quantity of all writeable data blocks is equal to or more than the corresponding writeable number of blocks threshold value of the file destination, It regard all writeable data blocks as the corresponding multiple writeable data blocks of the user equipment；Otherwise,

Other writeable data blocks are distributed for the file destination, and by other described writeable data blocks and described all writeable Data block as the corresponding multiple writeable data blocks of the user equipment, wherein, quantity of the multiple writeable data block etc. In writeable number of blocks threshold value.

3. method according to claim 2, wherein, it is described to distribute other writeable data block bags for the file destination Include：

Writing data block according to each file is currently corresponding and each writing the corresponding block server of data block, be the target File distributes other writeable data blocks, wherein, other each corresponding block servers of writeable data block are with each writing number It is less than or equal to predetermined block server according to the corresponding block server of block and repeats threshold value.

4. according to the method described in claim 1, wherein, methods described also includes：

The writeable number of blocks threshold value setting request on the file destination that the user equipment is sent is received, wherein, it is described Writeable number of blocks threshold value sets request to include the new writeable number of blocks threshold value on the file destination；

The writeable number of blocks threshold value is updated according to the new writeable number of blocks threshold value.

5. according to the method described in claim 1, wherein, methods described also includes：

The data block more new state of one of the multiple writeable data block that correspondence master data management equipment is sent is received, wherein, The data block more new state is not writeable.

6. a kind of method for being used to perform file write-in in distributed memory system at user equipment end, wherein, this method bag Include：

The write request on the file destination is sent with to the multiple writeable data block to correspondence master data management equipment One of perform file write-in.

7. method according to claim 6, wherein, methods described also includes：

When the multiple writeable data block is not writeable, resubmited to the metadata management equipment on the target The block message request of the writeable data block of file.

8. method according to claim 6, wherein, it is described to be sent to correspondence master data management equipment on target text The write request of part is included with performing file write-in to one of the multiple writeable data block：

Receive the write request on the file destination；

Said write request is forwarded to the corresponding master data of current first writeable data block in the multiple writeable data block Management equipment, to perform files write-in to the corresponding one or more data block copies of current first writeable data block；

Update current first writeable data block in the multiple writeable data block.

9. method according to claim 6, wherein, it is described to be sent to correspondence master data management equipment on target text The write request of part is also included with performing file write-in to one of the multiple writeable data block：

When receiving the file write-in successful information that the master data management equipment is returned, current first writeable number is reclaimed According to block to receive the new write request on the file destination.

10. method according to claim 6, wherein, methods described also includes：

The writeable number of blocks threshold value on the file destination that user is submitted sets request to send to the metadata management Equipment, wherein, the writeable number of blocks threshold value sets request to include the new writeable number of blocks threshold value on the file destination.

11. method according to claim 6, wherein, methods described also includes：

If occurring the failure that file write-in is performed to the data block, the corresponding data of the corresponding file destination will be write unsuccessfully Block message is deleted.

12. a kind of method for being used to perform file write-in in distributed memory system in master data management equipment end, wherein, should Method includes：

The request that file write-in is performed to data block that user equipment is sent is received, and execution is write to the file of the data block Enter；

If occur to the data block perform file write-in failure, to the user equipment report failure information, and refuse with The other users equipment of the user equipment correspondence identical file performs the request of file write-in to the data block；

The state for updating the data block is not writeable, and is reported to corresponding element data management apparatus after the data block renewal State.

13. a kind of metadata management equipment for being used to perform file write-in in distributed memory system, wherein, the equipment bag Include：

Block message acquisition request device, the block message of the writeable data block on file destination for obtaining user equipment submission Request；

Writeable data block determining device, for determining that the user sets according to the corresponding writeable number of blocks threshold value of the file destination Standby corresponding multiple writeable data blocks, wherein, the multiple writeable data block includes having distributed to other of the file destination The writeable data block of user equipment or newly assigned writeable data block；

14. equipment according to claim 13, wherein, the writeable data block determining device includes：

Candidate data block determining unit, for determining the corresponding all writeable data blocks of the file destination；

Direct feedback unit, if the quantity for all writeable data blocks is corresponding equal to or more than the file destination Writeable number of blocks threshold value, then regard all writeable data blocks as the corresponding multiple writeable data blocks of the user equipment； Otherwise,

Feedback unit is reassigned, other are writeable for distributing other writeable data blocks for the file destination, and by described Data block and all writeable data blocks as the corresponding multiple writeable data blocks of the user equipment, wherein, it is described many The quantity of individual writeable data block is equal to writeable number of blocks threshold value.

15. equipment according to claim 14, wherein, it is described to distribute other writeable data block bags for the file destination Include：

16. equipment according to claim 13, wherein, the equipment also includes：

Writeable number of blocks threshold value sets request receiving device, for receiving that the user equipment sends on the file destination Writeable number of blocks threshold value request is set, wherein, the writeable number of blocks threshold value sets request to include on the file destination New writeable number of blocks threshold value；

Writeable number of blocks threshold value updating device, for updating the writeable number of blocks threshold according to the new writeable number of blocks threshold value Value.

17. equipment according to claim 13, wherein, the equipment also includes：

Data block status receiving device, for receiving one of the multiple writeable data block that correspondence master data management equipment is sent Data block more new state, wherein, the data block more new state is not writeable.

18. a kind of user equipment for being used to perform file write-in in distributed memory system, wherein, the equipment includes：

Block message send-request unit, for submitting the block of the writeable data block on file destination to believe to metadata management equipment Breath request；

Block message receiving device, for receiving that the metadata management equipment is returned based on block message request, described use The corresponding multiple writeable data blocks of family equipment；

Data block write request dispensing device, for sending the write-in on the file destination to correspondence master data management equipment Request is write with performing file to one of the multiple writeable data block.

19. equipment according to claim 18, wherein, the equipment also includes：

Data block writes most processing unit, for when the multiple writeable data block is not writeable, to the metadata management Equipment resubmits the block message request of the writeable data block on the file destination.

20. equipment according to claim 18, wherein, the data block write request dispensing device includes：

Write request receiving unit, for receiving the write request on the file destination；

Data block write request transmitting element, for said write request to be forwarded to current the in the multiple writeable data block The corresponding master data management equipment of one writeable data block, with corresponding one or many to current first writeable data block Individual data block copy performs file write-in；

Data block updating block, for updating current first writeable data block in the multiple writeable data block.

21. equipment according to claim 18, wherein, the data block write request dispensing device also includes：

New write request receiving unit, for writing successful information when the file for receiving the master data management equipment return, Current first writeable data block is reclaimed to receive the new write request on the file destination.

22. equipment according to claim 18, wherein, the equipment also includes：

Writeable number of blocks threshold value sets send-request unit, for the writeable block number on the file destination for submitting user Measuring threshold value sets request to send to the metadata management equipment, wherein, the writeable number of blocks threshold value sets request to include closing In the new writeable number of blocks threshold value of the file destination.

23. equipment according to claim 18, wherein, the equipment also includes：

Device is unsuccessfully deleted in write-in, if for the failure for occurring to perform the data block file write-in, it is corresponding by writing unsuccessfully The corresponding data block information of the file destination is deleted.

24. a kind of master data management equipment for being used to perform file write-in in distributed memory system, wherein, the equipment bag Include：

File writes performs device, for receiving the request to data block execution file write-in of user equipment transmission, and performs File write-in to the data block；

Failure handling device is write, if for the failure for occurring to perform the data block file write-in, to the user equipment Failure information is reported, and refuses the other users equipment of identical file corresponding with the user equipment text is performed to the data block The request of part write-in；

Data block status updating device, the state for updating the data block is not writeable, and is set to correspondence metadata management The standby state reported after the data block renewal.

25. a kind of metadata management equipment for being used to perform file write-in in distributed memory system, wherein, the equipment bag Include：

Processor；

And it is arranged to store the memory of computer executable instructions, the executable instruction makes the place when executed Manage device：

The multiple writeable data block is back to the user equipment.

26. a kind of user equipment for being used to perform file write-in in distributed memory system, wherein, the equipment includes：

Processor；

27. a kind of master data management equipment for being used to perform file write-in in distributed memory system, wherein, the equipment bag Include：Processor；