CN103535014A

CN103535014A - A network storage system, data processing method and the client

Info

Publication number: CN103535014A
Application number: CN201380000732.4A
Authority: CN
Inventors: 程菊生
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2013-04-23
Filing date: 2013-04-23
Publication date: 2014-01-22
Anticipated expiration: 2033-04-23
Also published as: CN103535014B

Abstract

The present invention discloses a network storage system, data processing method, and a client, the system comprising a client and at least three data nodes, the nodes of each data storage area is divided into at least one memory cell, obtained within the at least three a plurality of data storage units in the node, and a plurality of memory cells are classified into a plurality of logical network elements. The client according to the type of the target RAID set parity block and the data block is cut at least two data points obtained, the plurality of object data transmitted to the target network node in a logical unit is located in a plurality of memory cells ; node receives the data of the target data block is sent by the client and/or parity block and the data blocks and/or parity block are stored to the target data stored in the nodes belonging to the target network logic unit. Through this system can improve the safety and reliability of data storage.

Description

A kind of network store system, data processing method and client

Technical field

The present invention relates to field of data storage, relate in particular to a kind of network store system, data processing method and client.

Background technology

Distributed memory system, disperses data to be stored on the equipment of many platform independent exactly.Traditional network store system adopts concentrated storage server to deposit all data, and storage server becomes the bottleneck of systematic function, can not meet the needs of Mass storage application.Distributed network storage system adopts extendible system configuration, utilizes many storage servers to share storage load, and it has not only improved reliability, availability and the access efficiency of system, is also easy to expansion.Distributed system is by several set that independently back end forms, and each back end can be a storage server.In distributed system, in each back end, all can carry out data storage and back end can be by the data of storing on other back end in network access system.

In actual applications, when need to obtain in distributed system particular data time, first need to determine the back end position at this data place, and then from this back end, get corresponding data.But, when the hard disk of certain back end or this back end break down, may cause the loss of data of storing on this back end, and then have influence on the normal access to corresponding data.Visible, how to make the Information Security of distributed system, the reliability that improves distributed system is those skilled in the art's technical problems in the urgent need to address.

Summary of the invention

In view of this, the invention provides a kind of network store system, data processing method and client, to improve the safety and reliability of data storage.

For achieving the above object, the invention provides following technical scheme:

A first aspect of the present invention provides a kind of network store system, and described system comprises a client and at least three back end, between described back end, by network, connects, and between described client and described back end, by network, is connected;

Wherein, the storage area of each back end is divided at least one memory cell, a plurality of memory cell of at least three back end described in being arranged in; Described a plurality of memory cell is incorporated into as a plurality of cellular logics unit;

Described client, for according to the RAID type of setting, target data being carried out to cutting, obtains check block and at least two data blocks; The objective network logical block that is identified for storing described data block and check block from described a plurality of cellular logics unit, and described data block and check block are sent to respectively in a plurality of target data nodes at a plurality of memory cell place in described objective network logical block;

Described target data node, the data block and/or the check block that for receiving described client, send, and described data block and/or check block are stored to respectively in described target data node and are belonged in the memory cell of described objective network logical block, so that all store a data block or the described check block in described data block in arbitrary memory cell of described objective network logical block.

In conjunction with first aspect, in the possible implementation of the first, a plurality of memory cell that comprise in described cellular logic unit all belong to different back end.

In conjunction with first aspect, in the possible implementation of the second, described client, specifically for from described a plurality of cellular logics unit, determine to support the objective network logical block of described RAID type, to utilize data block and described check block described in a plurality of cell stores in described objective network logical block.

The second possibility implementation in conjunction with first aspect or first aspect, in the third possible implementation, described client, specifically for calculating the first cryptographic Hash of described target data, according to described the first cryptographic Hash, be identified for storing the objective network logical block of described data block and check block, the cryptographic Hash scope that described objective network logical block is supported comprises described the first cryptographic Hash.

The second possibility implementation in conjunction with first aspect or first aspect, in the 4th kind of possible implementation, described client, specifically for be identified for storing the objective network logical block of described data block and check block according to the first filename of described target data, in the filename that described objective network logical block is supported, comprise described the first filename.

In conjunction with first aspect, in the 5th kind of possible implementation, at least one target data node in described a plurality of target data node, also for from described a plurality of target data nodes, obtain each memory cell in described objective network logical block storaging state information to described data block and check block, the described storaging state information getting is sent to other target data nodes, make each target data node preserve described storaging state information, wherein, described storaging state information comprises data block that each memory cell is stored or the quantity information of check block.

In conjunction with the possible implementation of the second of first aspect, in the 6th kind of possible implementation, described system also comprises: metadata node;

Described client also for: according to the RAID type of setting, target data being carried out to cutting, after obtaining described check block and at least two data blocks, the corresponding relation of described target data and described RAID type is sent to described metadata node;

Described metadata node, for storing the corresponding relation of described target data and described RAID type.

The 6th kind of possible implementation in conjunction with first aspect, in the 7th kind of possible implementation, described client, also for obtain the corresponding relation of described target data and described RAID type from described metadata node, determine to support the described objective network logical block of described RAID type, and send the request of reading described target data to the target data node at each memory cell place of described objective network logical block;

Described target data node, also for when receiving described client and read the request of described target data, be subordinated in the memory cell of described objective network logical block and read data block and/or the check block of stored described target data, and the data block reading and/or check block are sent to described client.

In conjunction with the second implementation of the possible implementation of the first of first aspect or first aspect or first aspect or the 5th kind of possible implementation of first aspect, in the 8th kind of possible implementation, described target data node, the also storaging state information to described data block and check block for each memory cell of obtaining described objective network logical block; When determining according to described storaging state information in described objective network logical block, exist when described data block or check block are stored to abnormal memory cell, according to the data block and the check block that do not occur in described objective network logical block storing in the abnormal memory cell of storage, recover described appearance and store data block or the check block in abnormal memory cell, wherein, described storaging state information comprises data block that memory cell is stored or the quantity information of check block.

A second aspect of the present invention provides a kind of data processing method, described method is applied to comprise the network store system of at least three back end, between described back end, by network, connect, the storage area of described back end is divided at least one memory cell, a plurality of memory cell of at least three back end described in being arranged in, and described a plurality of memory cell is incorporated into as a plurality of cellular logics unit, and described method comprises:

According to the RAID type of setting, target data is carried out to cutting, obtain check block and at least two data blocks;

The objective network logical block that is identified for storing described data block and check block from described a plurality of cellular logics unit;

Described data block and check block are sent to respectively in a plurality of target data nodes at a plurality of memory cell place in described objective network logical block, so that described target data node is stored the data block receiving and/or check block in its memory cell that belongs to described objective network logical block, make all to store in arbitrary memory cell of described objective network logical block data block or check block in described data block.

In conjunction with second aspect, in the possible implementation of the first, the memory cell that described cellular logic unit comprises all belongs to different back end;

Described described data block and check block are sent to respectively in a plurality of target data nodes at a plurality of memory cell place in described objective network logical block, comprise:

Arbitrary target data node in a plurality of target data nodes at a plurality of memory cell place in described objective network logical block sends a described data block or a described check block, and the data block sending to different target data nodes or check block different.

In conjunction with second aspect, in the possible implementation of the second, the described objective network logical block that is identified for storing described data block and check block from described a plurality of cellular logics unit, comprising:

From described a plurality of cellular logics unit, determine and support the cellular logic unit of described RAID type as the objective network logical block of the described data block of storage and check block.

In conjunction with the possible implementation of the second of second aspect or second aspect, in the third possible implementation, described from described a plurality of cellular logics unit, be identified for storing the objective network logical block of described data block and check block before, also comprise:

Calculate the first cryptographic Hash of described target data;

The described objective network logical block that is identified for storing described data block and check block from described a plurality of cellular logics unit, comprising:

According to described the first cryptographic Hash, be identified for storing the objective network logical block of described data block and check block, the cryptographic Hash scope that described objective network logical block is supported comprises described the first cryptographic Hash.

In conjunction with the possible implementation of the second of second aspect or second aspect, in the 4th kind of possible implementation, the described objective network logical block that is identified for storing described data block and check block from described a plurality of cellular logics unit, comprising:

According to the first filename of described target data, be identified for storing the objective network logical block of described data block and check block, in the filename that described objective network logical block is supported, comprise described the first filename.

A third aspect of the present invention also provides a kind of client, described client application is in the network store system that comprises at least three back end, between described back end, by network, connect, the storage area of described back end is divided at least one memory cell, a plurality of memory cell of at least three back end described in being arranged in, and described a plurality of memory cell is incorporated into as a plurality of cellular logics unit; Described client at least comprises processor, memory, communication interface and communication bus;

Wherein, described processor, memory and described communication interface communicate by described communication bus;

Described processor, for according to the RAID type of setting, target data being carried out to cutting, obtains check block and at least two data blocks; The objective network logical block that is identified for storing described data block and check block from described a plurality of cellular logics unit; Described data block and check block are sent to respectively in a plurality of target data nodes at a plurality of memory cell place in described objective network logical block;

Described memory is for storing the information of the program that includes described processor operation.

In conjunction with a third aspect of the present invention, in the possible implementation of the first, when the memory cell comprising when described cellular logic unit all belongs to different back end, the objective network logical block that described processor is identified for storing described data block and check block from described a plurality of cellular logics unit, is specially:

In conjunction with a third aspect of the present invention, in the possible implementation of the second, the objective network logical block that described processor is identified for storing described data block and check block from described a plurality of cellular logics unit, is specially:

In conjunction with the possible implementation of the second of a third aspect of the present invention or the third aspect, in the third possible implementation, described processor, also for described from described a plurality of cellular logics unit, be identified for storing the objective network logical block of described data block and check block before, calculate the first cryptographic Hash of described target data;

Accordingly, the objective network logical block that described processor is identified for storing described data block and check block from described a plurality of cellular logics unit, is specially:

In conjunction with the possible implementation of the second of a third aspect of the present invention or the third aspect, in the 4th kind of possible implementation, the objective network logical block that described processor is identified for storing described data block and check block from described a plurality of cellular logics unit, is specially:

In conjunction with a third aspect of the present invention, in the 5th kind of possible implementation, described network store system also comprises metadata node;

Described processor, also for the corresponding relation of described target data and described RAID type is sent to described metadata node, to store described corresponding relation in described metadata node.

In conjunction with a third aspect of the present invention, in the 6th kind of possible implementation, described processor, also for obtain the corresponding relation of described target data and described RAID type from described metadata node, determine the described objective network logical block of supporting described RAID type, and send the request of reading described target data to the target data node at each memory cell place of described objective network logical block, to obtain described data block and check block from described a plurality of target data nodes.

From above-mentioned technical scheme, can find out, the embodiment of the present invention discloses a kind of network store system, data processing method and client, because these these data blocks that cutting obtained when client and check block are stored to respectively after a plurality of memory cell that are positioned at a plurality of target data nodes in objective network logical block, between the data block of storing in each memory cell of objective network logical block and check block, also there is redundancy relationship.Like this, when because target data node breaks down, make in some or several memory cell the data block of storing or check block while going out the unusual conditions such as active or damage, utilize in the target data node not breaking down and belong to data block and the check block of storing in the memory cell of this objective network logical block, just can recover and occur abnormal data block or check block, thereby reduced irreclaimable situation after corrupted data, improved the safety and reliability of system data storage.

Accompanying drawing explanation

In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, to the accompanying drawing of required use in embodiment or description of the Prior Art be briefly described below, apparently, accompanying drawing in the following describes is only embodiments of the invention, for those of ordinary skills, do not paying under the prerequisite of creative work, other accompanying drawing can also be provided according to the accompanying drawing providing.

Fig. 1 is the structural representation of an embodiment of a kind of network store system of the present invention;

Fig. 2 shows the composition of cellular logic unit in invention and to the schematic diagram of storing data block or check block in the memory cell of cellular logic unit;

Fig. 3 shows the storage condition schematic diagram of this objective network logical block to the check block of each target data and data block;

The memory cell that Fig. 4 shows in objective network logical block occurs after storage failure and fault recovery, the contrast schematic diagram to data block sum check piece store status;

Fig. 5 shows the structural representation of a kind of another embodiment of network store system of the present invention;

Fig. 6 shows the schematic flow sheet of an embodiment of a kind of data processing method of the present invention;

Fig. 7 shows client in the present invention and carries out the mutual schematic diagram of flow process that data read;

Fig. 8 shows the structural representation of an embodiment of a kind of client of the present invention.

Embodiment

Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is only the present invention's part embodiment, rather than whole embodiment.Embodiment based in the present invention, those of ordinary skills, not making the every other embodiment obtaining under creative work prerequisite, belong to the scope of protection of the invention.

The invention discloses a kind of network store system, data processing method and client, with fail safe and the reliability of data storage in raising system.

Referring to Fig. 1, show the structural representation of an embodiment of a kind of network store system of the present invention, the system of the present embodiment comprises a client 101 and at least three back end 102.Between each back end, by network, connect, between this client and each back end, also by network, be connected.Wherein, the network that connects each back end and back end and client can be the Internet internet, Internet Protocol storage area network (IP SAN, Internet Protocol Storage Area Network) and private network etc.

In the present embodiment, the storage area of each back end 102 is divided at least one memory cell, obtains being arranged in a plurality of memory cell of these at least three back end.These a plurality of memory cell in system are incorporated into as a plurality of cellular logics unit (NLUN, Network Logical Unit Name).A plurality of memory cell are incorporated into as a plurality of cellular logics unit, be actually and constructed a plurality of cellular logics unit in system.Wherein, in each cellular logic unit, at least comprise three memory cell, and the memory cell comprising in cellular logic unit can not all be positioned at same back end.Cellular logic unit also can be called network volume, and in fact it can be understood as a storage system in a plurality of back end of its each memory cell distribution in network.

Be understandable that, the quantity of the memory cell that different cellular logic unit comprises also may be different.In order to build each cellular logic unit, can preset the quantity of the memory cell comprising in each cellular logic unit, the quantity of the memory cell that cellular logic unit comprises namely builds the quantity of the required memory cell in this cellular logic unit.Like this, according to the quantity that builds the required memory cell in each cellular logic unit, a plurality of memory cell in system can be incorporated into respectively in each cellular logic unit for needing in system to build, thereby construct a plurality of cellular logic unit that formed by the memory cell of varying number.Concrete, can be according to load balancing principle, as according to Crush algorithm, pseudo-random algorithm, taking turn algorithm or other load-balancing algorithms, and according to the quantity that builds the required memory cell in each cellular logic unit, from system, select the memory cell satisfying condition to form each cellular logic unit.

Wherein, it is a cellular logic unit that a memory cell only can be incorporated into, and when this memory cell is incorporated into behind a cellular logic unit, this memory cell just can not be incorporated in other cellular logic unit.Like this, for any one cellular logic unit, after the quantity of the memory cell comprising when this cellular logic unit is determined, can be according to the principle of load balancing, not incorporated into a plurality of memory cell of other cellular logic unit from system, the memory cell of the number of required memory cell builds this cellular logic unit lower than other memory cell and with building this cellular logic unit to select load capacity.

For example, in supposing the system, there is back end A, back end B, back end C and back end D.Wherein, the storage area of back end A is divided into 2 memory cell, the storage area of back end B is divided into 4 memory cell, using the whole storage area of back end C as a memory cell, the storage area of back end D is divided into 2 storage areas, thereby obtains being arranged in 9 memory cell of these 4 back end.In supposing the system, build cellular logic unit M and need 4 memory cell, build cellular logic unit N and need 5 memory cell, 4 memory cell in these 9 memory cell can be incorporated into as cellular logic unit M, remaining 5 cellular logic unit are incorporated into as cellular logic unit N.According to load balancing principle, when building cellular logic unit M, can be that 4 relatively little memory cell of load form this cellular logic unit M from system, as hypothesis incorporates 1 memory cell of 1 memory cell of 2 memory cell in back end A and back end B and back end D into cellular logic unit M into, the storage area of cellular logic unit M is actually by 2 memory cell that are positioned at back end A, and 1 memory cell that is positioned at 1 memory cell of back end B and is positioned at back end D forms.And cellular logic unit N needs 5 memory cell, remaining 5 memory cell can be incorporated into as cellular logic unit N, if after constructing cellular logic unit M, the memory cell of system spare is greater than 5, still needs to select 5 less memory cell of load to form this cellular logic unit N.But 4 memory cell that it should be noted that cellular logic unit M can not be all 4 memory cell in back end B, to avoid occurring that all memory cell in cellular logic unit are all positioned at the situation appearance of same back end.

For breaking down, some in system or several back end cause the data of storing in back end after active or damage etc. extremely, can recover and occur abnormal data, in client, to system, carry out before data storage, after target data to be stored need to being carried out to cutting according to RAID type, then the data block after cutting and check block are stored in the memory cell that the cellular logic unit of system comprises.

Concrete, this client 101, for according to the RAID type of setting, target data being carried out to cutting, obtains check block and at least two data blocks; The objective network logical block that is identified for storing this data block and check block from a plurality of cellular logics unit of system, and data block and check block are sent to respectively in a plurality of target data nodes at a plurality of memory cell place in this objective network logical block.

Accordingly, target data node, the data block and/or the check block that for receiving this client, send, and described data block and/or check block are stored to respectively in this target data node and are belonged in the memory cell of this objective network logical block, so that all store data block or the check block in the data block that this cutting obtains in arbitrary memory cell of this objective network logical block.

Wherein, this target data can, for a file, as document, picture, routine data etc., can be also a part of data in file.

When client is carried out cutting according to the RAID type of setting to target data, can to target data, carry out cutting according to the RAID algorithm corresponding with this RAID type.In client, can be set with multiple RAID type, as RAID type can comprise RAID1, RAID5, RAID6 etc., RAID type can also the concrete form with RAIDn+m represent, wherein, n represents the quantity of data block, and m represents the quantity of check block.For example, RAID5 can comprise RAID3+2, RAID4+1 etc.When client is carried out cutting to target data, the RAID type of different target data institute foundations also may be different, and concrete can select according to the required protection level of target data the RAID type of institute's foundation.

Client is carried out after cutting target data according to the RAID type of setting, obtain at least two data blocks and at least one check block, redundancy relationship between the data block that these cuttings obtain and check block, when occurring that data block or check block are lost or the abnormal conditions such as damage lower times, utilize other not occur that abnormal data block and check block just can recover again and lose or abnormal data block or the check block such as damage.Meanwhile, client is carried out the RAID type of cutting time institute foundation to target data, has also determined target data to carry out the data block that obtains after cutting and the quantity of check block.As, when target data being carried out to cutting according to RAID3+2, obtain 3 data blocks and 2 check blocks of this target data; When target data being carried out to cutting according to RAID3+1, obtain 3 data blocks and 1 check block of this target data.

For these data blocks that cutting can be obtained and check block are stored to respectively in each memory cell that cellular logic unit comprises, client is identified for storing data block that cutting obtains and the objective network logical block of check block in need to a plurality of cellular logics unit from system.Wherein, the total quantity that the quantity of the memory cell comprising in this objective network logical block that client is determined and client are carried out the data block that obtains after cutting and check block to target data is identical, so that all can store a data block or a check block in any one memory cell in succeeding target cellular logic unit, and the data block of storing in different memory cell or check block difference.As, client is carried out cutting to target data and is obtained 3 data blocks and a check block, need the cellular logic unit that includes 4 memory cell as objective network logical block, so that follow-up, these data blocks and check block can be stored to respectively in 4 memory cell of this objective network logical block, make only to store in a memory cell data block or check block.

These these data blocks and the check block that cutting are obtained when client are stored to respectively after a plurality of memory cell that are positioned at a plurality of target data nodes in objective network logical block, between the data block of storing in each memory cell of objective network logical block and check block, also there is redundancy relationship, when because target data node breaks down, make in some or several memory cell the data block of storing or check block while going out the unusual conditions such as active or damage, in the target data node that utilization is not broken down, belong to data block and the check block in the memory cell of this objective network logical block, stored and just can recover appearance abnormal data block or check block.

Certainly, in client, can store the distributed intelligence of each cellular logic unit constructed in system, as which memory cell cellular logic unit includes, and which back end is each memory cell be positioned at, so that determine after objective network logical block, can send data block or check block to corresponding back end.

Network store system at the present embodiment comprises client and at least three back end, the storage area of each back end is all divided at least one memory cell, obtain being arranged in a plurality of memory cell of these at least three back end into, and these a plurality of memory cell are incorporated as a plurality of cellular logics unit.According to the RAID type of setting, to target data, cutting obtains check block and at least two data blocks to client, and the data block that cutting is obtained and check block are stored to respectively each memory cell that is arranged in a plurality of target data nodes in objective network logical block.Because data block and check block obtain according to the cutting of RAID type, make to have redundancy relationship between data block that in objective network logical block, each memory cell is stored and check block, when there being target data node to break down, cause occurring in one or more memory cell that data block or check block are lost or during the abnormal conditions such as damage, can utilize and in this objective network logical block, not go out data block and the check block in existing abnormal memory cell, stored and again recover the data block that occurs abnormal conditions, thereby reduced the irrecoverability after system data damages, improved the fail safe of system data storage.

In above embodiment, in a back end, may exist two or more memory cell to be incorporated into the situation for same cellular logic unit.Accordingly, for the definite objective network unit of client, in this objective network logical block, also may exist two or more memory cell to be positioned at the situation of same target data node.At least two data blocks that client obtains cutting and at least one data block are sent to respectively after the target data node at a plurality of memory cell place that this objective network logical block comprises, if target data node only comprises a memory cell that belongs to this objective network logical block, this target data node can receive a data block or a check block; If there are two or more memory cell that belong to this objective network logical block in this target data node, the data block that this target data node receives and stores and the total quantity of check block can be also two or more.For example, while having two memory cell to belong to this objective network logical block in target data node, this target data node may receive two data blocks that client sends, or receives two check blocks that client sends, or a data block and a check block.

If while having two or more memory cell to belong to this objective network logical block in target data node, the data block of storing in this target data node and the total quantity of check block will be more than or equal to two, when this target data node breaks down, may cause data block or the check block in two or more memory cell, stored to go out the abnormalities such as active or damage like this.And go out the data blocks of abnormality such as active or damage and the quantity of check block when more, and can recover bring difficulty to data, even cause recovering abnormal data block or the situation of check block of occurring.

For the safety and reliability of data storage in further raising system, for any one cellular logic unit, a plurality of memory cell that comprise in cellular logic unit are all positioned at different back end.Therefore, when building cellular logic unit, according to the quantity of the required memory cell in the cellular logic unit building, from system, choose the back end of respective numbers, and from the arbitrary back end selecting, choose a memory cell, thereby obtain building the memory cell of this cellular logic unit requirement.

For the ease of understanding the process that builds cellular logic unit, below in conjunction with an example, be introduced, referring to Fig. 2, show cellular logic unit in the application and form schematic diagram, take that to include 5 back end in system be example, the storage area of each back end is all divided into three memory cell, and if the rectangular strip region of back end top in Fig. 2 is storage area, and each cylinder representative in this storage area is each memory cell marking off.

The cellular logic unit that the needs of take build comprises: cellular logic unit NLUN1 and cellular logic unit NLUN2 are example, and wherein, the RAID type of cellular logic unit NLUN1 is: RAID3+2(i.e. 3 data blocks, 2 check blocks); The RAID type of cellular logic unit NLUN2 is: RAID3+1(i.e. 3 data blocks, 1 check block), the RAID type of supporting according to cellular logic unit NLUN1, this NLUN1 need to comprise 5 memory cell.According to load balancing, select, from 5 back end, select respectively 5 minimum memory cell of load.First from back end 1, select the minimum memory cell of load, be assumed to be second memory cell (order from top to bottom of take is in this embodiment example); Then from back end 2, select the minimum memory cell of load in this back end 2, be made as the 3rd memory cell, the rest may be inferred.The memory cell that the NLUN2 finally constructing comprises is respectively second memory cell in back end 1, the 3rd memory cell in back end 2, first memory cell in back end 3, the 3rd memory cell in first memory cell in back end 4 and back end 5, memory cell as represented in the cylinder of runic black line in Fig. 2 is the memory cell that NLUN1 comprised.After constructing this NLUN1, can from remaining memory cell, build cellular logic unit NLUN2 by select storage unit, number of memory cells corresponding to RAID type of NLUN2 is 4, the concrete process that builds NLUN2 is identical with structure NLUN1, and 4 memory cell that the NLUN2 that supposes to construct comprises are 4 memory cell that are positioned at dotted line frame in Fig. 2.

A plurality of memory cell that comprise due to objective network logical block are all positioned at different back end, client need to send a data block or a check block to the arbitrary target data node in a plurality of target data nodes at a plurality of memory cell place in objective network logical block, and the data block sending to different target data nodes or check block different.Accordingly, any one target data node only receives client and is sent to a data block or check block, and the data block receiving or check block are stored in this target data node and are belonged in the memory cell of objective network logical block.Like this, if target data node sends fault, only can cause in this objective network logical block the data block in a memory cell or check block to occur abnormal, thereby reduced, there is abnormal data volume, improve data restorability, and then improved the fail safe of data storages.

In actual applications, in order to be convenient to distinguish the quantity of the memory cell that each cellular logic unit comprises, and make the data of storing in each memory cell of consolidated network logical block there is identical attribute, system is also provided with respectively the RAID type that each cellular logic unit is supported, makes any one cellular logic unit have a RAID type of supporting.

Wherein, the RAID type of supporting cellular logic unit is set be can be understood as the distribution of cellular logic unit or specifies a RAID type, its objective is data block and check block that the follow-up RAID type cutting of supporting according to this cellular logic module unit to storage in the memory cell of this cellular logic unit of client is obtained, is all to obtain according to identical RAID type cutting so that be stored in the data in storage unit block sum check piece of same cellular logic unit.

Visible, the data block and the check block that in order to store the RAID type cutting supported according to this cellular logic unit in the memory cell comprising in cellular logic unit, obtain, the quantity of the memory cell that cellular logic unit comprises should with the total quantity according to the RAID type data block that cutting obtains to target data and check block.When system is that a cellular logic unit has distributed after a RAID type, the quantity of this required memory cell comprising in cellular logic unit is also determined.Therefore, when building cellular logic unit, can, with reference to the RAID type of distributing for cellular logic unit, determine the quantity that builds the required memory cell in this cellular logic unit, and from a plurality of memory cell of system, select the memory cell of respective numbers to form this cellular logic unit.

Accordingly, this client, specifically for from a plurality of cellular logics unit, determine the objective network logical block of supporting the RAID type of target data cutting time institute foundation, the data block and the check block that with a plurality of cell stores cuttings that utilize in this objective network logical block, obtain.Like this, the quantity that the determined objective network logical block of client has a comprised memory cell is identical with the total quantity of the data block that cutting obtains to target data and check block, and the RAID type of supporting is carried out identical two features of RAID type of cutting institute foundation to target data with client.

For the ease of understanding, still take Fig. 2 as example, also show in the figure the schematic diagram of storing data block and check block in cellular logic unit, in supposing the system, there are three cellular logic unit, the cellular logic unit NLUN1 mentioning and cellular logic unit NLUN2, also there is cellular logic unit NLUN3, wherein in previous example, the RAID type that NLUN1 supports is RAID3+2, and its memory cell comprising is as the represented memory cell of cylinder of runic black line in Fig. 2.NLUN2 supports RAID3+1, and it includes 4 memory cell, as the memory cell that in figure, dotted line frame goes out.NLUN3 supports RAID4+1, and it comprises 5 memory cell, does not mark in the figure.

When client, according to RAID3+2, target data is carried out after cutting, can obtain three data blocks and two check blocks of this target data, as the data block going out with square frame frame in Fig. 2 and check block.Client can select to support the cellular logic unit of RAID3+2 as objective network logical block, needs in this embodiment to select NLUN1 as objective network logical block.Visible, although NLUN3 also includes 5 memory cell, the RAID type of supporting due to NLUN3 is RAID3+1, so client can be using NLUN3 as objective network logical block.

Determine after objective network logical block, the data block that client obtains cutting and check block send to respectively this 5 back end, any one back end receives after the data block or check block of client transmission, the data block receiving or check block are stored in its memory cell that belongs to this NLUN1, as the arrow with the curve of arrow in Fig. 2 has been pointed out data block or check block memory cell to be stored position, for example, the data block that is arranged in the upper left corner is finally stored in second memory cell of back end 1, and all the other similarly.

Certainly, Fig. 2 is stored in a kind of storage condition in 5 memory cell of this NLUN1 by each data block and check block, so long as in 5 memory cell that the data block after cutting and check block are stored to respectively NLUN1 the most at last, as for which data block or check block, need to store in which memory cell of this NLUN1 and not limit, for example, the data block that is arranged in the square frame upper left corner in figure also can be stored in the 3rd memory cell that this NLUN1 is arranged in back end 2, originally the check block being stored in the 3rd memory cell of this back end 2 can be stored in second memory cell of back end 1.

For the data that make to store in the memory cell of each cellular logic unit are the data that belong within the scope of identical cryptographic Hash, can also set in advance the cryptographic Hash scope that each cellular logic unit is supported, thus make cellular logic unit with institute be support cryptographic Hash scope there is corresponding relation.The data of storing in the memory cell of consolidated network logical block like this, all belong to the data within the scope of identical cryptographic Hash.The cryptographic Hash scope that different cellular logic unit is supported according to actual needs also may be different.Certainly, consider that actual needs can be that the cryptographic Hash scope of the cellular logic unit of the memory cell comprise equal number supporting is set is different, so that subsequent client is determined objective network logical block.

Accordingly, client, specifically for calculating the first cryptographic Hash of target data, is identified for storing data block that cutting obtains and the objective network logical block of check block according to the first cryptographic Hash, wherein, the cryptographic Hash scope that this objective network logical block is supported comprises this first cryptographic Hash.Client, before definite objective network logical block, need to calculate the first cryptographic Hash of target data.Calculate the cryptographic Hash that the data itself of file name that the first cryptographic Hash of this target data can have based on this target data or this target data calculate.Like this, when meeting cellular logic unit that the number of memory cells comprise is identical with the total quantity of the data block that cutting obtains to target data and check block, have when a plurality of, the cryptographic Hash that can further support according to this cellular logic unit is finally determined for storing the objective network logical block of data block and check block.

Especially, the in the situation that cellular logic unit being more in system, when the cellular logic unit in system has supported RAID type, may there is the cellular logic unit of the identical RAID type of a plurality of supports.Like this, when client is determined objective network logical block, also may have a plurality of satisfied supported RAID types and the cellular logic unit identical to the RAID type of target data cutting institute foundation, client may be selected at random a cellular logic unit from these a plurality of cellular logic unit that satisfy condition.

In order uniquely to determine an objective network logical block, system can arrange RAID type and the cryptographic Hash scope of supporting each cellular logic unit simultaneously, and the cryptographic Hash scope of supporting the cellular logic unit of identical RAID type to support is different.Accordingly, client calculates the first cryptographic Hash of target data and according to the RAID type of setting, target data is carried out after cutting, can will in system, support cellular logic unit that the RAID type of this setting and the cryptographic Hash scope of supporting comprise this first cryptographic Hash as objective network logical block, thereby can uniquely determine a cellular logic unit.

In addition, except arranging above the mode of the cryptographic Hash of supporting cellular logic unit, can also set in advance the filename that in system, each cellular logic unit is supported, the filename that different cellular logic unit is supported is different.Accordingly, client, specifically for be identified for storing data block that cutting obtains and the objective network logical block of check block according to the first filename of target data, comprises this first filename in the filename that this objective network logical block is supported.Wherein, the first filename of this target data can be that client is the filename that this target data is distributed, and can be also the filename that this target data has originally.Client can uniquely be determined one by this kind of mode and meet the quantity of the memory cell comprise and data block that cutting obtains and the total quantity of check block, and the filename of supporting includes the objective network logical block of this first filename.

Further, when having distributed for cellular logic unit in the situation of the RAID type of supporting, also can distribute the filename of supporting for cellular logic unit.Wherein, the filename of supporting the cellular logic unit of identical RAID type to support is different.Like this, the first filename that client can select the filename of support to comprise this target data from system, and the identical objective network logical block of RAID type of this target data institute foundation of the RAID type and cutting supported.

Be understandable that, in order to distinguish each cellular logic unit in network, can also be for each cellular logic unit distributes a sign that system is unique, this sign can be one or more signs of calculating in the RAID type of supporting according to cellular logic unit, the cryptographic Hash scope of supporting and filename.

In above arbitrary embodiment, construct behind cellular logic unit, can also in each memory cell of cellular logic unit, store total memory space and the current available memory space of this each memory cell of cellular logic unit, the current available memory space of each memory cell, the back end information at each memory cell place in this cellular logic unit, one or more in the information such as filename that the cryptographic Hash scope that the RAID type that this cellular logic unit is supported, this cellular logic unit are supported, this cellular logic unit are supported.As, in back end a, a memory cell a1 belongs to cellular logic unit A, and back end a can record the information that this cellular logic unit is relevant in this memory cell a1, total memory space of introducing as above, the information such as RAID type of being supported.

Because any one target data node all may include a plurality of memory cell, and these a plurality of memory cell may belong to different cellular logic unit.Therefore, in order to make target data node after receiving data block or check block, can determine and the data block receiving or check block need to be stored in the memory cell that belongs to which cellular logic unit, client, when sending data block or check block to back end, also needs the information of objective network logical block to send to back end.As, client can be by the sign of objective network logical block, or objective network logical block RAID type and the cryptographic Hash scope supported, or the information such as the RAID type supported of objective network logical block and filename sends to target data node, so that can determining, target data node which memory cell data block or check block need to be stored in.

In addition, after target data node is stored to by the data block receiving or check block the memory cell that belongs to objective network logical block, target data node can also record the store status of this memory cell to data block or check block, as whether being stored as merit, and the quantity of storage data block or check block, and the destination object of storing, for example, the data block that target data 1 cutting is obtained and check block etc.

Further, for target data node can be recognized the storage condition to the data block of target data or check block in each memory cell in objective network logical block, in the plurality of target data node, has a target data node at least, also for from these a plurality of target data nodes, obtain each memory cell in this objective network logical block storaging state information to data block sum check piece, and the storaging state information getting is sent to other target data nodes, make each target data node preserve this storaging state information.Have, each target data node is retained to this storaging state information in each memory cell that this objective network logical block comprises.Wherein, in this storaging state information, at least comprise the data block of this target data that each memory cell is stored or the quantity information of check block.In addition, this storaging state information can also comprise and in memory cell, stores data block or the check block which target data cutting is obtained.

Wherein, any one target data node in these a plurality of target data nodes all can be from receiving data block that client sends or check block all after specifying duration, to other target data nodes, send store status read requests, to get each target data node to the data block of client transmission or the storage resource request of check block.Certainly, any one target data node also can periodically send store status read requests to other target data nodes, to get the storaging state information of each memory cell to data block sum check piece in this objective network logical block by other target data nodes, and the storaging state information getting is sent to other back end.Certainly, the target data node of not initiating in the same time store status read requests may be different, but so long as can guarantee that the storaging state information of storage in back end can periodically be upgraded.

Target data nodes records can adopt any-mode to each memory cell to the mode of the storaging state information of data block sum check piece.Optionally, the numeral of the identical figure place of number that in this application can be by the memory cell that comprises with objective network logical block represents the storage condition of the redundant slices of storage object, the not coordination of this numeral represents different memory cell, the concrete numerical value on coordination has not represented the storage condition of the memory cell of the back end that this is corresponding to redundant slices, as, 0 represents not store data block or check block in this memory cell, and 1～F indicates 1～15 data block or check block.

Data block and check block due to the different target data of not storing to this objective network logical block in the same time, in the memory cell of objective network logical block, may store data block and the check block of a plurality of target datas, therefore, need to, for each target data, record respectively the storaging state information of each memory cell to respective data blocks and check block.Take that in the memory cell of objective network logical block, to store respectively data block and check block that three target data cuttings are obtained be example, referring to Fig. 3, show the storage condition of this objective network logical block to the check block of each target data and data block, and the corresponding storaging state information that records.

Data block and the check block of supposing to store in the memory cell of this objective network logical block three target datas, these three target datas are respectively obj-0, obj-1 and obj-2.Obj-0 is carried out to data block and check block that cutting obtains and be respectively 0-0,0-1,0-2,0-3 and the 0-4 in figure; Accordingly, obj-1 is carried out to data block and check block that cutting obtains and be respectively 1-0,1-1,1-2,1-3 and the 1-4 in figure; The data block that cutting obtains to obj-2 and check block are respectively 2-0,2-1,2-2,2-3 and the 1-4 in figure.

As shown in Figure 3, the storage area of the region representation back end that the square frame of this back end top surrounds, and the common factor of the storage area of dotted portion and this back end is each memory cell in this objective network logical block.This objective network logical block include 5 memory cell, and these 5 memory cell are arranged in respectively 5 back end, can represent the storaging state information of these 5 memory cell to the data block of this each target data and check block by 5 figure places.Specifically can record in the following way storaging state information:

For obj-0, storaging state information can be recorded as: 11111, it represents that the memory cell that is positioned at these 5 back end in this objective network logical block all stored normally the redundant slices of this storage object Obj0, in each memory cell, stores respectively data block or the check block of this obj-0.

For obj-1, storaging state information can be recorded as 11110, its expression is arranged in data block or the check block that the memory cell of back end 5 stores and loses, and in other four back end, in respective memory unit, all has data block or the check block of this storage object Obj1.

For obj-2, storaging state information can be recorded as 21110, data block or the check block of this storage object Obj2 not storing in the memory cell of its expression back end 5, and in back end 1, preserve the data block of this storage object Obj2 and the total quantity of check block is two, namely data block or the check block in the former memory cell that should store back end 5 into is stored in the memory cell of back end 1.And in the respective memory unit of other three back end, store respectively data block or the check block of this storage object Obj2.

Generally, while storing data block of same target data or check block in a memory cell, explanation does not occur that storage is abnormal, otherwise has occurred that storage is abnormal.

In order to break down at certain target data node, and when causing being positioned in objective network logical block data block that the memory cell on this target data node stores or check block and losing or damage, can recover data block or the check block in respective memory unit, losing or damage, at least one target data node in the plurality of target data node, also for each memory cell of obtaining this objective network logical block to stored data block and the storaging state information of check block; When determining according to this storaging state information in this objective network logical block, exist when data block or check block are stored to abnormal memory cell, according to the data block and the check block that do not occur in this objective network logical block storing in the abnormal memory cell of storage, recover data block or check block in the abnormal memory cell of this appearance storage.Wherein, this storaging state information comprises data block that memory cell is stored or the quantity information of check block.

Wherein, for a target data, in objective network logical block in a plurality of target data nodes at each memory cell place, have at least a target data node to obtain the storaging state information of each memory cell to the data block of this target data and check block in this objective network logical block according to the default cycle, and then judge according to storaging state information whether each memory cell in this objective network logical block all stores data block or the check block of this target data; If certain memory cell of this objective network logical block does not have data block or the check block of this target data, or the data block of storage and the total quantity of check block be greater than one, illustrates that this memory cell occurs that data storages is abnormal.Certainly, can be also not in the same time, by different target data nodes, initiate fault detect, and then obtain this storaging state information, and judge whether to occur the abnormal memory cell of storage according to storaging state information.

The mode that target data node obtains this storaging state information has two kinds, can be target data node by sending state read requests to other target data nodes, so that other target data nodes send to this target data node by belonging in self in the memory cell of this objective network logical block to the storaging state information of data block or check block.

Especially, in each target data node, recorded the storaging state information of each storage to the data block of target data and check block in this objective network logical block, and while periodically updating this storaging state information, when target data node need to judge whether that memory cell occurs that storage is abnormal, can read the storaging state information of self storing, and the storaging state information of storing according to self has judged whether that memory cell occurs that storage is abnormal.

In determining this objective network logical block, exist when data block or check block are stored to abnormal memory cell, according to the abnormal difference of storage, the process of recover data blocks and check block also may be different.When the storage of memory cell is extremely when having lost the former data block that should store and check block, this target data node specifically for, when the data block that existence loss is stored when determine described objective network logical block according to storaging state information in or the first memory cell of check block, utilize in this objective network logical block data block and the check block in each memory cell of not losing the data block of storing or check block, stored, recover data block or the check block in described the first memory cell, lost.

This target data node obtains in this objective network logical block data block and the check block of storing in each memory cell of not losing the data block of storing or check block from other target data nodes, utilizes the data block and the check block that get can rebuild out original target data.Then according to what store in target data node, this target data is carried out to the RAID type of cutting institute foundation or the RAID type that this objective network logical block is supported, this target data is reappeared to cutting, and then can again obtain occurring interior data block or the check block of storing of abnormal memory cell.

For the ease of understanding, still with the objective network logical block shown in Fig. 3, include 5 memory cell, 5 memory cell are arranged in respectively 5 back end for example.Known by introducing of the storaging state information of the target data Obj1 in Fig. 3, the memory cell that is arranged in back end 5 in cellular logic unit NLUN1 has been lost data block or the check block 1-4 of this Obj1, and all the other memory cell that are arranged in all the other four back end do not occur the data block of this storage object Obj1 or the storage of check block extremely.

Suppose that the back end of now initiating that the target data Obj1 in this NLUN1 is detected is made as back end 1, the memory cell that back end 1 finds to be positioned at back end 5 has been lost data block or the check block of this target data Obj1, this this back end 1 read data block or the check block 1-0 of Obj1 of self storage, and from back end 2, in back end 3 and back end 4, obtain data block or the check block 1-0 of this Obj1, 1-2 and 1-3, this back end 1 utilizes remaining data block or check block 1-0, 1-1, 1-2 and 1-3 reappear and construct this target data Obj1, and definite original client is carried out cutting institute according to RAID type to this Obj1, according to this RAID type, this storage object Obj1 is re-started to cutting, thereby obtain 5 data block sum check pieces, then remaining 4 data blocks and check are contrasted with anxious these 5 data block sum check pieces that separate soon, thereby determine data block or the check block of loss, and the data block of this loss of determining or check block are sent to back end 5, in this back end 5, the data block receiving and check block are stored in its memory cell that belongs to this NLUN1 again.

As Fig. 4, before showing storage abnormal restoring and after recovering, be positioned at the memory state comparison diagram of each memory cell of each back end to the data block of this Obj1 and check block.

When the storage of memory cell is extremely for the data block of this target data of storing in memory cell or the quantity of check block are when more than one, target data node specifically for, when according to storaging state information, determine while there is more than one the second memory cell of the quantity of stored data block or check block in this objective network logical block, utilize data block and the check block in each memory cell that does not belong to this second memory cell in objective network logical block, stored to construct described target data; Obtain the RAID type of this target data institute foundation of client cutting, according to this RAID type, the target data constructing is carried out to cutting again, and according to cutting obtains again data block and check block, determine data block or check block that this second memory cell should be stored.

Wherein, obtaining the RAID type of client cutting target data institute foundation, can be to client, to send inquiry request by this target data node, to obtain the RAID type of this cutting target data institute foundation; When can be also client send data block or check block to destination data node, send to target data node to store the RAID type of cutting target data institute foundation; In addition, when the support of objective network logical block has a RAID type, in this target data node, also can store the RAID type that this objective network logical block is supported, and the RAID type of this target data institute foundation of this RAID type and this client cutting is identical, naturally this target data can directly be obtained the RAID type supported of this objective network logical block of self storage.

Suppose also to store in each memory cell in NLUN1 each data block and the check block of target data Obj3, and it is example that the data block that in the back end in this NLUN1 1, memory cell is stored up and the quantity of check block are two, the form that storaging state information of record is 21111.When the back end of initiating fault detect in these 5 back end is determined memory cell that this NLUN1 is arranged in this back end 1 and is occurred that this kind of storage is abnormal, can utilize data block and the check block of this Obj3 storing in back end 2-5 in this NLUN1, construct this target data Obj3, and definite client is carried out the RAID type of cutting time institute foundation to this Obj3, this Obj3 this being constructed according to RAID type reappears cutting, obtains 5 new data blocks and check block.Data block and the check block of this Obj3 storing in current 5 new data blocks that are syncopated as and check block and back end 2-5 are contrasted, and by these 5 new data blocks and check block, do not belong to data block or the check block of required storage in the memory cell that the data block of this Obj3 of storing in this back end 2-5 and the database of check block or check block be defined as back end 1.Data block or check block that this back end of initiation fault detect is determined this send to back end 1, so that back end 1 utilizes this data block of determining or check block to replace 2 data block sum check pieces of this storage object Obj3 original stored in this back end 1.

The storage of corresponding another kind of memory cell is abnormal, this target data node specifically for, when according to this storaging state information, while exist losing more than one the second memory cell of the first memory cell of the data block of storing or check block and the data block of storage or the quantity of check block, indicate the back end at this second memory cell place, the data block having more in the second memory cell or check block are sent to the target data node at this first memory cell place, so that the target data node at this first memory cell place is stored to the data block receiving or check block in this first memory cell.

The objective network logical block NLUN1 shown in Fig. 3 of still take is example, in this NLUN1, also store target data Obj2, from Fig. 3 and the above description to the storaging state information relevant to Obj2 of record, for a certain reason, data block or check block that back end 5 memory cell that NLUN1 is positioned at should be stored are stored in the memory cell that this NLUN1 is positioned at back end 1, and making the data block of this memory cell storage of this back end 1 and the quantity of check block is two.Still suppose that the back end of initiating to detect is back end 1, when this back end 1 gets each memory cell in this NLUN1 to the storaging state information of the data block of this storage object Obj2 and check block, can determine data block or the check block that memory cell that this NLUN1 is arranged in back end 5 should store and be stored in the memory cell that is positioned at this back end 1, this back end 1 is sent to this back end 5 from two data blocks of self storage and/or any one check block, and then the memory cell that makes to be arranged in back end 5 is stored up data block or the check block of this Obj2.

As shown in Figure 4; show and having memory cell to occur storing extremely and after the storage abnormal restoring of memory cell; the store status comparison diagram of each memory cell to each data block of this Obj2 and check block in this NLUN1; visible; after abnormal restoring, in back end 1 and back end 5, all preserve data block or the check block of a storage object Obj2.

It should be noted that, above is only to take each memory cell that the data block of each target data and check block are stored in to an objective network logical block to be introduced as example.But in actual applications, because a target data node can comprise a plurality of memory cell, and a plurality of memory cell of this target data node may belong to different objective network logical blocks, thereby make this target data node may belong to a plurality of objective network logical blocks simultaneously.Therefore, when any one target data node need to carry out abnormality detection, first need to determine objective network logical block to be detected, and need to determine target data to be detected, determine and need to carry out abnormality detection to the data block of which target data of each cell stores of this objective network logical block or the storaging state information of check block.

By the store status of each memory cell of objective network logical block being detected according to storaging state information by target data node, and when memory cell occurs that storage is abnormal, recover the data block or the check block that occur that the abnormal memory cell of storage is stored, thereby realized the self-healing of objective network logical block to data, also reduced due to loss of data or damage and cannot recover the situation of data.

In order to manage the relevant information of data of storage, and be convenient to that client is follow-up carries out the relevant information that data read or inquired about the data in each memory cell that stores cellular logic unit into, this system also comprises metadata node.As shown in Figure 5, show the composition structural representation of a kind of another embodiment of network store system of the present invention.Different from above embodiment, system also comprises in the present embodiment: metadata node 3.This metadata node 3 is connected by network with client 1.

In the present embodiment, this client also for: according to the RAID type of setting, target data being carried out to cutting, after obtaining check block and at least two data blocks, the corresponding relation of target data and this RAID type is sent to this metadata node.

Accordingly, this metadata node, for storing the corresponding relation of this target data and this RAID type.

When client is carried out cutting to different target datas, the RAID type of institute's foundation may be different.For the ease of inquiring the RAID type to cutting target data institute foundation, client need to be sent to metadata node by the corresponding relation of the RAID type of target data and this target data institute foundation of cutting, to store this corresponding relation in metadata node.For example, client is carried out cutting according to RAID3+2 to target data 1, needs target data 1 and the corresponding relation of this RAID3+2 to be sent to this metadata node; And client is carried out cutting according to RAID3+1 to target data 2, need target data 2 and the corresponding relation of this RAID3+1 to be sent to metadata node.

Further, in cellular logic unit, have under the prerequisite of supported RAID type, after if client is stored in the data block that cutting obtains to target data and check block the target data node at each memory cell place of cellular logic unit, for when client need to read target data, can determine storage each data block of this target data and the objective network logical block of check block, this client, also for obtain the corresponding relation of this target data and RAID type from metadata node, determine the objective network logical block of supporting this RAID type, and send the request of reading this target data to the target data node at each memory cell place of this objective network logical block.

Accordingly, target data node, also for when receiving client and read the request of this target data, be subordinated in the memory cell of objective network logical block and read data block and/or the check block of stored target data, and the data block reading and/or check block are sent to client.

Any one target data node, when receiving client and read the request of this target data, just can read out and belongs to data block or the check block of this target data and send to client from respective memory unit.

Visible, this metadata node is different from the data content of storing in back end, this metadata node is only storing metadata information, to be to store the metadata relevant with target data, RAID type of target data being carried out to cutting institute foundation as mentioned above.

In addition, when the certain cryptographic Hash scope of cellular logic unit support or filename, client can also send to metadata node by one or more in the information such as the first cryptographic Hash of the first filename of target data, this target data of calculating, to store these information in metadata node.

When client need to read this target data, can from metadata node, obtain the RAID type of this target data institute foundation of cutting and the first filename of this target data, and determine the objective network logical block of supporting that this RAID type and the filename of supporting comprise this first filename, to send the read requests to this target data to the back end at each memory cell place of this objective network logical block; Or, from metadata node, obtain the RAID type of this target data institute foundation of cutting and the first cryptographic Hash of this target data, and determine the objective network logical block of supporting that this RAID type and the cryptographic Hash scope of supporting comprise this first cryptographic Hash, to send the read requests to this target data to the back end at each memory cell place of this objective network logical block.

Certainly, this client also can store into after each memory cell of objective network logical block in the data block that target data cutting is obtained and check block, the corresponding relation of the identification information of this target data and this objective network logical block is sent to metadata node, in the time of target data need to being read so that follow-up, from metadata node, obtain the storage data block of this target data and the objective network logical block of check block.

It should be noted that, any one embodiment above, the network store system application RAID algorithm of all take is described as example, as client is carried out cutting according to certain RAID type to target data, and to have supported RAID type be that example is described this network store system in cellular logic unit, but in fact, this network store system is also applicable to the scene of application correction code calculation, as, client is carried out after cutting target data according to the type of correction code N+M, wherein, N and M all can be for being greater than zero natural number, can make equally to have redundancy relationship between data block that cutting obtains and check block.When certain data block or check block loss or damage, utilize and do not lose or the data block of damage and data block or the check block that check block can recover loss or damage equally.Accordingly, system can be distributed the correction code type of supporting for cellular logic unit, so that client is selected for storing the objective network logical block of data block and check block according to correction code type, its detailed process and the described process of above embodiment are similar, do not repeat them here.

On the other hand, the present invention also provides a kind of data processing method, referring to Fig. 6, show the schematic flow sheet of an embodiment of a kind of data processing method of the present invention, the method of the present embodiment is applied to comprise the network store system of at least three back end, in this network store system, between each back end, by network, connect, and the storage area of each back end is divided at least one memory cell, obtain being arranged in a plurality of memory cell of these at least three back end, and these a plurality of memory cell are incorporated into as a plurality of cellular logics unit, the method of the present embodiment comprises:

Step 601: according to the RAID type of setting, target data is carried out to cutting, obtain check block and at least two data blocks.

Wherein, in client, default RAID type can have multiplely, and when different target datas is carried out to cutting, the RAID type of institute's foundation may be different.

According to the RAID type of setting, target data is carried out to cutting, be actually according to RAID algorithm corresponding to this RAID type target data is carried out to cutting, thereby obtain at least one check block and at least two data blocks, this process and the existing similar process of the cutting of redundant fashion that data carried out according to RAID type, do not repeat them here.

Wherein, according to RAID, target data is carried out between data block that cutting obtains and check block, have redundancy relationship, when having the faults such as dropout of data block or damage, utilize the data block and the check block that do not break down can recover data block or the check block breaking down.

Step 602: the objective network logical block that is identified for storing this data block and check block from a plurality of cellular logics unit of network store system.

Step 603: data block and check block are sent to respectively in a plurality of target data nodes at a plurality of memory cell place in this objective network logical block, so that target data node is stored the data block receiving and/or check block in its memory cell that belongs to objective network logical block, make all to store in arbitrary memory cell of objective network logical block data block or check block in the data block of cutting.

Wherein, client target data is carried out cutting institute foundation RAID type decided the data block that obtains of cutting and the total quantity of check block, as when target data being carried out to cutting according to RAID3+2, can obtain 3 data blocks and 2 check blocks, the total quantity of check block and data block is 5; When target data being carried out to cutting according to RAID3+1, can obtain 3 data blocks and 1 check block.

The data block that the number of memory cells that the definite objective network logical block of client comprises obtains with cutting and the total quantity of check block is identical, like this, just can make arbitrary memory cell of succeeding target cellular logic unit all can store data block or the check block that a cutting obtains, and the data block that different memory cell are stored or check block are different

In the present embodiment, because these these data blocks that cutting obtained when client and check block are stored to respectively after a plurality of memory cell that are positioned at a plurality of target data nodes in objective network logical block, between the data block of storing in each memory cell of objective network logical block and check block, also there is redundancy relationship.Like this, when because target data node breaks down, make in some or several memory cell the data block of storing or check block while going out the unusual conditions such as active or damage, in the target data node that utilization is not broken down, belong to data block and the check block in the memory cell of this objective network logical block, stored and just can recover appearance abnormal data block or check block, thereby reduced irreclaimable situation after corrupted data, improved the safety and reliability of system storage.

While breaking down for fear of a target data node, loss of data or the damage that can make a plurality of memory cell store, the memory cell that the cellular logic unit in network store system comprises is all from different back end.Accordingly, each memory cell that objective network logical block comprises is all arranged in different target data nodes.Correspondingly, client can only send a data block or check block to any one the target data node in a plurality of target data nodes at a plurality of memory cell place in objective network logical block, and the data block sending to different target data nodes or check block different.Like this, target data node data block or the check block that can store memory cell in objective network logical block that only break down impacts.

When definite objective network logical block, due to the RAID type decided of institute's foundation target data is carried out to the total quantity of the resulting data block of cutting and check block, therefore, can determine objective network logical block according to the RAID type of target data being carried out to cutting institute foundation, and then determine data block that the quantity of comprised memory cell obtains with cutting and the identical cellular logic unit of total quantity of check block.

When the quantity of the cellular logic unit in network store system is more, the number of memory cells that likely can exist a plurality of cellular logics unit to comprise is identical, like this, the cellular logic unit that the data block that the quantity of the comprised memory cell that satisfies condition obtains with cutting is identical with the total quantity of check block may have a plurality of, and client may be selected at random an objective network logical block from these a plurality of a plurality of cellular logics unit that satisfy condition.In order to reduce by this kind of situation, and be convenient to distinguish the quantity of the memory cell that cellular logic unit comprises, make again the data of storing in each memory cell of consolidated network logical block have same alike result, each cellular logic unit can also have a RAID type of supporting simultaneously.

Accordingly, client can be from this plurality of cellular logics unit, determine support to target data carry out cutting institute according to the cellular logic unit of RAID type as the objective network logical block of storing described data block and check block.Like this, even if the number of memory cells that two cellular logic unit comprise is identical, but the RAID type that these two cellular logic unit are supported is but not necessarily identical, thereby be conducive to client, bring in and determines objective network logical block.Data block and the check block in memory cell due to objective network logical block, stored are all to obtain according to identical RAID type cutting simultaneously, are also conducive to the management to data.

In addition, when each cellular logic unit can be also to have supported cryptographic Hash scope, like this, the data of storing in the memory cell of consolidated network logical block all belong to the data within the scope of identical cryptographic Hash.The cryptographic Hash scope that different cellular logic unit is supported according to actual needs also may be different.Certainly, consider that actual needs can be that the cryptographic Hash scope of the cellular logic unit of the memory cell comprise equal number supporting is set is different, so that subsequent client is determined objective network logical block.

Accordingly, client, be identified for storing the objective network logical block of data block and check block from a plurality of cellular logics unit before, can also be calculated the first cryptographic Hash of target data.And then when definite objective network logical block, according to this first cryptographic Hash, be identified for storing the objective network logical block of data block and check block, wherein, the cryptographic Hash scope that this objective network logical block is supported comprises this first cryptographic Hash.

Especially, the in the situation that cellular logic unit being more in system, when the cellular logic unit in system has supported RAID type, may there is the cellular logic unit of the identical RAID type of a plurality of supports.In order uniquely to determine an objective network logical block, cellular logic unit can, when having supported RAID type, have supported cryptographic Hash scope.Wherein, the cryptographic Hash scope of supporting the cellular logic unit of identical RAID type to support is different.Accordingly, client calculates the first cryptographic Hash of target data and according to the RAID type of setting, target data is carried out after cutting, can will in system, support cellular logic unit that the RAID type of this setting and the cryptographic Hash scope of supporting comprise this first cryptographic Hash as objective network logical block, thereby can uniquely determine a cellular logic unit.

Certainly, cellular logic unit can be also to have institute's supporting document name.The filename that different cellular logic unit is supported also may be different.Accordingly, client can be according to the first filename of target data, is identified for storing data block that cutting obtains and the objective network logical block of check block, wherein, in the filename that this objective network logical block is supported, comprises the first filename.

Wherein, the first filename of this target data can be that client is the filename that this target data is distributed, and can be also the filename that this target data has originally.Client can uniquely be determined one by this kind of mode and meet the quantity of the memory cell comprise and data block that cutting obtains and the total quantity of check block, and the filename of supporting includes the objective network logical block of this first filename.

Further, cellular logic unit can also, when having supported RAID type, have supported filename.Wherein, the filename of supporting the cellular logic unit of identical RAID type to support is different.Like this, the first filename that client can select the filename of support to comprise this target data, and the identical objective network logical block of RAID type of this target data institute foundation of the RAID type and cutting supported.

For client is follow-up, can read each data in storage unit block sum check piece that stores cellular logic unit into.In this network store system, can also comprise metadata node, between this metadata node and client, by network, be connected.

In client, according to the RAID type of setting, target data is carried out after cutting, the corresponding relation of this target data and described RAID type can also be sent to metadata node, the data block and the check block that to store described corresponding relation in metadata node, obtain.

Further, in cellular logic unit, have under the prerequisite of supported RAID type, client can be obtained the corresponding relation of this target data and RAID type from this metadata node, and definite objective network logical block of supporting this RAID type, and send the request of reading this target data to the target data node at each memory cell place of objective network logical block, to obtain described data block and check block from described a plurality of target data nodes.

In addition, when the certain cryptographic Hash scope of cellular logic unit support or filename, client can also send to metadata node by one or more in the information such as the first cryptographic Hash of the first filename of target data, this target data of calculating, to store these metadata informations in metadata node.

When client need to read this target data, can from metadata node, obtain the RAID type of this target data institute foundation of cutting and the first filename of this target data, and determine the objective network logical block of supporting that this RAID type and the filename of supporting comprise this first filename, to send the read requests to this target data to the back end at each memory cell place of this objective network logical block; Or, client is obtained the RAID type of this target data institute foundation of cutting and the first cryptographic Hash of this target data from metadata node, and determine the objective network logical block of supporting that this RAID type and the cryptographic Hash scope of supporting comprise this first cryptographic Hash, to send the read requests to this target data to the back end at each memory cell place of this objective network logical block.

For the ease of understanding client, read the process of target data, suppose that arbitrary cellular logic unit all has supported RAID type, and take client and determine that according to the first filename of target data and to a RAID type of target data cutting time institute foundation objective network logical block is introduced as example, wherein, the one RAID represents that client carries out the RAID type of cutting institute foundation to target data, and its RAID type that is only used to support with cellular logic unit is distinguished.Client can be by the first filename of this target data, and the corresponding relation of this target data and a RAID type is sent to metadata node storage.Referring to Fig. 7, client is shown and carries out the mutual schematic diagram of flow process that data read.In order to be more convenient to describe, in Fig. 7, not draw each back end that memory cell that each cellular logic unit comprises is positioned at, but directly drawn each cellular logic unit.But be understandable that, in fact cellular logic unit is exactly the distributed storage unit that storage area is distributed in different pieces of information node, and it is example that the system of take in the figure includes four cellular logic unit.Visible client is carried out the process that data read and can be comprised:

Step 701: client sends to metadata node by the first filename of target data to be read.

Step 702: metadata node, according to the first filename receiving, is searched the metadata information corresponding with this first filename.

Metadata node, according to this first filename, can get the metadata information corresponding with this target data.Certainly, client can be also the identification information that sends this target data, so that metadata node is inquired about corresponding metadata information.

Wherein, this metadata information at least comprises a RAID type corresponding with this target data, the first filename of target data, certainly can also comprise the sign of this target data, and cryptographic Hash etc.

Step 703: metadata node sends to this client by the metadata information getting.

Step 704: client is according to the first filename and a RAID type in the metadata information receiving, determine objective network logical block, and send the request of reading this target data to the target data node at each memory cell place of this objective network logical block.

Client is according to this first filename and a RAID type, and the cellular logic unit that the filename of supporting a RAID type and support is comprised to this first filename is as objective network logical block.

Step 705: target data node receives this request, and be subordinated in the memory cell of this objective network logical block and obtain data block or the check block of this target data, and send to client.

For example, hypothetical target cellular logic unit is NLUN1, client sends request to each target data node at each memory cell place of this NLUN1, memory cell comprises with the target data node of this NLUN1 and receives after request, reads corresponding data block or check block and sends to client.

In addition, the application also provides a kind of client, referring to Fig. 8, show the structural representation of a kind of client of the application, this client application, in the network store system that comprises at least three back end, connects by network between each back end, and the storage area of each back end is divided at least one memory cell, a plurality of memory cell of at least three back end described in being arranged in, and these a plurality of memory cell are incorporated into as a plurality of cellular logics unit.This client 800 at least comprises processor 801, memory 802, communication interface 803 and communication bus 804.

Wherein, processor 801, memory 802 and communication interface 803 realize communicating by letter each other by communication bus 804;

This processor 801, for according to the RAID type of setting, target data being carried out to cutting, obtains check block and at least two data blocks; The objective network logical block that is identified for storing this data block and check block from described a plurality of cellular logics unit; The data block that cutting is obtained and check block are sent to respectively in a plurality of target data nodes at a plurality of memory cell place in objective network logical block.

This memory 802 is for storing the information that includes the program that this processor moves.This memory may comprise high-speed RAM memory, also may also comprise nonvolatile memory.

This communication interface 803, for the net element communication with network store system, as back end etc.

The memory cell that cellular logic unit in this network store system comprises all belongs in the situation of different back end, the objective network logical block that this processor 801 is identified for storing described data block and check block from described a plurality of cellular logics unit, is specially:

Arbitrary target data node in a plurality of target data nodes at a plurality of memory cell place in objective network logical block sends a described data block or a described check block, and the data block sending to different target data nodes or check block different.

While having supported RAID type in cellular logic unit, this processor 801 is identified for storing the objective network logical block of this data block and check block from a plurality of cellular logics unit, is specially:

In cellular logic unit, support to some extent cryptographic Hash scope time, this processor 801, also, for before being identified for storing the objective network logical block of data block and check block from described a plurality of cellular logics unit, calculates the first cryptographic Hash of this target data;

Accordingly, this processor 801 is identified for storing the objective network logical block of this data block and check block from a plurality of cellular logics unit, is specially:

According to this first cryptographic Hash, be identified for storing the objective network logical block of this data block and check block, the cryptographic Hash scope that this objective network logical block is supported comprises described the first cryptographic Hash.

Further, when cellular logic unit has supported RAID type and cryptographic Hash scope simultaneously, the objective network logical block that this processor 801 is identified for storing this data block and check block from a plurality of cellular logics unit is specially, supported cryptographic Hash scope is comprised to this first cryptographic Hash, and support this data cutting unit to target data carry out cutting according to the cellular logic unit of RAID type as objective network logical block.

When cellular logic unit has the filename of support, this processor 801 is identified for storing the objective network logical block of this data block and check block from a plurality of cellular logics unit, be specially: according to the first filename of target data, be identified for storing the objective network logical block of data block and check block, in the filename that this objective network logical block is supported, comprise described the first filename.

Further, when cellular logic unit has supported RAID type and filename simultaneously, this processor 801 is identified for storing the objective network logical block of this data block and check block from a plurality of cellular logics unit, be specially: the first filename that supported filename is comprised to this target data, and support this data cutting unit to target data carry out cutting according to the cellular logic unit of RAID type as objective network logical block.

In order to manage about the metadata information to target data, this network store system also comprises metadata node.Accordingly, this processor 801, also for being sent to metadata node by the corresponding relation of target data and RAID type, to store corresponding relation in metadata node.

Further, this processor 801, also for obtain the corresponding relation of this target data and RAID type from metadata node, determine the objective network logical block of supporting this RAID type, and send the request of reading this target data to the target data node at each memory cell place of this objective network logical block, to obtain data block and check block from a plurality of target data nodes.

In this specification, each embodiment adopts the mode of going forward one by one to describe, and each embodiment stresses is the difference with other embodiment, between each embodiment identical similar part mutually referring to.For the disclosed device of embodiment, because it corresponds to the method disclosed in Example, so description is fairly simple, relevant part partly illustrates referring to method.

The software module that the method for describing in conjunction with embodiment disclosed herein or the step of algorithm can directly use hardware, processor to carry out, or the combination of the two is implemented.Software module can be placed in the storage medium of any other form known in random asccess memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technical field.

Above-mentioned explanation to the disclosed embodiments, makes professional and technical personnel in the field can realize or use the present invention.To the multiple modification of these embodiment, will be apparent for those skilled in the art, General Principle as defined herein can, in the situation that not departing from the spirit or scope of the present invention, realize in other embodiments.Therefore, the present invention will can not be restricted to these embodiment shown in this article, but will meet the widest scope consistent with principle disclosed herein and features of novelty.

Claims

1. a network store system, is characterized in that, described system comprises a client and at least three back end, between described back end, by network, connects, and between described client and described back end, by network, is connected;

2. system according to claim 1, is characterized in that, a plurality of memory cell that comprise in described cellular logic unit all belong to different back end.

3. system according to claim 1, it is characterized in that, described client, specifically for from described a plurality of cellular logics unit, determine to support the objective network logical block of described RAID type, to utilize data block and described check block described in a plurality of cell stores in described objective network logical block.

4. according to the system described in claim 1 or 3, it is characterized in that, described client, specifically for calculating the first cryptographic Hash of described target data, according to described the first cryptographic Hash, be identified for storing the objective network logical block of described data block and check block, the cryptographic Hash scope that described objective network logical block is supported comprises described the first cryptographic Hash.

5. according to the system described in claim 1 or 3, it is characterized in that, described client, specifically for be identified for storing the objective network logical block of described data block and check block according to the first filename of described target data, in the filename that described objective network logical block is supported, comprise described the first filename.

6. system according to claim 1, it is characterized in that, at least one target data node in described a plurality of target data node, also for from described a plurality of target data nodes, obtain each memory cell in described objective network logical block storaging state information to described data block and check block, the described storaging state information getting is sent to other target data nodes, make each target data node preserve described storaging state information, wherein, described storaging state information comprises data block that each memory cell is stored or the quantity information of check block.

7. system according to claim 3, is characterized in that, described system also comprises: metadata node;

8. system according to claim 7, it is characterized in that, described client, also for obtain the corresponding relation of described target data and described RAID type from described metadata node, determine to support the described objective network logical block of described RAID type, and send the request of reading described target data to the target data node at each memory cell place of described objective network logical block;

9. according to the system described in claim 1,2,3 or 6 any one, it is characterized in that described target data node, the also storaging state information to described data block and check block for each memory cell of obtaining described objective network logical block; When determining according to described storaging state information in described objective network logical block, exist when described data block or check block are stored to abnormal memory cell, according to the data block and the check block that do not occur in described objective network logical block storing in the abnormal memory cell of storage, recover described appearance and store data block or the check block in abnormal memory cell, wherein, described storaging state information comprises data block that memory cell is stored or the quantity information of check block.

10. system according to claim 9, it is characterized in that, described target data node specifically for, when determining according to described storaging state information in described objective network logical block while there is first memory cell of losing the data block of storing or check block, utilize in described objective network logical block data block and the check block in each memory cell of not losing the data block of storing or check block, stored, recover data block or the check block in described the first memory cell, lost.

11. systems according to claim 9, is characterized in that,

Described target data node specifically for, when according to described storaging state information, determine while there is more than one the second memory cell of the quantity of stored data block or check block in described objective network logical block, utilize data block and the check block in each memory cell that does not belong to described the second memory cell in described objective network logical block, stored to construct described target data;

Obtain the described RAID type of target data institute foundation described in described client cutting, according to described RAID type, the described target data constructing is carried out to cutting again, and according to cutting obtains again data block and check block, determine data block or check block that described the second memory cell should be stored.

12. systems according to claim 9, is characterized in that,

Described target data node specifically for, when according to described storaging state information, while exist losing more than one the second memory cell of the first memory cell of the data block of storing or check block and the data block of storage or the quantity of check block, indicate the back end at described the second memory cell place, the data block having more in described the second memory cell or check block are sent to the target data node at described the first memory cell place, so that the target data node at described the first memory cell place is stored to described the first memory cell by the data block receiving or check block.

13. 1 kinds of data processing methods, it is characterized in that, be applied to comprise the network store system of at least three back end, between described back end, by network, connect, the storage area of described back end is divided at least one memory cell, a plurality of memory cell of at least three back end described in being arranged in, and described a plurality of memory cell incorporated into as a plurality of cellular logics unit, and described method comprises:

14. methods according to claim 13, is characterized in that, the memory cell that described cellular logic unit comprises all belongs to different back end;

15. methods according to claim 13, is characterized in that, the described objective network logical block that is identified for storing described data block and check block from described a plurality of cellular logics unit, comprising:

16. according to the method described in claim 13 or 15, it is characterized in that, described from described a plurality of cellular logics unit, be identified for storing the objective network logical block of described data block and check block before, also comprise:

Calculate the first cryptographic Hash of described target data;

17. according to the method described in claim 13 or 15, it is characterized in that, the described objective network logical block that is identified for storing described data block and check block from described a plurality of cellular logics unit, comprising:

18. methods according to claim 13, is characterized in that, described network store system also comprises metadata node;

Described, according to the RAID type of setting, target data is carried out to cutting, after obtaining check block and at least two data blocks, also comprises:

The corresponding relation of described target data and described RAID type is sent to described metadata node, to store described corresponding relation in described metadata node.

19. methods according to claim 18, is characterized in that, also comprise:

From described metadata node, obtain the corresponding relation of described target data and described RAID type, determine the described objective network logical block of supporting described RAID type, and send the request of reading described target data to the target data node at each memory cell place of described objective network logical block, to obtain described data block and check block from described a plurality of target data nodes.

20. 1 kinds of clients, it is characterized in that, described client application is in the network store system that comprises at least three back end, between described back end, by network, connect, the storage area of described back end is divided at least one memory cell, a plurality of memory cell of at least three back end described in being arranged in, and described a plurality of memory cell is incorporated into as a plurality of cellular logics unit; Described client comprises processor, memory, communication interface and communication bus;

Described processor, memory and described communication interface communicate by described communication bus;

Wherein, described processor, for according to the RAID type of setting, target data being carried out to cutting, obtains check block and at least two data blocks; The objective network logical block that is identified for storing described data block and check block from described a plurality of cellular logics unit; Described data block and check block are sent to respectively in a plurality of target data nodes at a plurality of memory cell place in described objective network logical block;

21. clients according to claim 20, it is characterized in that, when the memory cell comprising when described cellular logic unit all belongs to different back end, the objective network logical block that described processor is identified for storing described data block and check block from described a plurality of cellular logics unit, is specially:

22. clients according to claim 20, is characterized in that, the objective network logical block that described processor is identified for storing described data block and check block from described a plurality of cellular logics unit, is specially:

23. according to the client described in claim 20 or 22, it is characterized in that, described processor, also for described from described a plurality of cellular logics unit, be identified for storing the objective network logical block of described data block and check block before, calculate the first cryptographic Hash of described target data;

24. according to the client described in claim 20 or 22, it is characterized in that, the objective network logical block that described processor is identified for storing described data block and check block from described a plurality of cellular logics unit, is specially:

25. clients according to claim 20, is characterized in that, described network store system also comprises metadata node;

26. clients according to claim 25, it is characterized in that, described processor, also for obtain the corresponding relation of described target data and described RAID type from described metadata node, determine the described objective network logical block of supporting described RAID type, and send the request of reading described target data to the target data node at each memory cell place of described objective network logical block, to obtain described data block and check block from described a plurality of target data nodes.