WO2018076954A1 - Data storage method, apparatus and system - Google Patents

Data storage method, apparatus and system Download PDF

Info

Publication number
WO2018076954A1
WO2018076954A1 PCT/CN2017/101847 CN2017101847W WO2018076954A1 WO 2018076954 A1 WO2018076954 A1 WO 2018076954A1 CN 2017101847 W CN2017101847 W CN 2017101847W WO 2018076954 A1 WO2018076954 A1 WO 2018076954A1
Authority
WO
WIPO (PCT)
Prior art keywords
storage
data block
block
data
stored
Prior art date
Application number
PCT/CN2017/101847
Other languages
French (fr)
Chinese (zh)
Inventor
陈伟
林起芊
汪渭春
Original Assignee
杭州海康威视数字技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视数字技术股份有限公司 filed Critical 杭州海康威视数字技术股份有限公司
Publication of WO2018076954A1 publication Critical patent/WO2018076954A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614Improving the reliability of storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1076Parity data used in redundant arrays of independent storages, e.g. in RAID systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/064Management of blocks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0683Plurality of storage devices
    • G06F3/0689Disk arrays, e.g. RAID, JBOD

Definitions

  • the present application relates to the field of computer technologies, and in particular, to a data storage method, apparatus, and system.
  • RAID 5 and RAID 6 in the RAID (Redundant Array of Independent Disks) mode.
  • RAID5 is a mode in which block cross-access and check information are evenly distributed.
  • the RAID 5 read and write operations are based on strips.
  • the strips are divided into multiple storage units, which are used to store data blocks and check blocks.
  • Figure 1a is a schematic structural view of a strip, wherein the strip includes n data blocks (D1, D2, ..., Dn) and m check blocks (P1, P2, ..., Pm), each data block And the check blocks are distributed on different disks.
  • disks can be on different servers.
  • n when n takes 3, m takes 1 , the strip corresponds to 4 servers, and FIG. 1b shows a schematic diagram of the logical relationship between the strip and the server in this example.
  • the horizontal 3 data blocks and 1 check block in Fig. 1b form a strip.
  • Table 1 is The composition of each specific strip.
  • the data blocks and check blocks in each stripe are located on different servers.
  • the data block is updated, the corresponding check block is also recalculated and updated.
  • a data block and a check block conform to the above relationship, there is consistency between the data block and the check block.
  • any data block or check block of not less than n is taken from any of them, and other data blocks or check blocks can be obtained according to the above relationship. That is, when a server fails, the data on that server can be recovered from the data on other servers.
  • the method of storing data is usually a full strip write to a disk in the server, that is, a check block is obtained according to the data block, and then the data block and the check block in the stripe are simultaneously written to the disk.
  • a check block is obtained according to the data block, and then the data block and the check block in the stripe are simultaneously written to the disk.
  • the stripe data may not be completely written successfully. After the system is restarted, the data in the stripe may be incomplete, and the valid data in the data block and the data in the check block may be inconsistent.
  • the purpose of the embodiments of the present application is to provide a data storage method, apparatus, and system to avoid the problem of inconsistency between data blocks and check blocks as much as possible.
  • the embodiment of the present application discloses a data storage method, which is applied to a primary storage server in a data storage system, where the data storage system includes: a management server and a storage server; and the storage server includes: a storage server and a non-primary storage server; the primary storage server is any one of the storage servers; the method includes:
  • the method further includes:
  • Deleting the stored data of the first data block when the first data block is not successfully stored, and performing the first data according to all data blocks that have been successfully stored in the first stripe The step of the first check block in the band.
  • the method further includes:
  • the method further includes:
  • the method further includes:
  • the method further includes:
  • the stored data of the second data block is deleted.
  • the method further includes:
  • the data in the first data block is from a file.
  • the embodiment of the present application discloses a data storage device, which is applied to a primary storage server in a data storage system, where the data storage system includes: a management server and a storage server; and the storage server includes: a storage server and a non-primary storage server; the primary storage server is any one of the storage servers; the device includes:
  • a first receiving module configured to receive a first data block to be stored to the first stripe
  • a first storage module configured to send, to the management server, a storage address obtaining request for the first data block, and obtain a first storage address determined by the management server according to a stripe storage mechanism, and use the first data
  • the block is stored to the first storage address
  • a first determining module configured to determine, when the first data block is successfully stored, whether the first data block is the last data block in the first stripe
  • a first calculating module configured to: when the determination result of the first determining module is YES, calculate a first check in the first strip according to all data blocks that have been successfully stored in the first stripe Piece;
  • a second storage module configured to send a storage address obtaining request for the first check block to the management server, and obtain a second storage address determined by the management server according to a stripe storage mechanism, where the first A parity block is stored to the second storage address.
  • the device further includes:
  • a first deleting module configured to delete the stored data of the first data block when the first data block is not stored successfully after storing the first data block to the first storage address, and Executing the first computing module.
  • the device further includes:
  • a second determining module configured to determine whether the first check block is successfully stored after storing the first check block to the second storage address, and if not, returning to execute the first calculating module .
  • the device further includes:
  • a detecting module configured to detect, when the first data block is not the last data block in the first strip, whether to receive the second data block to be stored in the first stripe in a preset time period ;
  • a second calculating module configured to: when the detection result of the detecting module is negative, calculate a second parity block in the first strip according to all data blocks that have been successfully stored in the first stripe;
  • a third storage module configured to send a storage address obtaining request for the second check block to the management server, and obtain a third storage address determined by the management server according to a stripe storage mechanism, and the second storage The check block is stored to the third storage address.
  • the device further includes:
  • a second receiving module when the storage unit for storing the data block in the first strip is not full, after receiving the second parity block to the third storage address, receiving the first Two data blocks;
  • a fourth storage module configured to send, to the management server, a storage address obtaining request for the second data block, and obtain a fourth storage address determined by the management server according to a stripe storage mechanism, and the second data is The block is stored to the fourth storage address;
  • a third determining module configured to determine, when the second data block is successfully stored, whether the second data block is the last data block in the first stripe
  • a third calculating module configured to: when the determination result of the third determining module is YES, calculate a third check in the first strip according to all data blocks that have been successfully stored in the first stripe Piece;
  • an update module configured to update the second check block to the third check block.
  • the device further includes:
  • a second deleting module configured to: after storing the second data block to the fourth storage address, when the second data block is the first data block received after storing the second parity block And storing the stored data of the second data block when the second data block is not successfully stored.
  • the device further includes:
  • a third deleting module configured to: after storing the second data block to the fourth storage address, when the second data block is not the first data block received after storing the second parity block And if the second data block is not successfully stored, the stored data of the second data block is deleted, and the third computing module is executed.
  • the present application discloses a data storage system including: a management server and a storage server; the storage server includes: a primary storage server and a non-primary storage server, wherein the primary storage server is the storage server Any of them;
  • the primary storage server is configured to receive a first data block to be stored to the first strip, and send a storage address obtaining request for the first data block to the management server;
  • the management server is configured to receive a storage address obtaining request for the first data block sent by the primary storage server, determine a first storage address according to a stripe storage mechanism, and send the first storage address to the Said primary storage server;
  • the primary storage server is configured to receive the first storage address sent by the management server, store the first data block to the first storage address, and when determining that the first data block is successfully stored, Determining whether the first data block is the last data block in the first strip, and if so, calculating the first strip according to all data blocks that have been successfully stored in the first strip a first parity block; sending a storage address obtaining request for the first parity block to the management server;
  • the management server is configured to receive a storage address obtaining request for the first check block sent by the primary storage server, determine a second storage address according to a stripe storage mechanism, and send the second storage address to The primary storage server;
  • the primary storage server is configured to receive the second storage address sent by the management server, and store the first parity block to the second storage address.
  • the management server is further configured to: when it is detected that the first data block is not successfully stored, and when detecting that an abnormality occurs in the primary storage server, deleting the stored data of the first data block, Selecting a new primary storage server from the non-primary storage server, switching the primary storage server to the new primary storage server, and transmitting a calculation instruction for the first parity block to the primary storage server
  • the calculation instruction carries a storage address of all data blocks that have been successfully stored in the first stripe
  • the primary storage server is configured to receive the calculation instruction sent by the management server, and obtain a data block according to the storage address carried in the calculation instruction, and calculate the first strip according to the obtained data block.
  • the first check block in .
  • the embodiment of the present application discloses a primary storage server in a data storage system, where the data storage system includes: a management server and a storage server; the storage server includes: a primary storage server and a non-primary storage server; The primary storage server is any one of the storage servers; the primary storage server includes:
  • the circuit board is disposed inside the space enclosed by the housing, the processor and the memory are disposed on the circuit board; and the power circuit is configured to Each circuit or device of the primary storage server is powered; the memory is for storing executable program code; the processor runs the program corresponding to the executable program code by reading executable program code stored in the memory for performing the following steps:
  • the embodiment of the present application discloses a storage medium for storing executable code, which is used to implement the data storage method provided by the embodiment of the present application.
  • the data storage method includes:
  • the primary storage server as the execution entity requests the first storage address of the first data block from the management server when receiving the first data block to be stored in the first stripe. Saving the first data block to the first storage location after obtaining the first storage address site.
  • the first data block is the last data block in the first strip, and if so, the first data is calculated according to all the data blocks that have been successfully stored in the first strip.
  • the first check block in the strip is obtained.
  • the data block is stored first, and when all the data blocks in the stripe are successfully stored, the check block is calculated according to the data block stored in the stripe, and the check block is stored.
  • the check block is first calculated according to the data block, and in the case where both the data block and the check block in the strip are available, the data block and the check block are simultaneously stored. If a power failure occurs during the storage process, both the data block and the check block may be unsuccessful in storage. In this case, the data block and the check block are inconsistent, and a Write Hole problem occurs.
  • the check block is calculated and stored, and even if a power failure occurs during the storage process, the data block and the data block can be avoided as much as possible.
  • the check block is inconsistent.
  • Figure 1a is a schematic structural view of a strip
  • Figure 1b is a schematic diagram of the logical relationship between a stripe and a server
  • FIG. 2 is a schematic structural diagram of a data storage system according to an embodiment of the present disclosure
  • FIG. 3 is a schematic flowchart of a data storage method according to an embodiment of the present application.
  • FIG. 4 is another schematic flowchart of a data storage method according to an embodiment of the present application.
  • Figure 5a is a schematic structural diagram of a strip after data has been stored
  • FIG. 5b is another schematic structural diagram of a strip after data has been stored
  • FIG. 5c is a schematic structural diagram of another strip after the data has been stored
  • FIG. 6 is a schematic structural diagram of a data storage device according to an embodiment of the present disclosure.
  • FIG. 7 is another schematic structural diagram of a data storage device according to an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a data storage system according to an embodiment of the present application.
  • the embodiment of the present application provides a data storage method, device, and system, which can avoid the problem of inconsistency between a data block and a check block as much as possible.
  • the application can be applied to a primary storage server in a data storage system.
  • the data storage system can include a management server and a storage server.
  • the storage server may in turn include a primary storage server and a non-primary storage server, and the primary storage server may be any one of the storage servers.
  • FIG. 2 is a schematic structural diagram of a data storage system according to an embodiment of the present application.
  • a data storage system can also be referred to as a distributed storage system.
  • the system can include a management server 21, a storage server 22, and a client 23.
  • the management server 21 (Master Data Services, MDS) is also referred to as a metadata server for organizing and managing storage space information in the storage server 22.
  • the client 23 is configured to send data to be stored to the storage server.
  • FIG. 3 is a schematic flowchart of a data storage method according to an embodiment of the present disclosure, which is applied to a primary storage server in a data storage system.
  • the data storage system includes: a management server and a storage server.
  • Storage servers include: primary storage servers and non-primary storage servers.
  • the primary storage server is any one of the storage servers. Specifically, the method includes the following steps:
  • Step S301 Receive a first data block to be stored to the first strip.
  • the number of the first data blocks is one.
  • the first data block received by the primary storage server may be the first data block of the first stripe or the last data block of the first stripe.
  • the primary storage server As the execution body does not know which strip the first stripe is.
  • the primary storage server may send a stripe obtaining request to the management server, and after receiving the stripe obtaining request sent by the primary storage server, the management server determines the first strip according to the strip allocation mechanism, and sends the first strip to the Primary storage server
  • the storage server receives and records the first strip.
  • the management server determines the first strip, which may include determining the identity of the first strip.
  • the foregoing strip allocation mechanism can be understood as a stripe allocation rule, which can adopt a stripe allocation rule in the related art, which is not described in detail in this embodiment.
  • the stripe obtains a format of the stripe to be acquired in the request, the format may include a first quantity of the storage unit for storing the data block in the stripe, and a second quantity of the storage unit for storing the check block. Quantity.
  • the stripe to be acquired by the main storage server as the execution subject includes four storage units that store data blocks and one storage unit that stores check blocks.
  • the main storage server as the execution body has determined which stripe the first stripe is. Specifically, when the first data block is not the first data block of the first strip, the primary storage server may determine, according to the saved label of the first strip, the first strip corresponding to the first data block. The mark of the first strip may be that when the first data block is the first data block of the first strip, the primary storage server saves after receiving the first strip determined by the management server.
  • the primary storage server may divide the file data according to the size of each storage unit for storing the data block in the strip stored by itself. Get multiple blocks of data one by one.
  • the size of the file A to be stored is 1.6 MB
  • the size of each storage unit for storing data blocks in the strip is 1 MB
  • the primary storage server can divide the file A into 2 data blocks, one is 1 MB.
  • the first data block received in step S301 may be the first data block or the second data block.
  • Step S302 Send a storage address obtaining request for the first data block to the management server, and obtain a first storage address determined by the management server according to the stripe storage mechanism, and store the first data block to the first storage address.
  • the stripe storage mechanism can also be understood as a stripe storage rule, which includes a storage address allocation rule of each data block in the stripe and each check block.
  • the stripe storage mechanism in this embodiment may adopt a storage mechanism in the related art, which is not detailed in this embodiment.
  • the primary storage server as the execution entity may simultaneously send a stripe obtaining request for the first data block to the management server and for the first The storage address of the data block obtains the request, and may also send a stripe obtaining request for the first data block and a storage address obtaining request for the first data block to the management server.
  • the first storage address determined by the management server for storing the first data block may include: a first storage server for storing the first data block and a disk therein.
  • the first storage server can be any of all storage servers in the data storage system, including the primary storage server itself. That is, the first storage address may be located on the primary storage server itself or on other storage servers in the data storage system.
  • storing the first data block to the first storage address may include: the primary storage server directly storing the first data block to the first storage address.
  • storing the first data block to the first storage address may include: the primary storage server sending the first data block to the other storage server, to Causes the other storage server to store the first data block to the first storage address.
  • Step S303 When it is determined that the first data block is successfully stored, it is determined whether the first data block is the last data block in the first strip, and if yes, step S304 is performed.
  • the step may include: determining whether the first data block has been successfully stored, and if yes, determining whether the first data block is the last data block in the first strip.
  • the primary storage server may divide the file to be stored into data blocks according to the manner in which one data block corresponds to the same file. That is, the data in the first data block comes from a file.
  • the file is stored as a block of data, and the data of other files is no longer filled in the block. In this way, the following situation can be avoided: when the data of one data block comes from at least two files, if the data storage of one file is successful, the data storage of the other file fails, and finally it may be impossible to determine whether the data block is successfully stored.
  • Determining whether the first data block has been successfully stored may include the following implementation manners:
  • the primary storage server may directly determine whether the first data block has been successfully stored.
  • the primary storage server may detect whether the first data block storage success confirmation message sent by the other storage server is received within the preset time period, and if received, Then it is determined that the first data block has been successfully stored.
  • Determining whether the first data block is the last data block in the first strip may include: the number of data blocks stored in the first strip according to the first storage server and the first strip in the first strip A total number of storage units for storing data blocks, and determining whether the first data block is the last data block in the first strip.
  • the primary storage server may continue to receive the next first data to be stored to the first strip. Blocking, sending a storage address obtaining request for the next first data block to the management server, and obtaining a next first storage address determined by the management server according to the stripe storage mechanism, storing the next first data block to the next A first storage address. Until the storage unit for storing the data block in the first strip is full of data, then step S304 is performed.
  • each data block in the first strip is stored sequentially, and after the first data block is successfully stored, the second data block is continued to be stored until the last data in the first strip.
  • the block is stored successfully before the check block is calculated and stored.
  • Step S304 Calculate the first check block in the first strip according to all the data blocks that have been successfully stored in the first strip.
  • the first check block in the first strip is calculated according to all the data blocks that have been successfully stored in the first strip.
  • the primary storage server caches all the data blocks that have been successfully stored in the first strip
  • the primary storage server obtains all the data blocks that have been successfully stored in the first strip of the cache, and calculates the first strip according to the obtained data blocks.
  • the first check block in .
  • the primary storage server When the primary storage server does not cache all the data blocks that have been successfully stored in the first strip, the primary storage server sends the first strip data block obtaining request to each of the first storage servers, where the first storage server is the storage first. A server that strips data blocks. The primary storage server obtains the data block in the first strip sent by each first storage server, and calculates the first check block in the first strip according to the obtained data block.
  • the method for calculating the check block according to the data block may adopt a corresponding method in the related art, and the specific process is not detailed herein.
  • Step S305 Send a storage address obtaining request for the first check block to the management server, and obtain a second storage address determined by the management server according to the stripe storage mechanism, and store the first check block to the second storage address.
  • step S305 is substantially similar to the step S302. Therefore, the related content in step S302 can be referred to in step S305, and the specific process is not described in detail.
  • the primary storage server that is the execution entity when the primary storage server that is the execution entity receives the first data block to be stored in the first strip, the first storage address of the first data block is requested from the management server. The first data block is stored to the first storage address after the first storage address. When it is determined that the first data block is successfully stored, it is determined whether the first data block is the last data block in the first strip, and if so, the first data is calculated according to all the data blocks that have been successfully stored in the first strip. The first check block in the strip. Then, the second storage address of the first parity block is requested from the management server, and the first parity block is stored to the second storage address after the second storage address is obtained.
  • the data block is stored first, and when all the data blocks in the stripe are successfully stored, the check block is calculated according to the data block stored in the stripe, and the check block is stored.
  • the check block is first calculated according to the data block, and in the case where both the data block and the check block in the strip are available, the data block and the check block are simultaneously stored. If a power failure occurs during the storage process, both the data block and the check block may be unsuccessful in storage. In this case, the data block and the check block are inconsistent, and a Write Hole problem occurs.
  • the check block is calculated and stored, and even if a power failure occurs during the storage process, the data block and the data block can be avoided as much as possible.
  • the check block is inconsistent.
  • the method may further include:
  • step S304 is performed, that is, the step of calculating the first parity block in the first strip according to all the data blocks that have been successfully stored in the first stripe is performed.
  • the first data block is not stored successfully, and the data of the first data block may only be stored in one part, and the other part is not stored successfully; or all the data of the first data block may be stored, but part of the data is stored. An error occurred.
  • deleting the stored data of the first data block may include the following implementation manners:
  • the primary storage server may directly delete the stored data of the first data block.
  • the primary storage server may send the first storage server to the first storage server.
  • the delete instruction of the first data block is deleted, and the first storage server deletes the stored data of the first data block after receiving the delete instruction sent by the primary storage server.
  • the first storage server is a server that stores the first data block.
  • the step of deleting the stored data of the first data block can also be performed by the management server. That is, the management server may determine whether the confirmation information that the first data block is successfully stored by the first storage server is received within the preset time period, and if not received, determining that the first data block is not successfully stored, to the first storage server. Sending a delete instruction for deleting the first data block, the first storage server deleting the stored data of the first data block after receiving the delete instruction sent by the management server.
  • the first storage server is a server that stores the first data block. In this embodiment, the first storage server may be a primary storage server or a non-primary storage server.
  • the first data block is the first data block in the first strip
  • the stored data of the first data block is deleted
  • the first data block is not the first data block in the first strip, after the stored data of the first data block is deleted, the data block that has been successfully stored in the first stripe may continue to perform the calculation.
  • a step of a check block is performed.
  • step S305 that is, after storing the first parity block to the second storage address
  • the method may further include:
  • step S304 It is judged whether the first check block is successfully stored, and if not, the process returns to step S304; if so, it may not be processed.
  • determining whether the first check block is successfully stored may include the following implementation manners:
  • the primary storage server may directly determine whether the first verification block has been successfully stored.
  • the primary storage server may detect whether the confirmation information that the first check block is successfully stored by the other storage server is received within the preset time period, if received Then, it is determined that the first check block has been successfully stored.
  • the primary storage server is again stored according to the first strip. All the data blocks that are successful, calculate the first check block in the first stripe; resend the storage address obtaining request for the first check block to the management server, and regain the second determined by the management server according to the stripe storage mechanism The address is stored, and the first check block is stored to the second storage address.
  • the first check block is not stored successfully, which may be because the storage server storing the first check block is abnormal, so the storage address of the first check block needs to be re-acquired.
  • the stored data of the first check block may also be deleted to save storage space.
  • the method may further include the steps as shown in FIG. 4:
  • Step S306 When the first data block is not the last data block in the first strip, detecting whether the second data block to be stored to the first stripe is received within the preset time period, if not, executing Step S307.
  • the check block cannot be calculated.
  • the check block can be calculated and stored when the first strip is not full. In this way, the management server can temporarily set the state of the first stripe to be unwritable when it detects that the check block storage is successful.
  • Step S307 Calculate a second check block in the first strip according to all data blocks that have been successfully stored in the first strip.
  • Step S308 Send a storage address obtaining request for the second check block to the management server, and obtain a third storage address determined by the management server according to the stripe storage mechanism, and store the second check block to the third storage address.
  • steps S307 and S308 in the embodiment shown in FIG. 4 are similar to steps S304 and S305 in the embodiment shown in FIG. 3, respectively. Therefore, reference may be made to the related description in the embodiment shown in FIG.
  • the method may further include:
  • Step 1 Receive the second data block.
  • Step 2 Send a storage address obtaining request for the second data block to the management server, and obtain a fourth storage address determined by the management server according to the stripe storage mechanism, and store the second data block to the fourth storage address.
  • Step 3 When it is determined that the second data block is successfully stored, it is determined whether the second data block is the last data block in the first strip, and if yes, step 4 is performed.
  • Step 4 Calculate the third check block in the first strip according to all the data blocks that have been successfully stored in the first strip.
  • Step 5 Update the second parity block to the third parity block.
  • the strip A shown in FIG. 5a four memory cells for storing data blocks and one memory cell for storing parity blocks are included. It is known that the data block D1, the data block D2, and the check block P are stored in the strip A.
  • the storage address obtaining request of the data block D3 is sent to the management server, and the storage address determined by the management server according to the stripe storage mechanism is obtained, and the data block D3 is stored to The storage address.
  • the data block D4 is further sent to the management server for the received data block D4 to be stored to the strip A.
  • the storage address obtains the request, and obtains the storage address determined by the management server according to the stripe storage mechanism, and stores the data block D4 to the storage address.
  • the parity block P' in the strip A is calculated based on the data blocks D1 to D4 that have been successfully stored in the strip A, and the parity block P is updated to P'.
  • the strip A becomes the form shown in Fig. 5b, which includes the data blocks D1 to D4 and the check block P'.
  • the primary storage server receives the data to be stored to the strip again.
  • these data blocks are stored.
  • the check block is recalculated and updated according to the data block in the strip.
  • the method may further include:
  • the stored data of the second data block is deleted.
  • the strips shown in Figures 5a and 5b are still taken as an example for illustration.
  • the data blocks D1 and D2 and the check block P have been stored in the strip A shown in FIG. 5a.
  • the data block D3 is the one to be stored to the strip A after the check block P is stored.
  • the first data block if the data block D3 is abnormal in the storage process, and the storage is not successful, the stored data of the D3 can be directly deleted, so that the consistency of the data block and the check block in the strip A can be ensured. That is, after the data block D3 is directly deleted, the strip A is restored to the form of FIG. 5a, and is retracted to the version when the data block D3 is not written.
  • the method may further include:
  • the stored data of the second data block is deleted, and the execution is performed according to the first All the data blocks that have been successfully stored in the stripe are calculated, and the step of calculating the third check block in the first stripe is returned to step 4.
  • the strips shown in Figures 5a and 5b are still taken as an example for illustration.
  • the data blocks D1 and D2 and the check block P are already stored in the strip A shown in FIG. 5a.
  • the data block D3 When the data block D3 is received, it can be determined that the data block D3 is received after the check block P is stored.
  • the first data block of strip A When the data block D3 is successfully stored, the data block D4 is continuously stored, and the data block D4 is not the first data block to be stored to the strip A received after the check block P is stored.
  • the primary storage server deletes the data block D4 and calculates the check block P ⁇ according to the successfully stored D1, D2, and D3 in the strip A, and updates the check block P to Check block P ⁇ .
  • the strip A becomes the form shown in Fig. 5c, which includes the data blocks D1, D2, D3 and the check block P ⁇ .
  • the successfully stored data block D3 has changed the consistency of the data block and the check block in the strip A. Therefore, the check block should be recalculated and updated to ensure the data block and the check block. consistency.
  • the method may further include:
  • Step 4 Determining whether the third check block is successfully updated, if not, returning to perform the step of calculating the third check block in the first strip according to all the data blocks that have been successfully stored in the first strip, that is, returning to step 4 .
  • the strips shown in Figures 5a and 5b are still taken as an example for illustration.
  • the data blocks D1 and D2 and the check block P have been stored in the strip A shown in FIG. 5a, the data blocks D3 and D4 are added to the strip A shown in FIG. 5b, and the check block P is updated to the school.
  • Check block P' When an abnormality occurs during the update process and P' is not updated successfully, the check block P0 can be calculated again according to the data blocks D1 to D4, and the check block P' is updated to P0 to ensure that the check block update is successful.
  • FIG. 6 is a schematic structural diagram of a data storage device according to an embodiment of the present disclosure, corresponding to the method embodiment shown in FIG. 3, applied to a primary storage server in a data storage system, where the data storage system includes: a management server and a storage server; the storage server includes: a primary storage server and a non-primary storage server; the primary storage server is any one of the storage servers.
  • the device includes:
  • the first receiving module 601 is configured to receive a first data block to be stored to the first stripe
  • a first storage module 602 configured to send, to the management server, a storage address obtaining request for the first data block, and obtain a first storage address determined by the management server according to a stripe storage mechanism, where the first The data block is stored to the first storage address;
  • the first determining module 603 is configured to determine, when the first data block is successfully stored, whether the first data block is the last data block in the first stripe;
  • the first calculating module 604 is configured to: when the determination result of the first determining module 603 is YES, calculate the first one of the first strips according to all the data blocks that have been successfully stored in the first stripe Check block
  • a second storage module 605 configured to send, to the management server, a storage address obtaining request for the first check block, and obtain a second storage address determined by the management server according to a stripe storage mechanism, where the A parity block is stored to the second storage address.
  • the device may further include:
  • a first deleting module (not shown), configured to delete the first data after storing the first data block to the first storage address, when the first data block is not successfully stored
  • the block has stored data and executes the first calculation module 604.
  • the device may further include:
  • a second determining module (not shown) for storing the first check block to the second After storing the address, it is determined whether the first check block is successfully stored, and if not, returning to the first calculating module 604.
  • the device may further include the following modules, which may be as shown in FIG. 7, FIG.
  • the illustrated embodiment corresponds to the method embodiment shown in FIG.
  • the detecting module 606 is configured to detect, when the first data block is not the last data block in the first strip, whether to receive the second data to be stored in the first stripe in the preset time period Piece;
  • the second calculating module 607 is configured to: when the detection result of the detecting module is negative, calculate a second check block in the first strip according to all data blocks that have been successfully stored in the first stripe ;
  • a third storage module 608 configured to send, to the management server, a storage address obtaining request for the second check block, and obtain a third storage address determined by the management server according to a stripe storage mechanism, where the first The second parity block is stored to the third storage address.
  • the device may further include:
  • a second receiving module (not shown) for storing the second parity block to the third storage when the storage unit for storing the data block in the first strip is not full After the address, receiving the second data block;
  • a fourth storage module (not shown), configured to send a storage address obtaining request for the second data block to the management server, and obtain a fourth storage address determined by the management server according to a stripe storage mechanism And storing the second data block to the fourth storage address;
  • a third determining module (not shown), configured to determine, when the second data block is successfully stored, whether the second data block is the last data block in the first stripe;
  • a third calculating module (not shown), configured to calculate the first piece according to all data blocks that have been successfully stored in the first strip when the determining result of the third judging module is YES a third check block in the band;
  • An update module (not shown) for updating the second check block to the third check block.
  • the device may further include:
  • a second deleting module (not shown), configured to: after storing the second data block to the fourth storage address, receive the second data block after storing the second parity block When the first data block is reached, and the second data block is not successfully stored, the stored data of the second data block is deleted.
  • the device may further include:
  • a third deleting module (not shown), configured to receive after the storing the second data block to the fourth storage address, when the third data block is not storing the second parity block When the first data block is reached, and the second data block is not successfully stored, the stored data of the second data block is deleted, and the third calculation module is executed.
  • FIG. 8 is a schematic structural diagram of a data storage system according to an embodiment of the present disclosure.
  • the system includes: a management server 81 and a storage server 82.
  • the storage server 82 includes a primary storage server 821 and a non-primary storage server 822.
  • the primary storage server is any one of the storage servers;
  • the primary storage server 821 is configured to receive a first data block to be stored to the first strip, and send a storage address obtaining request for the first data block to the management server 81;
  • the management server 81 is configured to receive a storage address obtaining request for the first data block sent by the primary storage server 821, determine a first storage address according to a stripe storage mechanism, and send the first storage address To the primary storage server 821;
  • the primary storage server 821 is configured to receive the first storage address sent by the management server 81, store the first data block to the first storage address, and determine that the first data block is successfully stored. Determining whether the first data block is the last data block in the first strip, and if so, calculating the first strip according to all data blocks in the first strip that have been successfully stored. a first parity block in the band; sending a storage address obtaining request for the first parity block to the management server 81;
  • the management server 81 is configured to receive a storage address obtaining request for the first check block sent by the primary storage server 821, determine a second storage address according to a strip storage mechanism, and set the second storage address Send to the primary storage server 821;
  • the primary storage server 821 is configured to receive the second storage address sent by the management server 81, and store the first parity block to the second storage address.
  • the management server 81 is further configured to: when detecting that the first data block is not successfully stored, and when detecting that the primary storage server 821 is generated When abnormal, deleting the stored data of the first data block, selecting a new primary storage server from the non-primary storage server 822, switching the primary storage server to the new primary storage server, and The primary storage server sends a calculation instruction for the first parity block, wherein the calculation instruction carries a storage address of all data blocks that have been successfully stored in the first stripe;
  • the primary storage server is configured to receive the calculation instruction sent by the management server 81, And obtaining a data block according to the storage address carried in the calculation instruction, and calculating a first parity block in the first strip according to the obtained data block.
  • the failure to store the first data block may be caused by various reasons, such as an abnormality occurs in the storage server storing the first data block, or an abnormality may occur in the primary storage server.
  • the management server detects that an abnormality has occurred in the primary storage server, it can reselect the new storage server as the primary storage server from the storage server.
  • the embodiment of the present application discloses a primary storage server in a data storage system, where the data storage system includes: a management server and a storage server; the storage server includes: a primary storage server and a non-primary storage server; The primary storage server is any one of the storage servers; the primary storage server includes:
  • the circuit board is disposed inside a space enclosed by the housing, the processor and the memory are disposed on the circuit board; and the power supply circuit is configured to be the main storage server
  • the memory is for storing executable program code
  • the processor runs the program corresponding to the executable program code by reading executable program code stored in the memory for performing the following steps:
  • the memory mentioned by the primary storage server may include a random access memory (RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.
  • RAM random access memory
  • NVM non-volatile memory
  • the memory may also be at least one storage device located away from the aforementioned processor.
  • the processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., or a digital signal processing (DSP), an application specific integrated circuit. (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component.
  • CPU central processing unit
  • NP network processor
  • DSP digital signal processing
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • the embodiment of the present invention and the method embodiment shown in FIG. 3 are based on the same inventive concept.
  • the specific embodiment of the present embodiment can adopt the content of the method embodiment shown in FIG.
  • the data block can be stored first.
  • the check block is calculated according to the data block successfully stored in the stripe, and then the check block is stored.
  • the check block is first calculated according to the data block, and in the case where both the data block and the check block in the strip are available, the data block and the check block are simultaneously stored. If a power failure occurs during the storage process, both the data block and the check block may be unsuccessful in storage. In this case, the data block and the check block are inconsistent, and a Write Hole problem occurs.
  • the check block is calculated and stored, and even if a power failure occurs during the storage process, the data block and the data block can be avoided as much as possible.
  • the check block is inconsistent.
  • the embodiment of the present application discloses a storage medium for storing executable code, which is used to implement the data storage method provided by the embodiment of the present application.
  • the data storage method includes:
  • the data block can be stored first.
  • the check block is calculated according to the data block successfully stored in the stripe, and then the check block is stored.
  • the check block is first calculated according to the data block, and in the case where both the data block and the check block in the strip are available, the data block and the check block are simultaneously stored. If a power failure occurs during the storage process, both the data block and the check block may be unsuccessful in storage. In this case, the data block and the check block are inconsistent, and a Write Hole problem occurs.
  • the check block is calculated and stored, and even if a power failure occurs during the storage process, the data block and the data block can be avoided as much as possible.
  • the check block is inconsistent.
  • the storage medium referred to herein means a ROM/RAM, a magnetic disk, an optical disk, or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Quality & Reliability (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Retry When Errors Occur (AREA)

Abstract

A data storage method, apparatus and system, relating to the field of computer technology. The data storage method is applied to a main storage server in a data storage system, and the data storage system comprises a management server and a storage server. The method comprises: receiving a first data block to be stored to a first stripe (S301); sending a storage address acquisition request for the first data block to a management server, acquiring a first storage address determined by the management server according to a stripe storage mechanism, and storing the first data block to the first storage address (S302); determining whether the first data block is the last data block in the first stripe (S303); if yes, calculating a first parity block in the first stripe according to all data blocks that have been successfully stored in the first stripe (S304); and sending a storage address acquisition request for the first parity block to the management server, acquiring a second storage address determined by the management server according to the stripe storage mechanism, and storing the first parity block to the second storage address (S305). Applying said method may to the greatest extent avoid inconsistent data blocks and parity blocks.

Description

一种数据存储方法、装置及系统Data storage method, device and system
本申请要求于2016年10月31日提交中国专利局、申请号为201610930966.4、发明名称为“一种数据存储方法、装置及系统”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。The present application claims priority to Chinese Patent Application No. 201610930966.4, entitled "A Data Storage Method, Apparatus and System", filed on October 31, 2016, the entire contents of which is incorporated herein by reference. In the application.
技术领域Technical field
本申请涉及计算机技术领域,特别涉及一种数据存储方法、装置及系统。The present application relates to the field of computer technologies, and in particular, to a data storage method, apparatus, and system.
背景技术Background technique
随着计算机技术的发展,需要存储的数据量呈几何级数增长。为了提高数据存储时的存储性能,目前常用的存储模式为RAID(Redundant Array of Independent Disks,独立磁盘冗余阵列)模式中的RAID5和RAID6。其中,RAID5是一种采用块交叉访问及校验信息均匀分布的模式。RAID5读写操作是以条带为基本单位的,条带被划分为多个存储单元,这些存储单元用于存储数据块和校验块。图1a为条带的一种结构示意图,其中,该条带包含n个数据块(D1、D2、…、Dn)和m个校验块(P1、P2、…、Pm),每个数据块和校验块均分布在不同的磁盘上。在分布式存储系统中,磁盘可以位于不同的服务器上。作为一个例子,当n取3,m取1时,条带对应着4个服务器,图1b所示即为该例中条带与服务器的逻辑关系示意图。其中,包括4个相互独立的服务器,每个服务器中存储有3个数据块和1个校验块,图1b中横向的3个数据块和1个校验块组成一个条带,表1为具体的各个条带的组成。这样,每个条带中的数据块和校验块均分别位于不同的服务器上。With the development of computer technology, the amount of data that needs to be stored has grown geometrically. In order to improve the storage performance during data storage, the current storage mode is RAID 5 and RAID 6 in the RAID (Redundant Array of Independent Disks) mode. Among them, RAID5 is a mode in which block cross-access and check information are evenly distributed. The RAID 5 read and write operations are based on strips. The strips are divided into multiple storage units, which are used to store data blocks and check blocks. Figure 1a is a schematic structural view of a strip, wherein the strip includes n data blocks (D1, D2, ..., Dn) and m check blocks (P1, P2, ..., Pm), each data block And the check blocks are distributed on different disks. In a distributed storage system, disks can be on different servers. As an example, when n takes 3, m takes 1 , the strip corresponds to 4 servers, and FIG. 1b shows a schematic diagram of the logical relationship between the strip and the server in this example. Among them, there are 4 independent servers, each of which stores 3 data blocks and 1 check block. The horizontal 3 data blocks and 1 check block in Fig. 1b form a strip. Table 1 is The composition of each specific strip. Thus, the data blocks and check blocks in each stripe are located on different servers.
表1Table 1
Figure PCTCN2017101847-appb-000001
Figure PCTCN2017101847-appb-000001
在一个条带中,校验块是根据数据块中的数据计算得到的,即数据块D1~Dn与校验块P满足以下关系:D1 xor D2 xor…Dn xor P=0。其中xor为异或运算符。当数据块更新时,对应的校验块也会重新计算并更新。通常,当数据块与校验块符合上述关系时,即称数据块与校验块之间存在一致性。 当数据块和校验块之间存在一致性时,从其中任取个数不少于n的数据块或校验块,即能根据上述关系得到其他数据块或校验块。也就是说,当一个服务器发生故障时,该服务器上的数据可以通过其他服务器上的数据进行恢复。In a stripe, the check block is calculated based on the data in the data block, that is, the data blocks D1 to Dn and the check block P satisfy the following relationship: D1 xor D2 xor...Dn xor P=0. Where xor is the XOR operator. When the data block is updated, the corresponding check block is also recalculated and updated. Generally, when a data block and a check block conform to the above relationship, there is consistency between the data block and the check block. When there is consistency between the data block and the check block, any data block or check block of not less than n is taken from any of them, and other data blocks or check blocks can be obtained according to the above relationship. That is, when a server fails, the data on that server can be recovered from the data on other servers.
相关技术中,存储数据的方法通常是满条带写入服务器中的磁盘,即根据数据块获得校验块,然后将条带中的数据块和校验块同时写入磁盘。但是,在将数据块和校验块同时写入磁盘的过程中,如果系统突然遭遇断电事故,条带数据可能没有完全写入成功。当系统重启之后,条带中的数据可能是不完整的,数据块中的有效数据和校验块中的数据可能存在不一致性。In the related art, the method of storing data is usually a full strip write to a disk in the server, that is, a check block is obtained according to the data block, and then the data block and the check block in the stripe are simultaneously written to the disk. However, in the process of simultaneously writing data blocks and parity blocks to the disk, if the system suddenly encounters a power outage, the stripe data may not be completely written successfully. After the system is restarted, the data in the stripe may be incomplete, and the valid data in the data block and the data in the check block may be inconsistent.
此时,如果某个磁盘失效,则利用未失效磁盘上的数据块和校验块恢复出的数据将是错误的。这种现象也被称为Write Hole(写漏洞)现象。At this point, if a disk fails, it will be wrong to recover the data recovered from the data blocks and check blocks on the non-failed disk. This phenomenon is also known as the Write Hole phenomenon.
发明内容Summary of the invention
本申请实施例的目的在于提供了一种数据存储方法、装置及系统,以尽可能避免出现数据块和校验块不一致的问题。The purpose of the embodiments of the present application is to provide a data storage method, apparatus, and system to avoid the problem of inconsistency between data blocks and check blocks as much as possible.
为了达到上述目的,本申请实施例公开了一种数据存储方法,应用于数据存储系统中的主存储服务器,其中,所述数据存储系统包括:管理服务器和存储服务器;所述存储服务器包括:主存储服务器和非主存储服务器;所述主存储服务器为所述存储服务器中的任意一个;所述方法包括:In order to achieve the above objective, the embodiment of the present application discloses a data storage method, which is applied to a primary storage server in a data storage system, where the data storage system includes: a management server and a storage server; and the storage server includes: a storage server and a non-primary storage server; the primary storage server is any one of the storage servers; the method includes:
接收待存储至第一条带的第一数据块;Receiving a first data block to be stored to the first stripe;
向所述管理服务器发送针对所述第一数据块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第一存储地址,将所述第一数据块存储至所述第一存储地址;Sending, to the management server, a storage address obtaining request for the first data block, and obtaining a first storage address determined by the management server according to a stripe storage mechanism, storing the first data block to the first Storage address
在确定所述第一数据块存储成功时,判断所述第一数据块是否为所述第一条带中的最后一个数据块;When it is determined that the first data block is successfully stored, determining whether the first data block is the last data block in the first stripe;
如果是,则根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第一校验块;If yes, calculating a first check block in the first strip according to all data blocks in the first strip that have been successfully stored;
向所述管理服务器发送针对所述第一校验块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第二存储地址,将所述第一校验块存储至所述第二存储地址。Sending a storage address obtaining request for the first check block to the management server, and obtaining a second storage address determined by the management server according to a stripe storage mechanism, storing the first check block to the The second storage address.
可选的,在将所述第一数据块存储至所述第一存储地址之后,所述方法还包括: Optionally, after the storing the first data block to the first storage address, the method further includes:
当所述第一数据块未存储成功时,删除所述第一数据块的已存储数据,并执行所述根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第一校验块的步骤。Deleting the stored data of the first data block when the first data block is not successfully stored, and performing the first data according to all data blocks that have been successfully stored in the first stripe The step of the first check block in the band.
可选的,在将所述第一校验块存储至所述第二存储地址之后,所述方法还包括:Optionally, after the storing the first check block to the second storage address, the method further includes:
判断所述第一校验块是否存储成功;Determining whether the first check block is successfully stored;
如果否,则返回执行所述根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第一校验块的步骤。If not, returning to performing the step of calculating the first check block in the first strip according to all data blocks that have been successfully stored in the first strip.
可选的,当判断出所述第一数据块不是所述第一条带中的最后一个数据块时,所述方法还包括:Optionally, when it is determined that the first data block is not the last data block in the first strip, the method further includes:
检测预设时间段内是否接收到待存储至所述第一条带的第二数据块;Detecting whether a second data block to be stored to the first stripe is received within a preset time period;
如果未接收到,则根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第二校验块;If not received, calculating a second check block in the first strip according to all data blocks in the first strip that have been successfully stored;
向所述管理服务器发送针对所述第二校验块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第三存储地址,将所述第二校验块存储至所述第三存储地址。Sending a storage address obtaining request for the second check block to the management server, and obtaining a third storage address determined by the management server according to the stripe storage mechanism, storing the second check block to the The third storage address.
可选的,所述第一条带中用于存储数据块的存储单元未存满时,在将所述第二校验块存储至所述第三存储地址之后,所述方法还包括:Optionally, after the storage unit for storing the data block in the first strip is not full, after the storing the second check block to the third storage address, the method further includes:
接收所述第二数据块;Receiving the second data block;
向所述管理服务器发送针对所述第二数据块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第四存储地址,将所述第二数据块存储至所述第四存储地址;Sending a storage address obtaining request for the second data block to the management server, and obtaining a fourth storage address determined by the management server according to the stripe storage mechanism, and storing the second data block to the fourth Storage address
在确定所述第二数据块存储成功时,判断所述第二数据块是否为所述第一条带中的最后一个数据块;When it is determined that the storing of the second data block is successful, determining whether the second data block is the last data block in the first stripe;
如果是,则根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第三校验块;If yes, calculating a third parity block in the first strip according to all data blocks in the first strip that have been successfully stored;
将所述第二校验块更新为所述第三校验块。Updating the second parity block to the third parity block.
可选的,在将所述第二数据块存储至所述第四存储地址之后,所述方法还包括:Optionally, after the storing the second data block to the fourth storage address, the method further includes:
当所述第二数据块为存储所述第二校验块后接收到的第一个数据块,且所述第二数据块未存储成功时,删除所述第二数据块的已存储数据。 When the second data block is the first data block received after storing the second parity block, and the second data block is not successfully stored, the stored data of the second data block is deleted.
可选的,在将所述第二数据块存储至所述第四存储地址之后,所述方法还包括:Optionally, after the storing the second data block to the fourth storage address, the method further includes:
当所述第二数据块不是存储所述第二校验块后接收到的第一个数据块,且所述第二数据块未存储成功时,删除所述第二数据块的已存储数据,并执行所述根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第三校验块的步骤。When the second data block is not the first data block received after storing the second parity block, and the second data block is not successfully stored, deleting the stored data of the second data block, And performing the step of calculating a third parity block in the first strip according to all data blocks that have been successfully stored in the first strip.
可选的,所述第一数据块中的数据来自一个文件。Optionally, the data in the first data block is from a file.
为了达到上述目的,本申请实施例公开了一种数据存储装置,应用于数据存储系统中的主存储服务器,其中,所述数据存储系统包括:管理服务器和存储服务器;所述存储服务器包括:主存储服务器和非主存储服务器;所述主存储服务器为所述存储服务器中的任意一个;所述装置包括:In order to achieve the above objective, the embodiment of the present application discloses a data storage device, which is applied to a primary storage server in a data storage system, where the data storage system includes: a management server and a storage server; and the storage server includes: a storage server and a non-primary storage server; the primary storage server is any one of the storage servers; the device includes:
第一接收模块,用于接收待存储至第一条带的第一数据块;a first receiving module, configured to receive a first data block to be stored to the first stripe;
第一存储模块,用于向所述管理服务器发送针对所述第一数据块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第一存储地址,将所述第一数据块存储至所述第一存储地址;a first storage module, configured to send, to the management server, a storage address obtaining request for the first data block, and obtain a first storage address determined by the management server according to a stripe storage mechanism, and use the first data The block is stored to the first storage address;
第一判断模块,用于在确定所述第一数据块存储成功时,判断所述第一数据块是否为所述第一条带中的最后一个数据块;a first determining module, configured to determine, when the first data block is successfully stored, whether the first data block is the last data block in the first stripe;
第一计算模块,用于当所述第一判断模块的判断结果为是时,根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第一校验块;a first calculating module, configured to: when the determination result of the first determining module is YES, calculate a first check in the first strip according to all data blocks that have been successfully stored in the first stripe Piece;
第二存储模块,用于向所述管理服务器发送针对所述第一校验块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第二存储地址,将所述第一校验块存储至所述第二存储地址。a second storage module, configured to send a storage address obtaining request for the first check block to the management server, and obtain a second storage address determined by the management server according to a stripe storage mechanism, where the first A parity block is stored to the second storage address.
可选的,所述装置还包括:Optionally, the device further includes:
第一删除模块,用于在将所述第一数据块存储至所述第一存储地址之后,当所述第一数据块未存储成功时,删除所述第一数据块的已存储数据,并执行所述第一计算模块。a first deleting module, configured to delete the stored data of the first data block when the first data block is not stored successfully after storing the first data block to the first storage address, and Executing the first computing module.
可选的,所述装置还包括:Optionally, the device further includes:
第二判断模块,用于在将所述第一校验块存储至所述第二存储地址之后,判断所述第一校验块是否存储成功,如果否,则返回执行所述第一计算模块。a second determining module, configured to determine whether the first check block is successfully stored after storing the first check block to the second storage address, and if not, returning to execute the first calculating module .
可选的,在所述第一判断模块之后,所述装置还包括: Optionally, after the first determining module, the device further includes:
检测模块,用于当所述第一数据块不是所述第一条带中的最后一个数据块时,检测预设时间段内是否接收到待存储至所述第一条带的第二数据块;a detecting module, configured to detect, when the first data block is not the last data block in the first strip, whether to receive the second data block to be stored in the first stripe in a preset time period ;
第二计算模块,用于当所述检测模块的检测结果为否时,根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第二校验块;a second calculating module, configured to: when the detection result of the detecting module is negative, calculate a second parity block in the first strip according to all data blocks that have been successfully stored in the first stripe;
第三存储模块,用于向所述管理服务器发送针对所述第二校验块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第三存储地址,将所述第二校验块存储至所述第三存储地址。a third storage module, configured to send a storage address obtaining request for the second check block to the management server, and obtain a third storage address determined by the management server according to a stripe storage mechanism, and the second storage The check block is stored to the third storage address.
可选的,所述装置还包括:Optionally, the device further includes:
第二接收模块,用于所述第一条带中用于存储数据块的存储单元未存满时,在将所述第二校验块存储至所述第三存储地址之后,接收所述第二数据块;a second receiving module, when the storage unit for storing the data block in the first strip is not full, after receiving the second parity block to the third storage address, receiving the first Two data blocks;
第四存储模块,用于向所述管理服务器发送针对所述第二数据块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第四存储地址,将所述第二数据块存储至所述第四存储地址;a fourth storage module, configured to send, to the management server, a storage address obtaining request for the second data block, and obtain a fourth storage address determined by the management server according to a stripe storage mechanism, and the second data is The block is stored to the fourth storage address;
第三判断模块,用于在确定所述第二数据块存储成功时,判断所述第二数据块是否为所述第一条带中的最后一个数据块;a third determining module, configured to determine, when the second data block is successfully stored, whether the second data block is the last data block in the first stripe;
第三计算模块,用于当所述第三判断模块的判断结果为是时,根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第三校验块;a third calculating module, configured to: when the determination result of the third determining module is YES, calculate a third check in the first strip according to all data blocks that have been successfully stored in the first stripe Piece;
更新模块,用于将所述第二校验块更新为所述第三校验块。And an update module, configured to update the second check block to the third check block.
可选的,所述装置还包括:Optionally, the device further includes:
第二删除模块,用于在将所述第二数据块存储至所述第四存储地址之后,当所述第二数据块为存储所述第二校验块后接收到的第一个数据块,且所述第二数据块未存储成功时,删除所述第二数据块的已存储数据。a second deleting module, configured to: after storing the second data block to the fourth storage address, when the second data block is the first data block received after storing the second parity block And storing the stored data of the second data block when the second data block is not successfully stored.
可选的,所述装置还包括:Optionally, the device further includes:
第三删除模块,用于在将所述第二数据块存储至所述第四存储地址之后,当所述第二数据块不是存储所述第二校验块后接收到的第一个数据块,且所述第二数据块未存储成功时,删除所述第二数据块的已存储数据,并执行所述第三计算模块。a third deleting module, configured to: after storing the second data block to the fourth storage address, when the second data block is not the first data block received after storing the second parity block And if the second data block is not successfully stored, the stored data of the second data block is deleted, and the third computing module is executed.
为了达到上述目的,本申请公开了一种数据存储系统,包括:管理服务器和存储服务器;所述存储服务器包括:主存储服务器和非主存储服务器,其中,所述主存储服务器为所述存储服务器中的任意一个; In order to achieve the above object, the present application discloses a data storage system including: a management server and a storage server; the storage server includes: a primary storage server and a non-primary storage server, wherein the primary storage server is the storage server Any of them;
所述主存储服务器,用于接收待存储至第一条带的第一数据块,向所述管理服务器发送针对所述第一数据块的存储地址获得请求;The primary storage server is configured to receive a first data block to be stored to the first strip, and send a storage address obtaining request for the first data block to the management server;
所述管理服务器,用于接收所述主存储服务器发送的针对所述第一数据块的存储地址获得请求,按照条带存储机制确定第一存储地址,并将所述第一存储地址发送至所述主存储服务器;The management server is configured to receive a storage address obtaining request for the first data block sent by the primary storage server, determine a first storage address according to a stripe storage mechanism, and send the first storage address to the Said primary storage server;
所述主存储服务器,用于接收所述管理服务器发送的所述第一存储地址,将所述第一数据块存储至所述第一存储地址;在确定所述第一数据块存储成功时,判断所述第一数据块是否为所述第一条带中的最后一个数据块,如果是,则根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第一校验块;向所述管理服务器发送针对所述第一校验块的存储地址获得请求;The primary storage server is configured to receive the first storage address sent by the management server, store the first data block to the first storage address, and when determining that the first data block is successfully stored, Determining whether the first data block is the last data block in the first strip, and if so, calculating the first strip according to all data blocks that have been successfully stored in the first strip a first parity block; sending a storage address obtaining request for the first parity block to the management server;
所述管理服务器,用于接收所述主存储服务器发送的针对所述第一校验块的存储地址获得请求,按照条带存储机制确定第二存储地址,并将所述第二存储地址发送至所述主存储服务器;The management server is configured to receive a storage address obtaining request for the first check block sent by the primary storage server, determine a second storage address according to a stripe storage mechanism, and send the second storage address to The primary storage server;
所述主存储服务器,用于接收所述管理服务器发送的所述第二存储地址,并将所述第一校验块存储至所述第二存储地址。The primary storage server is configured to receive the second storage address sent by the management server, and store the first parity block to the second storage address.
可选的,所述管理服务器,还用于当检测到所述第一数据块未存储成功,且当检测到所述主存储服务器发生异常时,删除所述第一数据块的已存储数据,从所述非主存储服务器中选择新的主存储服务器,将所述主存储服务器切换为所述新的主存储服务器,并向所述主存储服务器发送针对所述第一校验块的计算指令,其中,所述计算指令携带所述第一条带中已存储成功的所有数据块的存储地址;Optionally, the management server is further configured to: when it is detected that the first data block is not successfully stored, and when detecting that an abnormality occurs in the primary storage server, deleting the stored data of the first data block, Selecting a new primary storage server from the non-primary storage server, switching the primary storage server to the new primary storage server, and transmitting a calculation instruction for the first parity block to the primary storage server The calculation instruction carries a storage address of all data blocks that have been successfully stored in the first stripe;
所述主存储服务器,用于接收所述管理服务器发送的所述计算指令,并根据所述计算指令中携带的存储地址,获得数据块,根据所获得的数据块,计算所述第一条带中的第一校验块。The primary storage server is configured to receive the calculation instruction sent by the management server, and obtain a data block according to the storage address carried in the calculation instruction, and calculate the first strip according to the obtained data block. The first check block in .
为了达到上述目的,本申请实施例公开了一种数据存储系统中的主存储服务器,所述数据存储系统包括:管理服务器和存储服务器;所述存储服务器包括:主存储服务器和非主存储服务器;所述主存储服务器为所述存储服务器中的任意一个;所述主存储服务器包括:In order to achieve the above objective, the embodiment of the present application discloses a primary storage server in a data storage system, where the data storage system includes: a management server and a storage server; the storage server includes: a primary storage server and a non-primary storage server; The primary storage server is any one of the storage servers; the primary storage server includes:
壳体、处理器、存储器、电路板和电源电路,其中,电路板安置在壳体围成的空间内部,处理器和存储器设置在电路板上;电源电路,用于为所述 主存储服务器的各个电路或器件供电;存储器用于存储可执行程序代码;处理器通过读取存储器中存储的可执行程序代码来运行与可执行程序代码对应的程序,以用于执行以下步骤:a housing, a processor, a memory, a circuit board, and a power supply circuit, wherein the circuit board is disposed inside the space enclosed by the housing, the processor and the memory are disposed on the circuit board; and the power circuit is configured to Each circuit or device of the primary storage server is powered; the memory is for storing executable program code; the processor runs the program corresponding to the executable program code by reading executable program code stored in the memory for performing the following steps:
接收待存储至第一条带的第一数据块;Receiving a first data block to be stored to the first stripe;
向所述管理服务器发送针对所述第一数据块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第一存储地址,将所述第一数据块存储至所述第一存储地址;Sending, to the management server, a storage address obtaining request for the first data block, and obtaining a first storage address determined by the management server according to a stripe storage mechanism, storing the first data block to the first Storage address
在确定所述第一数据块存储成功时,判断所述第一数据块是否为所述第一条带中的最后一个数据块;When it is determined that the first data block is successfully stored, determining whether the first data block is the last data block in the first stripe;
如果是,则根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第一校验块;If yes, calculating a first check block in the first strip according to all data blocks in the first strip that have been successfully stored;
向所述管理服务器发送针对所述第一校验块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第二存储地址,将所述第一校验块存储至所述第二存储地址。Sending a storage address obtaining request for the first check block to the management server, and obtaining a second storage address determined by the management server according to a stripe storage mechanism, storing the first check block to the The second storage address.
为了达到上述目的,本申请实施例公开了一种存储介质,用于存储可执行代码,所述可执行代码在运行时用于实现本申请实施例提供的数据存储方法。其中,该数据存储方法包括:In order to achieve the above object, the embodiment of the present application discloses a storage medium for storing executable code, which is used to implement the data storage method provided by the embodiment of the present application. The data storage method includes:
接收待存储至第一条带的第一数据块;Receiving a first data block to be stored to the first stripe;
向所述管理服务器发送针对所述第一数据块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第一存储地址,将所述第一数据块存储至所述第一存储地址;Sending, to the management server, a storage address obtaining request for the first data block, and obtaining a first storage address determined by the management server according to a stripe storage mechanism, storing the first data block to the first Storage address
在确定所述第一数据块存储成功时,判断所述第一数据块是否为所述第一条带中的最后一个数据块;When it is determined that the first data block is successfully stored, determining whether the first data block is the last data block in the first stripe;
如果是,则根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第一校验块;If yes, calculating a first check block in the first strip according to all data blocks in the first strip that have been successfully stored;
向所述管理服务器发送针对所述第一校验块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第二存储地址,将所述第一校验块存储至所述第二存储地址。Sending a storage address obtaining request for the first check block to the management server, and obtaining a second storage address determined by the management server according to a stripe storage mechanism, storing the first check block to the The second storage address.
由上述技术方案可见,本申请实施例中,作为执行主体的主存储服务器在接收到待存储至第一条带的第一数据块时,向管理服务器请求第一数据块的第一存储地址,在获得第一存储地址后将第一数据块存储至第一存储地 址。在确定第一数据块存储成功时,判断该第一数据块是否为第一条带中的最后一个数据块,如果是,则根据第一条带中已存储成功的所有数据块,计算第一条带中的第一校验块。然后,向管理服务器请求第一校验块的第二存储地址,在获得第二存储地址后将第一校验块存储至第二存储地址。As shown in the foregoing technical solution, in the embodiment of the present application, the primary storage server as the execution entity requests the first storage address of the first data block from the management server when receiving the first data block to be stored in the first stripe. Saving the first data block to the first storage location after obtaining the first storage address site. When it is determined that the first data block is successfully stored, it is determined whether the first data block is the last data block in the first strip, and if so, the first data is calculated according to all the data blocks that have been successfully stored in the first strip. The first check block in the strip. Then, the second storage address of the first parity block is requested from the management server, and the first parity block is stored to the second storage address after the second storage address is obtained.
也就是说,本申请实施例中,先存储数据块,在条带中的所有数据块均存储成功时,根据条带中存储成功的数据块计算校验块,再存储校验块。而相关技术中,先根据数据块计算出校验块,在条带中的数据块和校验块均具备的情况下,将数据块和校验块同时进行存储。如果存储过程中遭遇断电事故,数据块和校验块均有可能出现存储不成功的情况,这时将导致数据块和校验块不一致,出现Write Hole问题。应用本申请实施例的方案,在第一条带中的数据块均存储成功的情况下,再计算并存储校验块,即便存储过程中遭遇断电事故,也能够尽可能避免出现数据块和校验块不一致的问题。That is to say, in the embodiment of the present application, the data block is stored first, and when all the data blocks in the stripe are successfully stored, the check block is calculated according to the data block stored in the stripe, and the check block is stored. In the related art, the check block is first calculated according to the data block, and in the case where both the data block and the check block in the strip are available, the data block and the check block are simultaneously stored. If a power failure occurs during the storage process, both the data block and the check block may be unsuccessful in storage. In this case, the data block and the check block are inconsistent, and a Write Hole problem occurs. Applying the solution of the embodiment of the present application, if the data blocks in the first strip are successfully stored, the check block is calculated and stored, and even if a power failure occurs during the storage process, the data block and the data block can be avoided as much as possible. The check block is inconsistent.
附图说明DRAWINGS
为了更清楚地说明本申请实施例或相关技术中的技术方案,下面将对实施例或相关技术描述中所需要使用的附图作简单的介绍。显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings to be used in the embodiments or the related art description will be briefly described below. Obviously, the drawings in the following description are only some of the embodiments of the present application, and those skilled in the art can obtain other drawings according to the drawings without any creative work.
图1a为条带的一种结构示意图;Figure 1a is a schematic structural view of a strip;
图1b为条带与服务器的逻辑关系的一种示意图;Figure 1b is a schematic diagram of the logical relationship between a stripe and a server;
图2为本申请实施例提供的一种实施方式中的数据存储系统的结构示意图;FIG. 2 is a schematic structural diagram of a data storage system according to an embodiment of the present disclosure;
图3为本申请实施例提供的数据存储方法的一种流程示意图;FIG. 3 is a schematic flowchart of a data storage method according to an embodiment of the present application;
图4为本申请实施例提供的数据存储方法的另一种流程示意图;4 is another schematic flowchart of a data storage method according to an embodiment of the present application;
图5a为已存储数据后的条带的一种结构示意图;Figure 5a is a schematic structural diagram of a strip after data has been stored;
图5b为已存储数据后的条带的另一种结构示意图;FIG. 5b is another schematic structural diagram of a strip after data has been stored;
图5c为已存储数据后的条带的再一种结构示意图;FIG. 5c is a schematic structural diagram of another strip after the data has been stored;
图6为本申请实施例提供的数据存储装置的一种结构示意图;FIG. 6 is a schematic structural diagram of a data storage device according to an embodiment of the present disclosure;
图7为本申请实施例提供的数据存储装置的另一种结构示意图;FIG. 7 is another schematic structural diagram of a data storage device according to an embodiment of the present application;
图8为本申请实施例提供的数据存储系统的一种结构示意图。 FIG. 8 is a schematic structural diagram of a data storage system according to an embodiment of the present application.
具体实施方式detailed description
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整的描述。显然,所描述的实施例仅仅是本申请的一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例,都属于本申请保护的范围。The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the accompanying drawings in the embodiments. It is apparent that the described embodiments are only a part of the embodiments of the present application, and not all of them. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.
本申请实施例提供了一种数据存储方法、装置及系统,能够尽可能避免出现数据块和校验块不一致的问题。本申请可以应用于数据存储系统中的主存储服务器。该数据存储系统可以包括管理服务器和存储服务器。存储服务器又可以包括主存储服务器和非主存储服务器,主存储服务器可以为存储服务器中的任意一个。The embodiment of the present application provides a data storage method, device, and system, which can avoid the problem of inconsistency between a data block and a check block as much as possible. The application can be applied to a primary storage server in a data storage system. The data storage system can include a management server and a storage server. The storage server may in turn include a primary storage server and a non-primary storage server, and the primary storage server may be any one of the storage servers.
图2为本申请实施例的一种实施方式中的数据存储系统的结构示意图。这种数据存储系统也可以称为分布式存储系统。该系统可以包括管理服务器21、存储服务器22和客户端23。其中,管理服务器21(Master Data Services,MDS)又称为元数据服务器,用于组织、管理存储服务器22中的存储空间信息。存储服务器22,又称为对象服务器、存储节点等,可以接收客户端23下发的待存储的数据以及向自身和其他存储服务器下发存储任务,承担部分计算任务等。客户端23,用于向存储服务器下发待存储数据。FIG. 2 is a schematic structural diagram of a data storage system according to an embodiment of the present application. Such a data storage system can also be referred to as a distributed storage system. The system can include a management server 21, a storage server 22, and a client 23. The management server 21 (Master Data Services, MDS) is also referred to as a metadata server for organizing and managing storage space information in the storage server 22. The storage server 22, which is also referred to as an object server, a storage node, and the like, can receive data to be stored by the client 23 and issue storage tasks to itself and other storage servers, and undertake partial computing tasks. The client 23 is configured to send data to be stored to the storage server.
下面通过具体实施例,对本申请进行详细说明。The present application will be described in detail below through specific embodiments.
图3为本申请实施例提供的数据存储方法的一种流程示意图,应用于数据存储系统中的主存储服务器。其中,数据存储系统包括:管理服务器和存储服务器。存储服务器包括:主存储服务器和非主存储服务器。主存储服务器为存储服务器中的任意一个。具体的,该方法包括如下步骤:FIG. 3 is a schematic flowchart of a data storage method according to an embodiment of the present disclosure, which is applied to a primary storage server in a data storage system. The data storage system includes: a management server and a storage server. Storage servers include: primary storage servers and non-primary storage servers. The primary storage server is any one of the storage servers. Specifically, the method includes the following steps:
步骤S301:接收待存储至第一条带的第一数据块。其中,第一数据块的数量为一个。Step S301: Receive a first data block to be stored to the first strip. The number of the first data blocks is one.
主存储服务器接收的第一数据块,可以是第一条带的第一个数据块,也可以是第一条带的最后一个数据块。The first data block received by the primary storage server may be the first data block of the first stripe or the last data block of the first stripe.
当第一数据块为第一条带的第一个数据块时,作为执行主体的主存储服务器并不知道第一条带是哪个条带。这时,主存储服务器可以向管理服务器发送条带获得请求,管理服务器接收到主存储服务器发送的条带获得请求后,按照条带分配机制确定第一条带,并将第一条带发送至主存储服务器,主存 储服务器接收并记录第一条带。管理服务器确定第一条带,可以包括确定第一条带的标识。When the first data block is the first data block of the first stripe, the primary storage server as the execution body does not know which strip the first stripe is. At this time, the primary storage server may send a stripe obtaining request to the management server, and after receiving the stripe obtaining request sent by the primary storage server, the management server determines the first strip according to the strip allocation mechanism, and sends the first strip to the Primary storage server The storage server receives and records the first strip. The management server determines the first strip, which may include determining the identity of the first strip.
其中,上述条带分配机制,可以理解为条带分配规则,其可以采用相关技术中的条带分配规则,本实施例对此不再细述。另外,该条带获得请求中可以携带所要获取的条带的格式,该格式可以包括条带中用于存储数据块的存储单元的第一数量、用于存储校验块的存储单元的第二数量。The foregoing strip allocation mechanism can be understood as a stripe allocation rule, which can adopt a stripe allocation rule in the related art, which is not described in detail in this embodiment. In addition, the stripe obtains a format of the stripe to be acquired in the request, the format may include a first quantity of the storage unit for storing the data block in the stripe, and a second quantity of the storage unit for storing the check block. Quantity.
例如,作为执行主体的主存储服务器所要获取的条带中包括4个存储数据块的存储单元和1个存储校验块的存储单元。For example, the stripe to be acquired by the main storage server as the execution subject includes four storage units that store data blocks and one storage unit that stores check blocks.
当第一数据块不为第一条带的第一个数据块时,作为执行主体的主存储服务器已经确定第一条带是哪个条带了。具体的,当第一数据块不为第一条带的第一个数据块时,主存储服务器可以根据保存的第一条带的标记,确定第一数据块对应的第一条带。其中,第一条带的标记可以是当第一数据块为第一条带的第一个数据块时,主存储服务器接收管理服务器确定的第一条带后保存的。When the first data block is not the first data block of the first stripe, the main storage server as the execution body has determined which stripe the first stripe is. Specifically, when the first data block is not the first data block of the first strip, the primary storage server may determine, according to the saved label of the first strip, the first strip corresponding to the first data block. The mark of the first strip may be that when the first data block is the first data block of the first strip, the primary storage server saves after receiving the first strip determined by the management server.
需要说明的是,当客户端将待存储的文件发送至主存储服务器时,主存储服务器可以根据自身存储的条带中每个用于存储数据块的存储单元的大小,对文件数据进行划分,一一获得多个数据块。It should be noted that when the client sends the file to be stored to the primary storage server, the primary storage server may divide the file data according to the size of each storage unit for storing the data block in the strip stored by itself. Get multiple blocks of data one by one.
例如,待存储的文件A的大小为1.6MB,条带中每个用于存储数据块的存储单元的大小为1MB,那么主存储服务器可以将该文件A划分成2个数据块,一个是1MB,一个是0.6MB。步骤S301中接收到的第一数据块,可以是第一个数据块,也可以是第二个数据块。For example, the size of the file A to be stored is 1.6 MB, and the size of each storage unit for storing data blocks in the strip is 1 MB, then the primary storage server can divide the file A into 2 data blocks, one is 1 MB. One is 0.6MB. The first data block received in step S301 may be the first data block or the second data block.
步骤S302:向管理服务器发送针对第一数据块的存储地址获得请求,并获得管理服务器按照条带存储机制确定的第一存储地址,将第一数据块存储至所述第一存储地址。Step S302: Send a storage address obtaining request for the first data block to the management server, and obtain a first storage address determined by the management server according to the stripe storage mechanism, and store the first data block to the first storage address.
其中,条带存储机制,也可以理解为条带存储规则,其中包括条带中各个数据块和各个校验块的存储地址分配规则。本实施例中的条带存储机制可以采用相关技术中的存储机制,本实施例不再对其细述。The stripe storage mechanism can also be understood as a stripe storage rule, which includes a storage address allocation rule of each data block in the stripe and each check block. The stripe storage mechanism in this embodiment may adopt a storage mechanism in the related art, which is not detailed in this embodiment.
具体的,当第一数据块为第一条带中的第一个数据块时,作为执行主体的主存储服务器,可以同时向管理服务器发送针对第一数据块的条带获得请求和针对第一数据块的存储地址获得请求,也可以先后向管理服务器发送针对第一数据块的条带获得请求和针对第一数据块的存储地址获得请求。 Specifically, when the first data block is the first data block in the first strip, the primary storage server as the execution entity may simultaneously send a stripe obtaining request for the first data block to the management server and for the first The storage address of the data block obtains the request, and may also send a stripe obtaining request for the first data block and a storage address obtaining request for the first data block to the management server.
管理服务器确定的用于存储第一数据块的第一存储地址,可以包括:用于存储第一数据块的第一存储服务器以及其中的磁盘。第一存储服务器可以为数据存储系统中所有存储服务器中的任意一个,包括主存储服务器自身。也就是说,第一存储地址可以位于主存储服务器自身,也可以位于该数据存储系统中的其他存储服务器上。The first storage address determined by the management server for storing the first data block may include: a first storage server for storing the first data block and a disk therein. The first storage server can be any of all storage servers in the data storage system, including the primary storage server itself. That is, the first storage address may be located on the primary storage server itself or on other storage servers in the data storage system.
当第一存储地址位于主存储服务器自身时,将第一数据块存储至第一存储地址,可以包括:主存储服务器直接将第一数据块存储至第一存储地址。When the first storage address is located in the primary storage server itself, storing the first data block to the first storage address may include: the primary storage server directly storing the first data block to the first storage address.
当第一存储地址位于该数据存储系统中的其他存储服务器上时,将第一数据块存储至第一存储地址,可以包括:主存储服务器将第一数据块发送至所述其他存储服务器,以使其他存储服务器将第一数据块存储至第一存储地址。When the first storage address is located on another storage server in the data storage system, storing the first data block to the first storage address may include: the primary storage server sending the first data block to the other storage server, to Causes the other storage server to store the first data block to the first storage address.
步骤S303:在确定第一数据块存储成功时,判断第一数据块是否为第一条带中的最后一个数据块,如果是,则执行步骤S304。Step S303: When it is determined that the first data block is successfully stored, it is determined whether the first data block is the last data block in the first strip, and if yes, step S304 is performed.
具体的,本步骤可以包括:判断第一数据块是否已存储成功,如果是,则判断第一数据块是否为第一条带中的最后一个数据块。Specifically, the step may include: determining whether the first data block has been successfully stored, and if yes, determining whether the first data block is the last data block in the first strip.
在判断第一数据块是否已存储成功时,为了提高判断结果的准确性,主存储服务器在将待存储的文件分割成数据块时,可以按照一个数据块对应同一个文件的方式来划分。也就是说,第一数据块中的数据来自一个文件。当一个文件的大小小于条带中用于存储数据的存储单元的大小时,这个文件被当作一个数据块来存储,而不再将其他文件的数据填充在该数据块中。这样,可以避免下述情况:当一个数据块的数据来自至少两个文件时,如果一个文件的数据存储成功,另一个文件的数据存储失败,最终可能无法确定该数据块是否存储成功。When determining whether the first data block has been successfully stored, in order to improve the accuracy of the determination result, the primary storage server may divide the file to be stored into data blocks according to the manner in which one data block corresponds to the same file. That is, the data in the first data block comes from a file. When the size of a file is smaller than the size of the storage unit in the strip for storing data, the file is stored as a block of data, and the data of other files is no longer filled in the block. In this way, the following situation can be avoided: when the data of one data block comes from at least two files, if the data storage of one file is successful, the data storage of the other file fails, and finally it may be impossible to determine whether the data block is successfully stored.
判断第一数据块是否已存储成功,可以包括以下实施方式:Determining whether the first data block has been successfully stored may include the following implementation manners:
当第一存储地址位于主存储服务器自身时,主存储服务器可以直接判断第一数据块是否已存储成功。When the first storage address is located on the primary storage server itself, the primary storage server may directly determine whether the first data block has been successfully stored.
当第一存储地址位于该数据存储系统中的其他存储服务器上时,主存储服务器可以检测预设时间段内是否接收到其他存储服务器发送的第一数据块存储成功的确认信息,如果接收到,则确定第一数据块已存储成功。When the first storage address is located on another storage server in the data storage system, the primary storage server may detect whether the first data block storage success confirmation message sent by the other storage server is received within the preset time period, and if received, Then it is determined that the first data block has been successfully stored.
判断第一数据块是否为第一条带中的最后一个数据块,可以包括:主存储服务器根据自身记录的第一条带中已存储的数据块的数量以及第一条带中 用于存储数据块的存储单元的总数量,判断第一数据块是否为第一条带中的最后一个数据块。Determining whether the first data block is the last data block in the first strip may include: the number of data blocks stored in the first strip according to the first storage server and the first strip in the first strip A total number of storage units for storing data blocks, and determining whether the first data block is the last data block in the first strip.
当判断出第一数据块不为第一条带中的最后一个数据块时,当第一数据块已存储成功时,主存储服务器可以继续接收待存储至第一条带的下一个第一数据块,向管理服务器发送针对该下一个第一数据块的存储地址获得请求,并获得管理服务器按照条带存储机制确定的下一个第一存储地址,将该下一个第一数据块存储至该下一个第一存储地址。直到将第一条带中用于存储数据块的存储单元存满数据,然后再执行步骤S304。When it is determined that the first data block is not the last data block in the first strip, when the first data block has been successfully stored, the primary storage server may continue to receive the next first data to be stored to the first strip. Blocking, sending a storage address obtaining request for the next first data block to the management server, and obtaining a next first storage address determined by the management server according to the stripe storage mechanism, storing the next first data block to the next A first storage address. Until the storage unit for storing the data block in the first strip is full of data, then step S304 is performed.
也就是说,第一条带中的每个数据块是顺序存储的,并且,在第一个数据块存储成功之后,才继续存储第二个数据块,直至第一条带中的最后一个数据块存储成功,才计算并存储校验块。That is to say, each data block in the first strip is stored sequentially, and after the first data block is successfully stored, the second data block is continued to be stored until the last data in the first strip. The block is stored successfully before the check block is calculated and stored.
可以理解的是,对于按照本实施例的方式存储的数据块,当第一条带中的最后一个数据块存储成功时,说明该第一条带中的所有数据块均存储成功。It can be understood that, for the data block stored in the manner of the embodiment, when the last data block in the first strip is successfully stored, all the data blocks in the first strip are stored successfully.
步骤S304:根据第一条带中已存储成功的所有数据块,计算第一条带中的第一校验块。Step S304: Calculate the first check block in the first strip according to all the data blocks that have been successfully stored in the first strip.
具体的,由于通常情况下条带中的各个数据块存储在不同的存储服务器上,因此,根据第一条带中已存储成功的所有数据块,计算第一条带中的第一校验块,可以包括以下实施方式:Specifically, since each data block in the strip is stored on a different storage server, the first check block in the first strip is calculated according to all the data blocks that have been successfully stored in the first strip. Can include the following implementations:
当主存储服务器缓存第一条带中已存储成功的所有数据块时,主存储服务器获得自身缓存的第一条带中已存储成功的所有数据块,并根据获得的数据块,计算第一条带中的第一校验块。When the primary storage server caches all the data blocks that have been successfully stored in the first strip, the primary storage server obtains all the data blocks that have been successfully stored in the first strip of the cache, and calculates the first strip according to the obtained data blocks. The first check block in .
当主存储服务器不缓存第一条带中已存储成功的所有数据块时,主存储服务器分别向各个第一存储服务器发送第一条带数据块的获得请求,其中,第一存储服务器为存储第一条带数据块的服务器。主存储服务器获得各个第一存储服务器发送的第一条带中的数据块,并根据获得的数据块,计算第一条带中的第一校验块。When the primary storage server does not cache all the data blocks that have been successfully stored in the first strip, the primary storage server sends the first strip data block obtaining request to each of the first storage servers, where the first storage server is the storage first. A server that strips data blocks. The primary storage server obtains the data block in the first strip sent by each first storage server, and calculates the first check block in the first strip according to the obtained data block.
需要说明的是,根据数据块计算校验块的方法可以采用相关技术中对应的方法,其具体过程此处不再细述。It should be noted that the method for calculating the check block according to the data block may adopt a corresponding method in the related art, and the specific process is not detailed herein.
步骤S305:向管理服务器发送针对第一校验块的存储地址获得请求,并获得管理服务器按照条带存储机制确定的第二存储地址,将第一校验块存储至第二存储地址。 Step S305: Send a storage address obtaining request for the first check block to the management server, and obtain a second storage address determined by the management server according to the stripe storage mechanism, and store the first check block to the second storage address.
需要说明的是,步骤S305与步骤S302基本类似,因此步骤S305可以参照步骤S302中的相关内容,其具体过程不再细述。It should be noted that the step S305 is substantially similar to the step S302. Therefore, the related content in step S302 can be referred to in step S305, and the specific process is not described in detail.
由上述内容可知,本实施例中,作为执行主体的主存储服务器在接收到待存储至第一条带的第一数据块时,向管理服务器请求第一数据块的第一存储地址,在获得第一存储地址后将第一数据块存储至第一存储地址。在确定第一数据块存储成功时,判断该第一数据块是否为第一条带中的最后一个数据块,如果是,则根据第一条带中已存储成功的所有数据块,计算第一条带中的第一校验块。然后,向管理服务器请求第一校验块的第二存储地址,在获得第二存储地址后将第一校验块存储至第二存储地址。As can be seen from the above, in the embodiment, when the primary storage server that is the execution entity receives the first data block to be stored in the first strip, the first storage address of the first data block is requested from the management server. The first data block is stored to the first storage address after the first storage address. When it is determined that the first data block is successfully stored, it is determined whether the first data block is the last data block in the first strip, and if so, the first data is calculated according to all the data blocks that have been successfully stored in the first strip. The first check block in the strip. Then, the second storage address of the first parity block is requested from the management server, and the first parity block is stored to the second storage address after the second storage address is obtained.
也就是说,本申请实施例中,先存储数据块,在条带中的所有数据块均存储成功时,根据条带中存储成功的数据块计算校验块,再存储校验块。而相关技术中,先根据数据块计算出校验块,在条带中的数据块和校验块均具备的情况下,将数据块和校验块同时进行存储。如果存储过程中遭遇断电事故,数据块和校验块均有可能出现存储不成功的情况,这时将导致数据块和校验块不一致,出现Write Hole问题。应用本申请实施例的方案,在第一条带中的数据块均存储成功的情况下,再计算并存储校验块,即便存储过程中遭遇断电事故,也能够尽可能避免出现数据块和校验块不一致的问题。That is to say, in the embodiment of the present application, the data block is stored first, and when all the data blocks in the stripe are successfully stored, the check block is calculated according to the data block stored in the stripe, and the check block is stored. In the related art, the check block is first calculated according to the data block, and in the case where both the data block and the check block in the strip are available, the data block and the check block are simultaneously stored. If a power failure occurs during the storage process, both the data block and the check block may be unsuccessful in storage. In this case, the data block and the check block are inconsistent, and a Write Hole problem occurs. Applying the solution of the embodiment of the present application, if the data blocks in the first strip are successfully stored, the check block is calculated and stored, and even if a power failure occurs during the storage process, the data block and the data block can be avoided as much as possible. The check block is inconsistent.
作为一种具体的实施方式,在图3所示实施例中,为了进一步保证数据块和校验块的一致性,在步骤S302之后,即在将第一数据块存储至第一存储地址之后,当第一数据块未存储成功时,所述方法还可以包括:As a specific implementation manner, in the embodiment shown in FIG. 3, in order to further ensure the consistency of the data block and the check block, after step S302, that is, after storing the first data block to the first storage address, When the first data block is not successfully stored, the method may further include:
删除第一数据块的已存储数据,并执行步骤S304,即执行根据第一条带中已存储成功的所有数据块,计算第一条带中的第一校验块的步骤。The stored data of the first data block is deleted, and step S304 is performed, that is, the step of calculating the first parity block in the first strip according to all the data blocks that have been successfully stored in the first stripe is performed.
需要说明的是,第一数据块未存储成功,可能是第一数据块的数据只存储了一部分,另一部分没有存储成功;也可能是第一数据块的全部数据均已存储,但是其中部分数据发生错误。It should be noted that the first data block is not stored successfully, and the data of the first data block may only be stored in one part, and the other part is not stored successfully; or all the data of the first data block may be stored, but part of the data is stored. An error occurred.
具体的,当第一数据块未存储成功时,删除第一数据块的已存储数据,可以包括如下实施方式:Specifically, when the first data block is not successfully stored, deleting the stored data of the first data block may include the following implementation manners:
当第一存储地址位于主存储服务器自身时,即第一数据块存储于主存储服务器自身时,主存储服务器可以直接删除第一数据块的已存储数据。When the first storage address is located in the primary storage server itself, that is, when the first data block is stored in the primary storage server itself, the primary storage server may directly delete the stored data of the first data block.
当第一存储地址位于该数据存储系统中的其他存储服务器上时,即第一数据块存储于第一存储服务器时,主存储服务器可以向第一存储服务器发送 删除第一数据块的删除指令,第一存储服务器在接收到主存储服务器发送的删除指令之后删除第一数据块的已存储数据。其中,第一存储服务器为存储第一数据块的服务器。When the first storage address is located on another storage server in the data storage system, that is, when the first data block is stored in the first storage server, the primary storage server may send the first storage server to the first storage server. The delete instruction of the first data block is deleted, and the first storage server deletes the stored data of the first data block after receiving the delete instruction sent by the primary storage server. The first storage server is a server that stores the first data block.
当然,删除第一数据块的已存储数据的步骤,也可以由管理服务器来执行。即,管理服务器可以判断预设时间段内是否接收到第一存储服务器发送的第一数据块存储成功的确认信息,如果未接收到,则确定第一数据块未存储成功,向第一存储服务器发送删除第一数据块的删除指令,第一存储服务器在接收到管理服务器发送的删除指令之后删除第一数据块的已存储数据。其中,第一存储服务器为存储第一数据块的服务器。在这种实施方式中,第一存储服务器可以是主存储服务器,也可以是非主存储服务器。Of course, the step of deleting the stored data of the first data block can also be performed by the management server. That is, the management server may determine whether the confirmation information that the first data block is successfully stored by the first storage server is received within the preset time period, and if not received, determining that the first data block is not successfully stored, to the first storage server. Sending a delete instruction for deleting the first data block, the first storage server deleting the stored data of the first data block after receiving the delete instruction sent by the management server. The first storage server is a server that stores the first data block. In this embodiment, the first storage server may be a primary storage server or a non-primary storage server.
需要说明的是,当第一数据块为第一条带中的第一个数据块时,删除第一数据块的已存储数据之后,该第一条带中不存在已存储成功的数据块,也就不需要执行计算第一校验块的步骤。当第一数据块不为第一条带中的第一个数据块时,删除第一数据块的已存储数据之后,该第一条带中存在已存储成功的数据块,可以继续执行计算第一校验块的步骤。It should be noted that, when the first data block is the first data block in the first strip, after the stored data of the first data block is deleted, there is no data block that has been successfully stored in the first strip. There is no need to perform the step of calculating the first parity block. When the first data block is not the first data block in the first strip, after the stored data of the first data block is deleted, the data block that has been successfully stored in the first stripe may continue to perform the calculation. A step of a check block.
可见,在这种实施方式中,当第一条带中的某个数据块未存储成功时,删除该数据块的已存储数据,根据第一条带中已存储成功的所有数据块计算校验块,并存储校验块。这样,能够避免未存储成功的数据块污染该条带,进一步保证条带中数据块与校验块之间的一致性。It can be seen that, in this implementation manner, when a certain data block in the first strip is not successfully stored, the stored data of the data block is deleted, and the check is calculated according to all the data blocks that have been successfully stored in the first strip. Block and store the check block. In this way, it is possible to prevent the data block that has not been successfully stored from contaminating the stripe, and further ensure the consistency between the data block and the check block in the stripe.
作为一种具体的实施方式,在图3所示实施例中,为了进一步保证数据块和校验块的一致性,在步骤S305之后,即在将第一校验块存储至第二存储地址之后,所述方法还可以包括:As a specific implementation manner, in the embodiment shown in FIG. 3, in order to further ensure the consistency of the data block and the check block, after step S305, that is, after storing the first parity block to the second storage address The method may further include:
判断第一校验块是否存储成功,如果否,则返回执行步骤S304;如果是,则可以不予以处理。It is judged whether the first check block is successfully stored, and if not, the process returns to step S304; if so, it may not be processed.
具体的,判断第一校验块是否存储成功,可以包括以下实施方式:Specifically, determining whether the first check block is successfully stored may include the following implementation manners:
当第二存储地址位于主存储服务器自身时,主存储服务器可以直接判断第一校验块是否已存储成功。When the second storage address is located on the primary storage server itself, the primary storage server may directly determine whether the first verification block has been successfully stored.
当第二存储地址位于该数据存储系统中的其他存储服务器上时,主存储服务器可以检测预设时间段内是否接收到其他存储服务器发送的第一校验块存储成功的确认信息,如果接收到,则确定第一校验块已存储成功。When the second storage address is located on another storage server in the data storage system, the primary storage server may detect whether the confirmation information that the first check block is successfully stored by the other storage server is received within the preset time period, if received Then, it is determined that the first check block has been successfully stored.
当第一校验块未存储成功时,主存储服务器重新根据第一条带中已存储 成功的所有数据块,计算第一条带中的第一校验块;重新向管理服务器发送针对第一校验块的存储地址获得请求,并重新获得管理服务器按照条带存储机制确定的第二存储地址,将第一校验块存储至第二存储地址。When the first parity block is not stored successfully, the primary storage server is again stored according to the first strip. All the data blocks that are successful, calculate the first check block in the first stripe; resend the storage address obtaining request for the first check block to the management server, and regain the second determined by the management server according to the stripe storage mechanism The address is stored, and the first check block is stored to the second storage address.
需要说明的是,第一校验块未存储成功,可能是由于存储第一校验块的存储服务器发生异常,因此需要重新获取第一校验块的存储地址。It should be noted that the first check block is not stored successfully, which may be because the storage server storing the first check block is abnormal, so the storage address of the first check block needs to be re-acquired.
当然,当判断出第一校验块未存储成功时,还可以删除第一校验块的已存储数据,以节省存储空间。Certainly, when it is determined that the first check block is not successfully stored, the stored data of the first check block may also be deleted to save storage space.
在本申请的另一实施例中,在图3所示实施例的基础上,当第一条带中用于存储数据块的存储单元未存满时,如果长时间接收不到其他文件数据,则为了保证数据的安全性,这种情况下仍然要计算并存储校验块。因此,在步骤S303的判断结果为否时,所述方法还可以包括如图4所示的步骤:In another embodiment of the present application, on the basis of the embodiment shown in FIG. 3, when the storage unit for storing the data block in the first strip is not full, if other file data is not received for a long time, In order to ensure the security of the data, in this case, the check block is still calculated and stored. Therefore, when the result of the determination in step S303 is NO, the method may further include the steps as shown in FIG. 4:
步骤S306:当第一数据块不是第一条带中的最后一个数据块时,检测预设时间段内是否接收到待存储至第一条带的第二数据块,如果未接收到,则执行步骤S307。Step S306: When the first data block is not the last data block in the first strip, detecting whether the second data block to be stored to the first stripe is received within the preset time period, if not, executing Step S307.
需要说明的是,如果主存储服务器在预设时间段内未接收到待存储至第一条带的第二数据块,为了避免第一条带长期处于可写入状态,无法计算校验块,可以在第一条带未存满时,计算并存储校验块。这样,管理服务器在检测到校验块存储成功时,可以将第一条带的状态暂时设置为不可写入。It should be noted that, if the primary storage server does not receive the second data block to be stored in the first stripe within the preset time period, in order to prevent the first strip from being in a writable state for a long time, the check block cannot be calculated. The check block can be calculated and stored when the first strip is not full. In this way, the management server can temporarily set the state of the first stripe to be unwritable when it detects that the check block storage is successful.
步骤S307:根据第一条带中已存储成功的所有数据块,计算第一条带中的第二校验块。Step S307: Calculate a second check block in the first strip according to all data blocks that have been successfully stored in the first strip.
步骤S308:向所述管理服务器发送针对第二校验块的存储地址获得请求,并获得管理服务器按照条带存储机制确定的第三存储地址,将第二校验块存储至第三存储地址。Step S308: Send a storage address obtaining request for the second check block to the management server, and obtain a third storage address determined by the management server according to the stripe storage mechanism, and store the second check block to the third storage address.
需要指出的是,图4所示实施例中的步骤S307和S308分别与图3所示实施例中的步骤S304和S305类似,因此,可以参照图3所示实施例中的相关描述。It should be noted that steps S307 and S308 in the embodiment shown in FIG. 4 are similar to steps S304 and S305 in the embodiment shown in FIG. 3, respectively. Therefore, reference may be made to the related description in the embodiment shown in FIG.
可见,在本实施例中,当第一条带中用于存储数据块的存储单元未存满时,如果预设时间段内主存储服务器没有接收到待存储至第一条带的新的数据块,则直接计算并存储校验块,从而可以避免条带长时间处于可写入状态,进一步保证条带中已存储数据的安全性。It can be seen that, in this embodiment, when the storage unit for storing the data block in the first strip is not full, if the primary storage server does not receive the new data to be stored in the first stripe within the preset time period Block, the calculation block is directly calculated and stored, so that the strip can be prevented from being written for a long time, and the security of the stored data in the strip is further ensured.
作为一种具体的实施方式,在图4所示实施例中,当第一条带中用于存 储数据块的存储单元未存满时,在将第二校验块存储至第三存储地址之后,如果接收到了第二数据块,还可以将第二数据块再存储至第一条带。因此,所述方法还可以包括:As a specific implementation manner, in the embodiment shown in FIG. 4, when used in the first strip When the storage unit of the storage block is not full, after storing the second parity block to the third storage address, if the second data block is received, the second data block may be further stored to the first strip. Therefore, the method may further include:
步骤1:接收所述第二数据块。Step 1: Receive the second data block.
步骤2:向管理服务器发送针对第二数据块的存储地址获得请求,并获得管理服务器按照条带存储机制确定的第四存储地址,将第二数据块存储至第四存储地址。Step 2: Send a storage address obtaining request for the second data block to the management server, and obtain a fourth storage address determined by the management server according to the stripe storage mechanism, and store the second data block to the fourth storage address.
步骤3:在确定第二数据块存储成功时,判断第二数据块是否为第一条带中的最后一个数据块,如果是,则执行步骤4。Step 3: When it is determined that the second data block is successfully stored, it is determined whether the second data block is the last data block in the first strip, and if yes, step 4 is performed.
步骤4:根据第一条带中已存储成功的所有数据块,计算第一条带中的第三校验块。Step 4: Calculate the third check block in the first strip according to all the data blocks that have been successfully stored in the first strip.
步骤5:将第二校验块更新为第三校验块。Step 5: Update the second parity block to the third parity block.
下面以一个具体的实例来详细说明上述过程。The above process will be described in detail below with a specific example.
在图5a所示条带A中,包括4个用于存储数据块的存储单元和1个用于存储校验块的存储单元。已知该条带A中存储了数据块D1、数据块D2以及校验块P。当接收到待存储至该条带A中的数据块D3时,向管理服务器发送数据块D3的存储地址获得请求,并获得管理服务器按照条带存储机制确定的存储地址,将数据块D3存储至该存储地址。In the strip A shown in FIG. 5a, four memory cells for storing data blocks and one memory cell for storing parity blocks are included. It is known that the data block D1, the data block D2, and the check block P are stored in the strip A. When receiving the data block D3 to be stored in the strip A, the storage address obtaining request of the data block D3 is sent to the management server, and the storage address determined by the management server according to the stripe storage mechanism is obtained, and the data block D3 is stored to The storage address.
在确定数据块D3存储成功时,如果判断出数据块D3不为条带A中的最后一个数据块,则继续针对接收的待存储至条带A的数据块D4,向管理服务器发送数据块D4的存储地址获得请求,并获得管理服务器按照条带存储机制确定的存储地址,将数据块D4存储至该存储地址。When it is determined that the data block D3 is successfully stored, if it is determined that the data block D3 is not the last data block in the strip A, the data block D4 is further sent to the management server for the received data block D4 to be stored to the strip A. The storage address obtains the request, and obtains the storage address determined by the management server according to the stripe storage mechanism, and stores the data block D4 to the storage address.
在确定数据块D4存储成功时,判断出数据块D4为条带A中的最后一个数据块。此时,根据条带A中已存储成功的数据块D1~D4,计算条带A中的校验块P′,并将校验块P更新为P′。此时条带A变为图5b所示的形式,其中包含数据块D1~D4以及校验块P′。When it is determined that the data block D4 is successfully stored, it is judged that the data block D4 is the last data block in the strip A. At this time, the parity block P' in the strip A is calculated based on the data blocks D1 to D4 that have been successfully stored in the strip A, and the parity block P is updated to P'. At this time, the strip A becomes the form shown in Fig. 5b, which includes the data blocks D1 to D4 and the check block P'.
可见,在本实施方式中,在条带中用于存储数据块的存储单元未存满的情况下,当校验块已被存储之后,主存储服务器再次接收到待存储至该条带的数据块时,存储这些数据块。当该条带中用于存储数据块的存储单元存满时,并且最后一个数据块也存储成功时,重新根据该条带中的数据块计算并更新校验块。 It can be seen that, in this embodiment, in a case where the storage unit for storing the data block in the strip is not full, after the check block has been stored, the primary storage server receives the data to be stored to the strip again. When the block is stored, these data blocks are stored. When the storage unit for storing the data block in the strip is full, and the last data block is also successfully stored, the check block is recalculated and updated according to the data block in the strip.
作为一种具体的实施方式,在图4所示实施例中,为了进一步保证数据块和校验块的一致性,在上述步骤2之后,即在将第二数据块存储至第四存储地址之后,所述方法还可以包括:As a specific implementation manner, in the embodiment shown in FIG. 4, in order to further ensure the consistency of the data block and the check block, after the above step 2, that is, after storing the second data block to the fourth storage address The method may further include:
当第二数据块为存储第二校验块后接收到的第一个数据块,且第二数据块未存储成功时,删除第二数据块的已存储数据。When the second data block is the first data block received after storing the second parity block, and the second data block is not stored successfully, the stored data of the second data block is deleted.
仍然以图5a和图5b所示的条带为例来说明。图5a所示的条带A中已经存储了数据块D1和D2以及校验块P,当接收到数据块D3时,数据块D3是存储校验块P之后接收到的待存储至条带A的第一个数据块,如果数据块D3在存储过程中发生异常,未存储成功,则直接删除D3的已存储数据即可,从而可以保证条带A中数据块和校验块的一致性。即直接删除数据块D3之后,条带A恢复成了图5a的形式,回退到了未写入数据块D3时的版本。The strips shown in Figures 5a and 5b are still taken as an example for illustration. The data blocks D1 and D2 and the check block P have been stored in the strip A shown in FIG. 5a. When the data block D3 is received, the data block D3 is the one to be stored to the strip A after the check block P is stored. The first data block, if the data block D3 is abnormal in the storage process, and the storage is not successful, the stored data of the D3 can be directly deleted, so that the consistency of the data block and the check block in the strip A can be ensured. That is, after the data block D3 is directly deleted, the strip A is restored to the form of FIG. 5a, and is retracted to the version when the data block D3 is not written.
作为一种具体的实施方式,在图4所示实施例中,为了进一步保证数据块和校验块的一致性,在上述步骤2之后,在将第二数据块存储至第四存储地址之后,所述方法还可以包括:As a specific implementation manner, in the embodiment shown in FIG. 4, in order to further ensure the consistency of the data block and the check block, after the step 2 is performed, after storing the second data block to the fourth storage address, The method may further include:
当第二数据块不是存储第二校验块后接收到的第一个数据块,且第二数据块未存储成功时,删除第二数据块的已存储数据,并返回执行根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第三校验块的步骤,即返回执行步骤4。When the second data block is not the first data block received after storing the second parity block, and the second data block is not stored successfully, the stored data of the second data block is deleted, and the execution is performed according to the first All the data blocks that have been successfully stored in the stripe are calculated, and the step of calculating the third check block in the first stripe is returned to step 4.
仍然以图5a和图5b所示的条带为例来说明。图5a所示的条带A中已经存储了数据块D1和D2以及校验块P,当接收到数据块D3时,可以判断出数据块D3是存储校验块P之后接收到的待存储至条带A的第一个数据块。当数据块D3存储成功时,继续存储数据块D4,数据块D4不是存储校验块P之后接收到的待存储至条带A的第一个数据块。当数据块D4未存储成功时,主存储服务器在删除数据块D4的同时,还要根据条带A中已存储成功的D1、D2和D3计算校验块P〞,将校验块P更新为校验块P〞。这时,条带A变成了图5c所示的形式,其中包含数据块D1、D2、D3和校验块P〞。The strips shown in Figures 5a and 5b are still taken as an example for illustration. The data blocks D1 and D2 and the check block P are already stored in the strip A shown in FIG. 5a. When the data block D3 is received, it can be determined that the data block D3 is received after the check block P is stored. The first data block of strip A. When the data block D3 is successfully stored, the data block D4 is continuously stored, and the data block D4 is not the first data block to be stored to the strip A received after the check block P is stored. When the data block D4 is not successfully stored, the primary storage server deletes the data block D4 and calculates the check block P〞 according to the successfully stored D1, D2, and D3 in the strip A, and updates the check block P to Check block P〞. At this time, the strip A becomes the form shown in Fig. 5c, which includes the data blocks D1, D2, D3 and the check block P〞.
在该实施方式中,存储成功的数据块D3已经改变了条带A中的数据块与校验块的一致性,因此,应重新计算并更新校验块,以保证数据块与校验块的一致性。In this embodiment, the successfully stored data block D3 has changed the consistency of the data block and the check block in the strip A. Therefore, the check block should be recalculated and updated to ensure the data block and the check block. consistency.
作为一种具体的实施方式,在图4所示实施例中,为了进一步保证数据块和校验块的一致性,所述方法还可以包括: As a specific implementation manner, in the embodiment shown in FIG. 4, in order to further ensure the consistency of the data block and the check block, the method may further include:
判断第三校验块是否更新成功,如果否,则返回执行根据第一条带中已存储成功的所有数据块,计算第一条带中的第三校验块的步骤,即返回执行步骤4。Determining whether the third check block is successfully updated, if not, returning to perform the step of calculating the third check block in the first strip according to all the data blocks that have been successfully stored in the first strip, that is, returning to step 4 .
仍然以图5a和图5b所示的条带为例来说明。图5a所示的条带A中已经存储了数据块D1和D2以及校验块P,图5b所示的条带A中又添加了数据块D3和D4,并且将校验块P更新为了校验块P′。当检测到更新过程中发生了异常,P′未更新成功时,则可以重新根据数据块D1~D4计算校验块P0,将校验块P′更新为P0,以保证校验块更新成功。The strips shown in Figures 5a and 5b are still taken as an example for illustration. The data blocks D1 and D2 and the check block P have been stored in the strip A shown in FIG. 5a, the data blocks D3 and D4 are added to the strip A shown in FIG. 5b, and the check block P is updated to the school. Check block P'. When an abnormality occurs during the update process and P' is not updated successfully, the check block P0 can be calculated again according to the data blocks D1 to D4, and the check block P' is updated to P0 to ensure that the check block update is successful.
图6为本申请实施例提供的数据存储装置的一种结构示意图,与图3所示方法实施例相对应,应用于数据存储系统中的主存储服务器,其中,数据存储系统包括:管理服务器和存储服务器;所述存储服务器包括:主存储服务器和非主存储服务器;所述主存储服务器为所述存储服务器中的任意一个。FIG. 6 is a schematic structural diagram of a data storage device according to an embodiment of the present disclosure, corresponding to the method embodiment shown in FIG. 3, applied to a primary storage server in a data storage system, where the data storage system includes: a management server and a storage server; the storage server includes: a primary storage server and a non-primary storage server; the primary storage server is any one of the storage servers.
具体的,所述装置包括:Specifically, the device includes:
第一接收模块601,用于接收待存储至第一条带的第一数据块;The first receiving module 601 is configured to receive a first data block to be stored to the first stripe;
第一存储模块602,用于向所述管理服务器发送针对所述第一数据块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第一存储地址,将所述第一数据块存储至所述第一存储地址;a first storage module 602, configured to send, to the management server, a storage address obtaining request for the first data block, and obtain a first storage address determined by the management server according to a stripe storage mechanism, where the first The data block is stored to the first storage address;
第一判断模块603,用于在确定所述第一数据块存储成功时,判断所述第一数据块是否为所述第一条带中的最后一个数据块;The first determining module 603 is configured to determine, when the first data block is successfully stored, whether the first data block is the last data block in the first stripe;
第一计算模块604,用于当所述第一判断模块603的判断结果为是时,根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第一校验块;The first calculating module 604 is configured to: when the determination result of the first determining module 603 is YES, calculate the first one of the first strips according to all the data blocks that have been successfully stored in the first stripe Check block
第二存储模块605,用于向所述管理服务器发送针对所述第一校验块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第二存储地址,将所述第一校验块存储至所述第二存储地址。a second storage module 605, configured to send, to the management server, a storage address obtaining request for the first check block, and obtain a second storage address determined by the management server according to a stripe storage mechanism, where the A parity block is stored to the second storage address.
作为一种具体的实施方式,在图6所示实施例中,所述装置还可以包括:As a specific implementation manner, in the embodiment shown in FIG. 6, the device may further include:
第一删除模块(图中未示出),用于在将所述第一数据块存储至所述第一存储地址之后,当所述第一数据块未存储成功时,删除所述第一数据块的已存储数据,并执行所述第一计算模块604。a first deleting module (not shown), configured to delete the first data after storing the first data block to the first storage address, when the first data block is not successfully stored The block has stored data and executes the first calculation module 604.
作为一种具体的实施方式,在图6所示实施例中,所述装置还可以包括:As a specific implementation manner, in the embodiment shown in FIG. 6, the device may further include:
第二判断模块(图中未示出),用于在将所述第一校验块存储至所述第二 存储地址之后,判断所述第一校验块是否存储成功,如果否,则返回执行所述第一计算模块604。a second determining module (not shown) for storing the first check block to the second After storing the address, it is determined whether the first check block is successfully stored, and if not, returning to the first calculating module 604.
在本申请的另一实施方式中,在图6所示实施例中,在所述第一判断模块603之后,所述装置还可以包括以下模块,这些模块可以如图7所示,图7所示实施例与与图4所示方法实施例相对应。In another embodiment of the present application, in the embodiment shown in FIG. 6, after the first determining module 603, the device may further include the following modules, which may be as shown in FIG. 7, FIG. The illustrated embodiment corresponds to the method embodiment shown in FIG.
检测模块606,用于当所述第一数据块不是所述第一条带中的最后一个数据块时,检测预设时间段内是否接收到待存储至所述第一条带的第二数据块;The detecting module 606 is configured to detect, when the first data block is not the last data block in the first strip, whether to receive the second data to be stored in the first stripe in the preset time period Piece;
第二计算模块607,用于当所述检测模块的检测结果为否时,根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第二校验块;The second calculating module 607 is configured to: when the detection result of the detecting module is negative, calculate a second check block in the first strip according to all data blocks that have been successfully stored in the first stripe ;
第三存储模块608,用于向所述管理服务器发送针对所述第二校验块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第三存储地址,将所述第二校验块存储至所述第三存储地址。a third storage module 608, configured to send, to the management server, a storage address obtaining request for the second check block, and obtain a third storage address determined by the management server according to a stripe storage mechanism, where the first The second parity block is stored to the third storage address.
作为一种具体的实施方式,在图7所示实施例中,所述装置还可以包括:As a specific implementation manner, in the embodiment shown in FIG. 7, the device may further include:
第二接收模块(图中未示出),用于所述第一条带中用于存储数据块的存储单元未存满时,在将所述第二校验块存储至所述第三存储地址之后,接收所述第二数据块;a second receiving module (not shown) for storing the second parity block to the third storage when the storage unit for storing the data block in the first strip is not full After the address, receiving the second data block;
第四存储模块(图中未示出),用于向所述管理服务器发送针对所述第二数据块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第四存储地址,将所述第二数据块存储至所述第四存储地址;a fourth storage module (not shown), configured to send a storage address obtaining request for the second data block to the management server, and obtain a fourth storage address determined by the management server according to a stripe storage mechanism And storing the second data block to the fourth storage address;
第三判断模块(图中未示出),用于在确定所述第二数据块存储成功时,判断所述第二数据块是否为所述第一条带中的最后一个数据块;a third determining module (not shown), configured to determine, when the second data block is successfully stored, whether the second data block is the last data block in the first stripe;
第三计算模块(图中未示出),用于当所述第三判断模块的判断结果为是时,根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第三校验块;a third calculating module (not shown), configured to calculate the first piece according to all data blocks that have been successfully stored in the first strip when the determining result of the third judging module is YES a third check block in the band;
更新模块(图中未示出),用于将所述第二校验块更新为所述第三校验块。An update module (not shown) for updating the second check block to the third check block.
作为一种具体的实施方式,在图7所示实施例中,所述装置还可以包括:As a specific implementation manner, in the embodiment shown in FIG. 7, the device may further include:
第二删除模块(图中未示出),用于在将所述第二数据块存储至所述第四存储地址之后,当所述第二数据块为存储所述第二校验块后接收到的第一个数据块,且所述第二数据块未存储成功时,删除所述第二数据块的已存储数据。a second deleting module (not shown), configured to: after storing the second data block to the fourth storage address, receive the second data block after storing the second parity block When the first data block is reached, and the second data block is not successfully stored, the stored data of the second data block is deleted.
作为一种具体的实施方式,在图7所示实施例中,所述装置还可以包括: As a specific implementation manner, in the embodiment shown in FIG. 7, the device may further include:
第三删除模块(图中未示出),用于在将所述第二数据块存储至所述第四存储地址之后,当所述第三数据块不是存储所述第二校验块后接收到的第一个数据块,且所述第二数据块未存储成功时,删除所述第二数据块的已存储数据,并执行所述第三计算模块。a third deleting module (not shown), configured to receive after the storing the second data block to the fourth storage address, when the third data block is not storing the second parity block When the first data block is reached, and the second data block is not successfully stored, the stored data of the second data block is deleted, and the third calculation module is executed.
图8为本申请实施例提供的数据存储系统的一种结构示意图,该系统包括:管理服务器81和存储服务器82;所述存储服务器82包括:主存储服务器821和非主存储服务器822,其中,所述主存储服务器为所述存储服务器中的任意一个;FIG. 8 is a schematic structural diagram of a data storage system according to an embodiment of the present disclosure. The system includes: a management server 81 and a storage server 82. The storage server 82 includes a primary storage server 821 and a non-primary storage server 822. The primary storage server is any one of the storage servers;
所述主存储服务器821,用于接收待存储至第一条带的第一数据块,向所述管理服务器81发送针对所述第一数据块的存储地址获得请求;The primary storage server 821 is configured to receive a first data block to be stored to the first strip, and send a storage address obtaining request for the first data block to the management server 81;
所述管理服务器81,用于接收所述主存储服务器821发送的针对所述第一数据块的存储地址获得请求,按照条带存储机制确定第一存储地址,并将所述第一存储地址发送至所述主存储服务器821;The management server 81 is configured to receive a storage address obtaining request for the first data block sent by the primary storage server 821, determine a first storage address according to a stripe storage mechanism, and send the first storage address To the primary storage server 821;
所述主存储服务器821,用于接收所述管理服务器81发送的所述第一存储地址,将所述第一数据块存储至所述第一存储地址;在确定所述第一数据块存储成功时,判断所述第一数据块是否为所述第一条带中的最后一个数据块,如果是,则根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第一校验块;向所述管理服务器81发送针对所述第一校验块的存储地址获得请求;The primary storage server 821 is configured to receive the first storage address sent by the management server 81, store the first data block to the first storage address, and determine that the first data block is successfully stored. Determining whether the first data block is the last data block in the first strip, and if so, calculating the first strip according to all data blocks in the first strip that have been successfully stored. a first parity block in the band; sending a storage address obtaining request for the first parity block to the management server 81;
所述管理服务器81,用于接收所述主存储服务器821发送的针对所述第一校验块的存储地址获得请求,按照条带存储机制确定第二存储地址,并将所述第二存储地址发送至所述主存储服务器821;The management server 81 is configured to receive a storage address obtaining request for the first check block sent by the primary storage server 821, determine a second storage address according to a strip storage mechanism, and set the second storage address Send to the primary storage server 821;
所述主存储服务器821,用于接收所述管理服务器81发送的所述第二存储地址,并将所述第一校验块存储至所述第二存储地址。The primary storage server 821 is configured to receive the second storage address sent by the management server 81, and store the first parity block to the second storage address.
作为一种具体的实施方式,在图8所示实施例中,所述管理服务器81,还用于当检测到所述第一数据块未存储成功,且当检测到所述主存储服务器821发生异常时,删除所述第一数据块的已存储数据,从所述非主存储服务器822中选择新的主存储服务器,将所述主存储服务器切换为所述新的主存储服务器,并向所述主存储服务器发送针对所述第一校验块的计算指令,其中,所述计算指令携带所述第一条带中已存储成功的所有数据块的存储地址;As a specific implementation manner, in the embodiment shown in FIG. 8, the management server 81 is further configured to: when detecting that the first data block is not successfully stored, and when detecting that the primary storage server 821 is generated When abnormal, deleting the stored data of the first data block, selecting a new primary storage server from the non-primary storage server 822, switching the primary storage server to the new primary storage server, and The primary storage server sends a calculation instruction for the first parity block, wherein the calculation instruction carries a storage address of all data blocks that have been successfully stored in the first stripe;
所述主存储服务器,用于接收所述管理服务器81发送的所述计算指令, 并根据所述计算指令中携带的存储地址,获得数据块,根据所获得的数据块,计算所述第一条带中的第一校验块。The primary storage server is configured to receive the calculation instruction sent by the management server 81, And obtaining a data block according to the storage address carried in the calculation instruction, and calculating a first parity block in the first strip according to the obtained data block.
可以理解的是,第一数据块未存储成功可能是由多种原因造成的,可以是存储该第一数据块的存储服务器发生了异常,也可以是主存储服务器发生了异常。当管理服务器检测到主存储服务器发生了异常时,可以从存储服务器中重新选择新的存储服务器作为主存储服务器。It can be understood that the failure to store the first data block may be caused by various reasons, such as an abnormality occurs in the storage server storing the first data block, or an abnormality may occur in the primary storage server. When the management server detects that an abnormality has occurred in the primary storage server, it can reselect the new storage server as the primary storage server from the storage server.
由于上述装置实施例和系统实施例是基于方法实施例得到的,与该方法具有相同的技术效果,因此装置实施例和系统实施例的技术效果在此不再赘述。The above-mentioned device embodiments and system embodiments are obtained based on the method embodiments, and have the same technical effects as the method. Therefore, the technical effects of the device embodiments and system embodiments are not described herein again.
对于装置实施例和系统实施例而言,由于其基本相似于方法实施例,所以描述得比较简单,相关之处参见方法实施例的部分说明即可。For the device embodiment and the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.
为了达到上述目的,本申请实施例公开了一种数据存储系统中的主存储服务器,所述数据存储系统包括:管理服务器和存储服务器;所述存储服务器包括:主存储服务器和非主存储服务器;所述主存储服务器为所述存储服务器中的任意一个;所述主存储服务器包括:In order to achieve the above objective, the embodiment of the present application discloses a primary storage server in a data storage system, where the data storage system includes: a management server and a storage server; the storage server includes: a primary storage server and a non-primary storage server; The primary storage server is any one of the storage servers; the primary storage server includes:
壳体、处理器、存储器、电路板和电源电路,其中,电路板安置在壳体围成的空间内部,处理器和存储器设置在电路板上;电源电路,用于为所述主存储服务器的各个电路或器件供电;存储器用于存储可执行程序代码;处理器通过读取存储器中存储的可执行程序代码来运行与可执行程序代码对应的程序,以用于执行以下步骤:a housing, a processor, a memory, a circuit board, and a power supply circuit, wherein the circuit board is disposed inside a space enclosed by the housing, the processor and the memory are disposed on the circuit board; and the power supply circuit is configured to be the main storage server Each circuit or device is powered; the memory is for storing executable program code; the processor runs the program corresponding to the executable program code by reading executable program code stored in the memory for performing the following steps:
接收待存储至第一条带的第一数据块;Receiving a first data block to be stored to the first stripe;
向所述管理服务器发送针对所述第一数据块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第一存储地址,将所述第一数据块存储至所述第一存储地址;Sending, to the management server, a storage address obtaining request for the first data block, and obtaining a first storage address determined by the management server according to a stripe storage mechanism, storing the first data block to the first Storage address
在确定所述第一数据块存储成功时,判断所述第一数据块是否为所述第一条带中的最后一个数据块;When it is determined that the first data block is successfully stored, determining whether the first data block is the last data block in the first stripe;
如果是,则根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第一校验块;If yes, calculating a first check block in the first strip according to all data blocks in the first strip that have been successfully stored;
向所述管理服务器发送针对所述第一校验块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第二存储地址,将所述第一校验块存储至所述第二存储地址。 Sending a storage address obtaining request for the first check block to the management server, and obtaining a second storage address determined by the management server according to a stripe storage mechanism, storing the first check block to the The second storage address.
其中,上述主存储服务器提到的存储器可以包括随机存取存储器(Random Access Memory,RAM),也可以包括非易失性存储器(Non-Volatile Memory,NVM),例如至少一个磁盘存储器。可选的,存储器还可以是至少一个位于远离前述处理器的存储装置。The memory mentioned by the primary storage server may include a random access memory (RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
上述处理器可以是通用处理器,包括中央处理器(Central Processing Unit,CPU)、网络处理器(Network Processor,NP)等;还可以是数字信号处理器(Digital Signal Processing,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现场可编程门阵列(Field-Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。The processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., or a digital signal processing (DSP), an application specific integrated circuit. (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component.
本实施例与图3所示方法实施例是基于同一发明构思得到的实施例,本实施例的具体实施方式均可以采用图3所示方法实施例中的内容。The embodiment of the present invention and the method embodiment shown in FIG. 3 are based on the same inventive concept. The specific embodiment of the present embodiment can adopt the content of the method embodiment shown in FIG.
可见,本实施例可以先存储数据块,在条带中的所有数据块均存储成功时,根据条带中存储成功的数据块计算校验块,再存储校验块。而相关技术中,先根据数据块计算出校验块,在条带中的数据块和校验块均具备的情况下,将数据块和校验块同时进行存储。如果存储过程中遭遇断电事故,数据块和校验块均有可能出现存储不成功的情况,这时将导致数据块和校验块不一致,出现Write Hole问题。应用本申请实施例的方案,在第一条带中的数据块均存储成功的情况下,再计算并存储校验块,即便存储过程中遭遇断电事故,也能够尽可能避免出现数据块和校验块不一致的问题。It can be seen that, in this embodiment, the data block can be stored first. When all the data blocks in the stripe are successfully stored, the check block is calculated according to the data block successfully stored in the stripe, and then the check block is stored. In the related art, the check block is first calculated according to the data block, and in the case where both the data block and the check block in the strip are available, the data block and the check block are simultaneously stored. If a power failure occurs during the storage process, both the data block and the check block may be unsuccessful in storage. In this case, the data block and the check block are inconsistent, and a Write Hole problem occurs. Applying the solution of the embodiment of the present application, if the data blocks in the first strip are successfully stored, the check block is calculated and stored, and even if a power failure occurs during the storage process, the data block and the data block can be avoided as much as possible. The check block is inconsistent.
为了达到上述目的,本申请实施例公开了一种存储介质,用于存储可执行代码,所述可执行代码在运行时用于实现本申请实施例提供的数据存储方法。其中,该数据存储方法包括:In order to achieve the above object, the embodiment of the present application discloses a storage medium for storing executable code, which is used to implement the data storage method provided by the embodiment of the present application. The data storage method includes:
接收待存储至第一条带的第一数据块;Receiving a first data block to be stored to the first stripe;
向所述管理服务器发送针对所述第一数据块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第一存储地址,将所述第一数据块存储至所述第一存储地址;Sending, to the management server, a storage address obtaining request for the first data block, and obtaining a first storage address determined by the management server according to a stripe storage mechanism, storing the first data block to the first Storage address
在确定所述第一数据块存储成功时,判断所述第一数据块是否为所述第一条带中的最后一个数据块;When it is determined that the first data block is successfully stored, determining whether the first data block is the last data block in the first stripe;
如果是,则根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第一校验块; If yes, calculating a first check block in the first strip according to all data blocks in the first strip that have been successfully stored;
向所述管理服务器发送针对所述第一校验块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第二存储地址,将所述第一校验块存储至所述第二存储地址。Sending a storage address obtaining request for the first check block to the management server, and obtaining a second storage address determined by the management server according to a stripe storage mechanism, storing the first check block to the The second storage address.
可见,本实施例可以先存储数据块,在条带中的所有数据块均存储成功时,根据条带中存储成功的数据块计算校验块,再存储校验块。而相关技术中,先根据数据块计算出校验块,在条带中的数据块和校验块均具备的情况下,将数据块和校验块同时进行存储。如果存储过程中遭遇断电事故,数据块和校验块均有可能出现存储不成功的情况,这时将导致数据块和校验块不一致,出现Write Hole问题。应用本申请实施例的方案,在第一条带中的数据块均存储成功的情况下,再计算并存储校验块,即便存储过程中遭遇断电事故,也能够尽可能避免出现数据块和校验块不一致的问题。It can be seen that, in this embodiment, the data block can be stored first. When all the data blocks in the stripe are successfully stored, the check block is calculated according to the data block successfully stored in the stripe, and then the check block is stored. In the related art, the check block is first calculated according to the data block, and in the case where both the data block and the check block in the strip are available, the data block and the check block are simultaneously stored. If a power failure occurs during the storage process, both the data block and the check block may be unsuccessful in storage. In this case, the data block and the check block are inconsistent, and a Write Hole problem occurs. Applying the solution of the embodiment of the present application, if the data blocks in the first strip are successfully stored, the check block is calculated and stored, and even if a power failure occurs during the storage process, the data block and the data block can be avoided as much as possible. The check block is inconsistent.
需要说明的是,在本文中,诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来,而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且,术语“包括”、“包含”或者任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。It should be noted that, in this context, relational terms such as first and second are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply such entities or operations. There is any such actual relationship or order between them. Furthermore, the terms "comprises," "comprising," or "includes" or "includes" or "includes" or "includes" or "includes" or "includes" Other elements, or elements that are inherent to such a process, method, item, or device. An element that is defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device that comprises the element.
本领域普通技术人员可以理解,上述实施方式中的全部或部分步骤是能够通过程序指令相关的硬件来完成的,所述的程序可以存储于计算机可读取存储介质中。这里所称存储介质,是指ROM/RAM、磁碟、光盘等。It will be understood by those skilled in the art that all or part of the steps in the above embodiments can be implemented by hardware related to program instructions, and the programs can be stored in a computer readable storage medium. The storage medium referred to herein means a ROM/RAM, a magnetic disk, an optical disk, or the like.
以上所述仅为本申请的较佳实施例而已,并非用于限定本申请的保护范围。凡在本申请的精神和原则之内所做的任何修改、等同替换、改进等,均包含在本申请的保护范围内。 The above description is only the preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application are included in the scope of the present application.

Claims (19)

  1. 一种数据存储方法,其特征在于,应用于数据存储系统中的主存储服务器,其中,所述数据存储系统包括:管理服务器和存储服务器;所述存储服务器包括:主存储服务器和非主存储服务器;所述主存储服务器为所述存储服务器中的任意一个;所述方法包括:A data storage method, which is applied to a primary storage server in a data storage system, wherein the data storage system includes: a management server and a storage server; the storage server includes: a primary storage server and a non-primary storage server The primary storage server is any one of the storage servers; the method includes:
    接收待存储至第一条带的第一数据块;Receiving a first data block to be stored to the first stripe;
    向所述管理服务器发送针对所述第一数据块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第一存储地址,将所述第一数据块存储至所述第一存储地址;Sending, to the management server, a storage address obtaining request for the first data block, and obtaining a first storage address determined by the management server according to a stripe storage mechanism, storing the first data block to the first Storage address
    在确定所述第一数据块存储成功时,判断所述第一数据块是否为所述第一条带中的最后一个数据块;When it is determined that the first data block is successfully stored, determining whether the first data block is the last data block in the first stripe;
    如果是,则根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第一校验块;If yes, calculating a first check block in the first strip according to all data blocks in the first strip that have been successfully stored;
    向所述管理服务器发送针对所述第一校验块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第二存储地址,将所述第一校验块存储至所述第二存储地址。Sending a storage address obtaining request for the first check block to the management server, and obtaining a second storage address determined by the management server according to a stripe storage mechanism, storing the first check block to the The second storage address.
  2. 根据权利要求1所述的方法,其特征在于,在将所述第一数据块存储至所述第一存储地址之后,所述方法还包括:The method according to claim 1, wherein after storing the first data block to the first storage address, the method further comprises:
    当所述第一数据块未存储成功时,删除所述第一数据块的已存储数据,并执行所述根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第一校验块的步骤。Deleting the stored data of the first data block when the first data block is not successfully stored, and performing the first data according to all data blocks that have been successfully stored in the first stripe The step of the first check block in the band.
  3. 根据权利要求2所述的方法,其特征在于,在将所述第一校验块存储至所述第二存储地址之后,所述方法还包括:The method of claim 2, after the storing the first check block to the second storage address, the method further comprises:
    判断所述第一校验块是否存储成功;Determining whether the first check block is successfully stored;
    如果否,则返回执行所述根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第一校验块的步骤。If not, returning to performing the step of calculating the first check block in the first strip according to all data blocks that have been successfully stored in the first strip.
  4. 根据权利要求2所述的方法,其特征在于,当判断出所述第一数据块不是所述第一条带中的最后一个数据块时,所述方法还包括:The method according to claim 2, wherein when it is determined that the first data block is not the last data block in the first strip, the method further comprises:
    检测预设时间段内是否接收到待存储至所述第一条带的第二数据块;Detecting whether a second data block to be stored to the first stripe is received within a preset time period;
    如果未接收到,则根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第二校验块; If not received, calculating a second check block in the first strip according to all data blocks in the first strip that have been successfully stored;
    向所述管理服务器发送针对所述第二校验块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第三存储地址,将所述第二校验块存储至所述第三存储地址。Sending a storage address obtaining request for the second check block to the management server, and obtaining a third storage address determined by the management server according to the stripe storage mechanism, storing the second check block to the The third storage address.
  5. 根据权利要求4所述的方法,其特征在于,所述第一条带中用于存储数据块的存储单元未存满时,在将所述第二校验块存储至所述第三存储地址之后,所述方法还包括:The method according to claim 4, wherein when the storage unit for storing the data block in the first strip is not full, storing the second parity block to the third storage address Thereafter, the method further includes:
    接收所述第二数据块;Receiving the second data block;
    向所述管理服务器发送针对所述第二数据块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第四存储地址,将所述第二数据块存储至所述第四存储地址;Sending a storage address obtaining request for the second data block to the management server, and obtaining a fourth storage address determined by the management server according to the stripe storage mechanism, and storing the second data block to the fourth Storage address
    在确定所述第二数据块存储成功时,判断所述第二数据块是否为所述第一条带中的最后一个数据块;When it is determined that the storing of the second data block is successful, determining whether the second data block is the last data block in the first stripe;
    如果是,则根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第三校验块;If yes, calculating a third parity block in the first strip according to all data blocks in the first strip that have been successfully stored;
    将所述第二校验块更新为所述第三校验块。Updating the second parity block to the third parity block.
  6. 根据权利要求5所述的方法,其特征在于,在将所述第二数据块存储至所述第四存储地址之后,所述方法还包括:The method according to claim 5, wherein after storing the second data block to the fourth storage address, the method further comprises:
    当所述第二数据块为存储所述第二校验块后接收到的第一个数据块,且所述第二数据块未存储成功时,删除所述第二数据块的已存储数据。When the second data block is the first data block received after storing the second parity block, and the second data block is not successfully stored, the stored data of the second data block is deleted.
  7. 根据权利要求5所述的方法,其特征在于,在将所述第二数据块存储至所述第四存储地址之后,所述方法还包括:The method according to claim 5, wherein after storing the second data block to the fourth storage address, the method further comprises:
    当所述第二数据块不是存储所述第二校验块后接收到的第一个数据块,且所述第二数据块未存储成功时,删除所述第二数据块的已存储数据,并执行所述根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第三校验块的步骤。When the second data block is not the first data block received after storing the second parity block, and the second data block is not successfully stored, deleting the stored data of the second data block, And performing the step of calculating a third parity block in the first strip according to all data blocks that have been successfully stored in the first strip.
  8. 根据权利要求1~7任一项所述的方法,其特征在于,所述第一数据块中的数据来自一个文件。The method according to any one of claims 1 to 7, wherein the data in the first data block is from a file.
  9. 一种数据存储装置,其特征在于,应用于数据存储系统中的主存储服务器,其中,所述数据存储系统包括:管理服务器和存储服务器;所述存储服务器包括:主存储服务器和非主存储服务器;所述主存储服务器为所述存储服务器中的任意一个;所述装置包括: A data storage device, characterized by being applied to a primary storage server in a data storage system, wherein the data storage system comprises: a management server and a storage server; the storage server comprises: a primary storage server and a non-primary storage server The primary storage server is any one of the storage servers; the device includes:
    第一接收模块,用于接收待存储至第一条带的第一数据块;a first receiving module, configured to receive a first data block to be stored to the first stripe;
    第一存储模块,用于向所述管理服务器发送针对所述第一数据块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第一存储地址,将所述第一数据块存储至所述第一存储地址;a first storage module, configured to send, to the management server, a storage address obtaining request for the first data block, and obtain a first storage address determined by the management server according to a stripe storage mechanism, and use the first data The block is stored to the first storage address;
    第一判断模块,用于在确定所述第一数据块存储成功时,判断所述第一数据块是否为所述第一条带中的最后一个数据块;a first determining module, configured to determine, when the first data block is successfully stored, whether the first data block is the last data block in the first stripe;
    第一计算模块,用于当所述第一判断模块的判断结果为是时,根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第一校验块;a first calculating module, configured to: when the determination result of the first determining module is YES, calculate a first check in the first strip according to all data blocks that have been successfully stored in the first stripe Piece;
    第二存储模块,用于向所述管理服务器发送针对所述第一校验块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第二存储地址,将所述第一校验块存储至所述第二存储地址。a second storage module, configured to send a storage address obtaining request for the first check block to the management server, and obtain a second storage address determined by the management server according to a stripe storage mechanism, where the first A parity block is stored to the second storage address.
  10. 根据权利要求9所述的装置,其特征在于,所述装置还包括:The device according to claim 9, wherein the device further comprises:
    第一删除模块,用于在将所述第一数据块存储至所述第一存储地址之后,当所述第一数据块未存储成功时,删除所述第一数据块的已存储数据,并执行所述第一计算模块。a first deleting module, configured to delete the stored data of the first data block when the first data block is not stored successfully after storing the first data block to the first storage address, and Executing the first computing module.
  11. 根据权利要求10所述的装置,其特征在于,所述装置还包括:The device according to claim 10, wherein the device further comprises:
    第二判断模块,用于在将所述第一校验块存储至所述第二存储地址之后,判断所述第一校验块是否存储成功,如果否,则返回执行所述第一计算模块。a second determining module, configured to determine whether the first check block is successfully stored after storing the first check block to the second storage address, and if not, returning to execute the first calculating module .
  12. 根据权利要求10所述的装置,其特征在于,在所述第一判断模块之后,所述装置还包括:The device according to claim 10, wherein after the first determining module, the device further comprises:
    检测模块,用于当所述第一数据块不是所述第一条带中的最后一个数据块时,检测预设时间段内是否接收到待存储至所述第一条带的第二数据块;a detecting module, configured to detect, when the first data block is not the last data block in the first strip, whether to receive the second data block to be stored in the first stripe in a preset time period ;
    第二计算模块,用于当所述检测模块的检测结果为否时,根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第二校验块;a second calculating module, configured to: when the detection result of the detecting module is negative, calculate a second parity block in the first strip according to all data blocks that have been successfully stored in the first stripe;
    第三存储模块,用于向所述管理服务器发送针对所述第二校验块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第三存储地址,将所述第二校验块存储至所述第三存储地址。a third storage module, configured to send a storage address obtaining request for the second check block to the management server, and obtain a third storage address determined by the management server according to a stripe storage mechanism, and the second storage The check block is stored to the third storage address.
  13. 根据权利要求12所述的装置,其特征在于,所述装置还包括:The device of claim 12, wherein the device further comprises:
    第二接收模块,用于所述第一条带中用于存储数据块的存储单元未存满时,在将所述第二校验块存储至所述第三存储地址之后,接收所述第二数据块; a second receiving module, when the storage unit for storing the data block in the first strip is not full, after receiving the second parity block to the third storage address, receiving the first Two data blocks;
    第四存储模块,用于向所述管理服务器发送针对所述第二数据块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第四存储地址,将所述第二数据块存储至所述第四存储地址;a fourth storage module, configured to send, to the management server, a storage address obtaining request for the second data block, and obtain a fourth storage address determined by the management server according to a stripe storage mechanism, and the second data is The block is stored to the fourth storage address;
    第三判断模块,用于在确定所述第二数据块存储成功时,判断所述第二数据块是否为所述第一条带中的最后一个数据块;a third determining module, configured to determine, when the second data block is successfully stored, whether the second data block is the last data block in the first stripe;
    第三计算模块,用于当所述第三判断模块的判断结果为是时,根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第三校验块;a third calculating module, configured to: when the determination result of the third determining module is YES, calculate a third check in the first strip according to all data blocks that have been successfully stored in the first stripe Piece;
    更新模块,用于将所述第二校验块更新为所述第三校验块。And an update module, configured to update the second check block to the third check block.
  14. 根据权利要求13所述的装置,其特征在于,所述装置还包括:The device according to claim 13, wherein the device further comprises:
    第二删除模块,用于在将所述第二数据块存储至所述第四存储地址之后,当所述第二数据块为存储所述第二校验块后接收到的第一个数据块,且所述第二数据块未存储成功时,删除所述第二数据块的已存储数据。a second deleting module, configured to: after storing the second data block to the fourth storage address, when the second data block is the first data block received after storing the second parity block And storing the stored data of the second data block when the second data block is not successfully stored.
  15. 根据权利要求13所述的装置,其特征在于,所述装置还包括:The device according to claim 13, wherein the device further comprises:
    第三删除模块,用于在将所述第二数据块存储至所述第四存储地址之后,当所述第二数据块不是存储所述第二校验块后接收到的第一个数据块,且所述第二数据块未存储成功时,删除所述第二数据块的已存储数据,并执行所述第三计算模块。a third deleting module, configured to: after storing the second data block to the fourth storage address, when the second data block is not the first data block received after storing the second parity block And if the second data block is not successfully stored, the stored data of the second data block is deleted, and the third computing module is executed.
  16. 一种数据存储系统,其特征在于,包括:管理服务器和存储服务器;所述存储服务器包括:主存储服务器和非主存储服务器,其中,所述主存储服务器为所述存储服务器中的任意一个;A data storage system, comprising: a management server and a storage server; the storage server comprises: a primary storage server and a non-primary storage server, wherein the primary storage server is any one of the storage servers;
    所述主存储服务器,用于接收待存储至第一条带的第一数据块,向所述管理服务器发送针对所述第一数据块的存储地址获得请求;The primary storage server is configured to receive a first data block to be stored to the first strip, and send a storage address obtaining request for the first data block to the management server;
    所述管理服务器,用于接收所述主存储服务器发送的针对所述第一数据块的存储地址获得请求,按照条带存储机制确定第一存储地址,并将所述第一存储地址发送至所述主存储服务器;The management server is configured to receive a storage address obtaining request for the first data block sent by the primary storage server, determine a first storage address according to a stripe storage mechanism, and send the first storage address to the Said primary storage server;
    所述主存储服务器,用于接收所述管理服务器发送的所述第一存储地址,将所述第一数据块存储至所述第一存储地址;在确定所述第一数据块存储成功时,判断所述第一数据块是否为所述第一条带中的最后一个数据块,如果是,则根据所述第一条带中已存储成功的所有数据块,计算所述第一条带中的第一校验块;向所述管理服务器发送针对所述第一校验块的存储地址获得请求; The primary storage server is configured to receive the first storage address sent by the management server, store the first data block to the first storage address, and when determining that the first data block is successfully stored, Determining whether the first data block is the last data block in the first strip, and if so, calculating the first strip according to all data blocks that have been successfully stored in the first strip a first parity block; sending a storage address obtaining request for the first parity block to the management server;
    所述管理服务器,用于接收所述主存储服务器发送的针对所述第一校验块的存储地址获得请求,按照条带存储机制确定第二存储地址,并将所述第二存储地址发送至所述主存储服务器;The management server is configured to receive a storage address obtaining request for the first check block sent by the primary storage server, determine a second storage address according to a stripe storage mechanism, and send the second storage address to The primary storage server;
    所述主存储服务器,用于接收所述管理服务器发送的所述第二存储地址,并将所述第一校验块存储至所述第二存储地址。The primary storage server is configured to receive the second storage address sent by the management server, and store the first parity block to the second storage address.
  17. 根据权利要求16所述的系统,其特征在于,The system of claim 16 wherein:
    所述管理服务器,还用于当检测到所述第一数据块未存储成功,且当检测到所述主存储服务器发生异常时,删除所述第一数据块的已存储数据,从所述非主存储服务器中选择新的主存储服务器,将所述主存储服务器切换为所述新的主存储服务器,并向所述主存储服务器发送针对所述第一校验块的计算指令,其中,所述计算指令携带所述第一条带中已存储成功的所有数据块的存储地址;The management server is further configured to: when it is detected that the first data block is not successfully stored, and when detecting that an abnormality occurs in the primary storage server, deleting the stored data of the first data block, from the non- Selecting a new primary storage server from the primary storage server, switching the primary storage server to the new primary storage server, and transmitting a calculation instruction for the first parity block to the primary storage server, where The calculation instruction carries a storage address of all data blocks that have been successfully stored in the first stripe;
    所述主存储服务器,用于接收所述管理服务器发送的所述计算指令,并根据所述计算指令中携带的存储地址,获得数据块,根据所获得的数据块,计算所述第一条带中的第一校验块。The primary storage server is configured to receive the calculation instruction sent by the management server, and obtain a data block according to the storage address carried in the calculation instruction, and calculate the first strip according to the obtained data block. The first check block in .
  18. 一种数据存储系统中的主存储服务器,其特征在于,所述数据存储系统包括:管理服务器和存储服务器;所述存储服务器包括:主存储服务器和非主存储服务器;所述主存储服务器为所述存储服务器中的任意一个;所述主存储服务器包括:A primary storage server in a data storage system, the data storage system includes: a management server and a storage server; the storage server includes: a primary storage server and a non-primary storage server; Any one of the storage servers; the primary storage server includes:
    壳体、处理器、存储器、电路板和电源电路,其中,电路板安置在壳体围成的空间内部,处理器和存储器设置在电路板上;电源电路,用于为所述主存储服务器的各个电路或器件供电;存储器用于存储可执行程序代码;处理器通过读取存储器中存储的可执行程序代码来运行与可执行程序代码对应的程序,以用于执行以下步骤:a housing, a processor, a memory, a circuit board, and a power supply circuit, wherein the circuit board is disposed inside a space enclosed by the housing, the processor and the memory are disposed on the circuit board; and the power supply circuit is configured to be the main storage server Each circuit or device is powered; the memory is for storing executable program code; the processor runs the program corresponding to the executable program code by reading executable program code stored in the memory for performing the following steps:
    接收待存储至第一条带的第一数据块;Receiving a first data block to be stored to the first stripe;
    向所述管理服务器发送针对所述第一数据块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第一存储地址,将所述第一数据块存储至所述第一存储地址;Sending, to the management server, a storage address obtaining request for the first data block, and obtaining a first storage address determined by the management server according to a stripe storage mechanism, storing the first data block to the first Storage address
    在确定所述第一数据块存储成功时,判断所述第一数据块是否为所述第一条带中的最后一个数据块;When it is determined that the first data block is successfully stored, determining whether the first data block is the last data block in the first stripe;
    如果是,则根据所述第一条带中已存储成功的所有数据块,计算所述第 一条带中的第一校验块;If yes, calculating the number according to all the data blocks in the first strip that have been successfully stored. a first check block in a strip;
    向所述管理服务器发送针对所述第一校验块的存储地址获得请求,并获得所述管理服务器按照条带存储机制确定的第二存储地址,将所述第一校验块存储至所述第二存储地址。Sending a storage address obtaining request for the first check block to the management server, and obtaining a second storage address determined by the management server according to a stripe storage mechanism, storing the first check block to the The second storage address.
  19. 一种存储介质,其特征在于,用于存储可执行代码,所述可执行代码在运行时用于执行权利要求1-8任一项所述的数据存储方法步骤。 A storage medium for storing executable code, the executable code being operative to perform the data storage method steps of any of claims 1-8.
PCT/CN2017/101847 2016-10-31 2017-09-15 Data storage method, apparatus and system WO2018076954A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610930966.4 2016-10-31
CN201610930966.4A CN108008909B (en) 2016-10-31 2016-10-31 A kind of date storage method, apparatus and system

Publications (1)

Publication Number Publication Date
WO2018076954A1 true WO2018076954A1 (en) 2018-05-03

Family

ID=62024307

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/101847 WO2018076954A1 (en) 2016-10-31 2017-09-15 Data storage method, apparatus and system

Country Status (2)

Country Link
CN (1) CN108008909B (en)
WO (1) WO2018076954A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110471615B (en) * 2018-05-10 2022-08-02 杭州宏杉科技股份有限公司 Data writing method and device
CN111399780B (en) * 2020-03-19 2021-08-24 蚂蚁金服(杭州)网络技术有限公司 Data writing method, device and equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110154105A1 (en) * 2009-12-22 2011-06-23 David Woodhouse Redundant File System
CN102750195A (en) * 2012-06-07 2012-10-24 浪潮电子信息产业股份有限公司 Method for cluster file system data fault tolerance
CN105404469A (en) * 2015-10-22 2016-03-16 浙江宇视科技有限公司 Video data storage method and system
CN106027638A (en) * 2016-05-18 2016-10-12 华中科技大学 Hadoop data distribution method based on hybrid coding

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8527698B2 (en) * 2010-06-22 2013-09-03 Lsi Corporation Parity-based raid system configured to protect against data corruption caused by the occurrence of write holes
CN103729150A (en) * 2014-01-14 2014-04-16 浪潮电子信息产业股份有限公司 Method for processing Write Hole in memory array
CN105930097B (en) * 2015-05-20 2019-01-29 德州学院 The distribution of small write operation verifies formula disk array in a kind of elimination local parallel

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110154105A1 (en) * 2009-12-22 2011-06-23 David Woodhouse Redundant File System
CN102750195A (en) * 2012-06-07 2012-10-24 浪潮电子信息产业股份有限公司 Method for cluster file system data fault tolerance
CN105404469A (en) * 2015-10-22 2016-03-16 浙江宇视科技有限公司 Video data storage method and system
CN106027638A (en) * 2016-05-18 2016-10-12 华中科技大学 Hadoop data distribution method based on hybrid coding

Also Published As

Publication number Publication date
CN108008909B (en) 2019-11-22
CN108008909A (en) 2018-05-08

Similar Documents

Publication Publication Date Title
AU2017228544B2 (en) Nonvolatile media dirty region tracking
US9268648B1 (en) System and method for consistency verification of replicated data in a recovery system
US10705918B1 (en) Online metadata backup consistency check
US9372743B1 (en) System and method for storage management
US9471448B2 (en) Performing an atomic write operation across multiple storage devices
KR101870521B1 (en) Methods and systems for improving storage journaling
US9836231B2 (en) Managing updates and copying data in a point-in-time copy relationship expressed as source logical addresses and target logical addresses
US10474527B1 (en) Host-assisted error recovery
US10261705B2 (en) Efficient data consistency verification for flash storage
US9740634B2 (en) Establishing a point-in-time copy relationship between source logical addresses and target logical addresses
US20150378642A1 (en) File system back-up for multiple storage medium device
US9329799B2 (en) Background checking for lost writes and data corruption
EP2567320B1 (en) Methods and system for verifying memory device integrity
US20180232277A1 (en) Systems and methods for eliminating write-hole problems on parity-based storage resources during an unexpected power loss
WO2018086409A1 (en) Disk data reading/writing method and device
US11625303B2 (en) Automatic incremental repair of granular filesystem objects
WO2018076954A1 (en) Data storage method, apparatus and system
WO2017087015A1 (en) Count of metadata operations
US20160170842A1 (en) Writing to files and file meta-data
US7577804B2 (en) Detecting data integrity
WO2023147738A1 (en) Data storage method and apparatus of virtual machine, virtual machine and storage medium
US20180246788A1 (en) Method and apparatus for recovering in-memory data processing system
US20220374310A1 (en) Write request completion notification in response to partial hardening of write data
US10956052B1 (en) Online address to hash (A2H) metadata scanner
US9418100B1 (en) System and method for storage management

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17864327

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17864327

Country of ref document: EP

Kind code of ref document: A1