WO2018107828A1 - Method and device for information processing - Google Patents

Method and device for information processing Download PDF

Info

Publication number
WO2018107828A1
WO2018107828A1 PCT/CN2017/101221 CN2017101221W WO2018107828A1 WO 2018107828 A1 WO2018107828 A1 WO 2018107828A1 CN 2017101221 W CN2017101221 W CN 2017101221W WO 2018107828 A1 WO2018107828 A1 WO 2018107828A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
information
strip
block
normal
Prior art date
Application number
PCT/CN2017/101221
Other languages
French (fr)
Chinese (zh)
Inventor
曹力
陈伟
丁智勇
Original Assignee
杭州海康威视数字技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视数字技术股份有限公司 filed Critical 杭州海康威视数字技术股份有限公司
Publication of WO2018107828A1 publication Critical patent/WO2018107828A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614Improving the reliability of storage systems
    • G06F3/0619Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/064Management of blocks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Definitions

  • the present application relates to the field of data processing technologies, and in particular, to an information processing method and apparatus.
  • streaming data such as video data
  • streaming data plays an increasingly important role in human life.
  • streaming data may be lost due to disk corruption, storage node failure, and the like.
  • it is necessary to repair the streaming data.
  • a stripe may correspond to a plurality of stripe blocks, as shown in FIG. 1, N+M stripe blocks, wherein a part of the strips
  • N stripe blocks are used as data blocks to store streaming data
  • another part of stripe blocks, such as M stripe blocks in FIG. 1 as check blocks to store parity data.
  • the check data stored in the check block is obtained by performing erasure coding on the stream data stored in the data block.
  • the data block corresponding to the strip is damaged, it is referred to as a "bad block", resulting in the bad block.
  • the erasure code calculation may be performed according to the check data in the check block, and the streaming data in the bad block is repaired.
  • the number of bad blocks corresponding to the stripe is not greater than the number of check blocks corresponding to the stripe, if the stream type data stored in the bad block corresponding to the stripe is repaired based on the erasure code; If the number of bad blocks corresponding to the strip is greater than the number of check blocks corresponding to the strip, the stream data stored in the bad block corresponding to the strip cannot be repaired. At this time, the strip can be determined to be damaged. Bands. In this case, if the normal data block corresponding to the strip is damaged, which is referred to as "normal block", and the damaged normal block occurs again, the streaming data stored in the re-damaged normal block will no longer be repaired.
  • the purpose of the embodiment of the present application is to provide an information processing method and apparatus, so that when the normal data block corresponding to the damaged strip is damaged again, the streaming data stored in the corrupted normal data block can be repaired.
  • the specific technical solutions are as follows:
  • the embodiment of the present application discloses an information processing method, which is applied to a storage server.
  • the method includes:
  • the first strip information includes: normal block identifier and normal Streaming object data to which the streaming data stored in the block belongs;
  • the normal block corresponding to the identifier of the normal block is used as the first data block, and the first data block is grouped according to the streaming object data to which the streaming data stored in the normal block belongs;
  • the method before the step of acquiring the strip information of the at least one damaged strip from the metadata server, the method further includes:
  • the identifier of the bad block is sent to the metadata server, so that the metadata server updates the stripe information of the normal stripe and the stripe information of the damaged strip according to the identifier of the bad block.
  • the step of obtaining the strip information of the at least one damaged strip from the metadata server includes:
  • the first strip information further includes: an offset of the streaming data stored in the normal block in the streaming object data;
  • the method further includes:
  • the offsets in the streaming object data are in descending order.
  • the first data block in each set of first data blocks is sorted.
  • each stripe corresponds to N data blocks and M check blocks, where N and M are positive integers;
  • the step of constructing at least one new normal strip for each set of first data blocks includes:
  • the first data block in the first data block of the group is subjected to erasure coding calculation for the streaming data stored in each of the N first data blocks in order from the front to the back.
  • NS preset data blocks when the first first data blocks remain in the first data block of the group; and forming the N normal blocks by the S first data blocks and the NS preset data blocks, And performing the erasure code calculation on the stream data stored in the S first data blocks and the data stored in the NS preset data blocks, and obtaining the verification data of the M check blocks, where the S first Constructing a new normal stripe, wherein S is a positive integer, the data block and the M check blocks corresponding to the NS preset data blocks, the S first data blocks, and the NS preset data blocks. S ⁇ N.
  • the method further includes:
  • the embodiment of the present application further discloses an information processing method, which is applied to a metadata server, and the method includes:
  • the storage server uses the strip information of the at least one damaged strip as the first strip information
  • the first strip information includes: The identifier of the block and the streaming object data to which the streaming data stored in the normal block belongs; the normal block corresponding to the identifier of the normal block is used as the first data block, and the streaming object belongs to the streaming data stored in the normal block Data, grouping the first data blocks; constructing at least one new normal strip for each set of first data blocks;
  • the second strip information is stripe information of the at least one new normal stripe
  • the second strip information is stored.
  • each stripe corresponds to N data blocks and M check blocks, where N and M are positive integers;
  • the method further includes:
  • the strip information of the normal strip is updated to the strip information of the damaged strip.
  • the step of sending the stripe information of the at least one damaged strip to the storage server includes:
  • the stripe information of the preset number of damaged strips is currently stored in the metadata server, the stripe information of all the damaged strips is sent to the storage server; or
  • Timing information of all damaged strips currently stored is periodically sent to the storage server.
  • the method further includes:
  • the embodiment of the present application further discloses an information processing apparatus, which is applied to a storage server, where the apparatus includes:
  • a first information acquiring unit configured to acquire strip information of at least one damaged strip from the metadata server, and use the strip information of the at least one damaged strip as the first strip information;
  • the first strip information The method includes: an identifier of a normal block and streaming object data to which the streaming data stored in the normal block belongs;
  • a block grouping unit configured to use the normal block corresponding to the identifier of the normal block as the first data block, and group the first data block according to the streaming object data to which the streaming data stored in the normal block belongs;
  • a strip building unit for constructing at least one new normal strip for each set of first data blocks
  • a second information acquiring unit configured to acquire stripe information of the at least one new normal stripe, and use the stripe information of the at least one new normal stripe as the second stripe information
  • an information sending unit configured to send the second stripe information to the metadata server, so that the metadata server adds the second stripe information.
  • the device further includes:
  • a block detecting unit configured to detect whether a local bad block exists before acquiring the stripe information of the at least one damaged stripe from the metadata server;
  • An identifier sending unit configured to send, by the block detection unit, an identifier of the bad block to the metadata server, so that the metadata server is configured according to the identifier of the bad block Update the strip information of the normal strip and the strip information of the damaged strip.
  • the first information acquiring unit is specifically configured to:
  • the first strip information further includes: an offset of the streaming data stored in the normal block in the streaming object data;
  • the device also includes:
  • a block sorting unit configured to: after the first data block is grouped according to the streaming object data to which the streaming data stored in the normal block belongs, according to the streaming data stored in the normal block, in the streaming object data The order of the shifts from small to large, the first data block in each set of the first data block is sorted.
  • each stripe corresponds to N data blocks and M check blocks, where N and M are positive integers;
  • the strip building unit is specifically configured to:
  • the first data block in the first data block of the group is subjected to erasure coding calculation for the streaming data stored in each of the N first data blocks in order from the front to the back.
  • NS preset data blocks when the first first data blocks remain in the first data block of the group; and forming N normal blocks by the S first data blocks and the NS preset data blocks;
  • the M parity blocks corresponding to the preset data blocks, the S first data blocks, and the NS preset data blocks construct a new normal stripe, where S is a positive integer, S ⁇ N.
  • the device further includes:
  • a storage rejecting unit configured to refuse to store data in the N-S preset data blocks to the storage server.
  • the embodiment of the present application further discloses an information processing apparatus, which is applied to a metadata server, where the apparatus includes:
  • An information sending unit configured to send the strip information of the at least one damaged strip to the storage server, so that the storage server uses the strip information of the at least one damaged strip as the first strip information, the first
  • the stripe information includes: an identifier of a normal block and streaming object data to which the streaming data stored in the normal block belongs; a normal block corresponding to the identifier of the normal block is used as the first data block, and the flow is stored in the normal block.
  • Streaming object data to which the data belongs grouping the first data block; constructing at least one new normal strip for each set of the first data block;
  • An information receiving unit configured to receive second strip information sent by the storage server; the second strip information is stripe information of the at least one new normal stripe;
  • An information storage unit configured to store the second strip information.
  • each stripe corresponds to N data blocks and M check blocks, where N and M are positive integers;
  • the device also includes:
  • An identifier receiving unit configured to receive an identifier of a bad block sent by the storage server before sending the stripe information of the at least one damaged strip to the storage server;
  • An information determining unit configured to determine strip information of a normal stripe corresponding to the identifier of the bad block
  • a block marking unit configured to mark a normal block corresponding to the identifier of the bad block in the strip information of the normal stripe as a bad block
  • An information updating unit configured to: mark a normal block marked as a bad block in the strip information of the normal stripe When the number of the numbers is greater than M, the strip information of the normal strip is updated to the strip information of the damaged strip.
  • the information sending unit is specifically configured to:
  • the stripe information of the preset number of damaged strips is currently stored in the metadata server, the stripe information of all the damaged strips is sent to the storage server; or
  • Timing information of all damaged strips currently stored is periodically sent to the storage server.
  • the device further includes:
  • an information deleting unit configured to delete the first strip information after storing the second strip information.
  • the embodiment of the present application discloses a storage server, including a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus. ;
  • the memory is configured to store an application
  • the processor when used to execute an application stored on the memory, implements the above-mentioned information processing method steps applied to the storage server.
  • the embodiment of the present application discloses a metadata server, including a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory complete each other through the communication bus. Communication;
  • the memory is configured to store an application
  • the processor is configured to implement the information processing method step applied to the metadata server when the application stored on the memory is executed.
  • the embodiment of the present application discloses an application program, which is used to implement the above-mentioned information processing method step applied to the storage server at runtime. .
  • the embodiment of the present application discloses an application program for implementing the above-mentioned information processing method step applied to the metadata server at runtime.
  • the embodiment of the present application discloses a readable storage medium, and the readable storage medium is used.
  • the application is implemented to implement the above-described information processing method steps applied to the storage server.
  • the embodiment of the present application discloses a readable storage medium for storing an application, and the application is implemented to implement the above-mentioned information processing method step applied to the metadata server.
  • An embodiment of the present application provides an information processing method and apparatus, where a storage server obtains first strip information of at least one damaged strip from a metadata server, where the first strip information includes: an identifier of a normal block and a normal block.
  • the streamed object data to which the stored stream data belongs is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and
  • the second stripe information is sent to the metadata server, and the metadata server adds the second stripe information.
  • the normal block corresponding to the damaged strip when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again.
  • the streaming data stored in it can be repaired.
  • any of the products or methods of the present application necessarily does not necessarily require all of the advantages described above to be achieved at the same time.
  • Figure 1 is a schematic view of a strip structure
  • FIG. 2 is a schematic flowchart of an information processing method according to an embodiment of the present application.
  • FIG. 3 is another schematic flowchart of an information processing method according to an embodiment of the present disclosure.
  • FIG. 4 is a schematic diagram of a damaged strip obtained in an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a new normal strip reconstructed in the embodiment of the present application.
  • FIG. 6 is a schematic flowchart diagram of another information processing method according to an embodiment of the present disclosure.
  • FIG. 7 is another schematic flowchart of another information processing method according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic structural diagram of an information processing apparatus according to an embodiment of the present disclosure.
  • FIG. 9 is another schematic structural diagram of an information processing apparatus according to an embodiment of the present disclosure.
  • FIG. 10 is a schematic structural diagram of another information processing apparatus according to an embodiment of the present disclosure.
  • FIG. 11 is another schematic structural diagram of another information processing apparatus according to an embodiment of the present application.
  • the stripe block can store streaming data as a data block, or can store parity data as a check block;
  • the normal block is a normal data block, and the storage server can normally read the streaming data from the normal block;
  • the bad block is a corrupted data block, that is, a damaged normal block, and the storage server cannot read the streaming data from the bad block;
  • normal blocks and bad blocks are collectively referred to as data blocks.
  • FIG. 2 is a schematic flowchart of an information processing method according to an embodiment of the present disclosure, which is applied to a storage server, and the method includes:
  • S201 Obtain at least one stripe information of the damaged strip from the metadata server, and use the stripe information of the at least one damaged strip as the first stripe information;
  • the first strip information may include: an identifier of the normal block and streaming object data to which the streaming data stored in the normal block belongs.
  • the streaming object data may be all the streaming data from the same terminal, or all the streaming data from the same time period, or some streaming data divided by the user, which is not performed by the embodiment of the present application. limited. In the streaming data loss part of the streaming object data, the user can still read other streaming data in the streaming object data.
  • the stripe information of the damaged stripe acquired by the storage server may be: stripe information of all damaged strips currently stored in the metadata server.
  • the metadata server may periodically send the stripe information of all the damaged strips currently stored to the storage server, or when the stripe information of the preset number of damaged strips is stored in the metadata server, The stripe information of the preset number of damaged strips currently stored is sent to the storage server.
  • the information processing method may further include:
  • the storage server can periodically detect whether there is a bad block locally.
  • the streaming data in a normal block fails to be read, the streaming data in the normal block is read again, and if it still fails, the normal block can be determined as a bad block.
  • S302 Send the identifier of the bad block to the metadata server.
  • the metadata server may store the metadata of the stripe, and the metadata of the stripe is stripe information, which may include: an identifier of the hard disk, an offset of the data block in the hard disk, an identifier of the data block, and a storage in the data block.
  • the flow object data to which the streaming data belongs, the offset of the streaming data stored in the data block in the streaming object data, and the storage state of the data block are as shown in Table 1.
  • the stripe block is divided into a data block and a check block.
  • the strip block can be understood as a data block and a strip.
  • a set of strips is a set of data blocks corresponding to one stripe;
  • the hard disk identifier wwn (World Wide Name) is a unique 48-bit or 64-bit number assigned by an authoritative organization. It is specifically defined as a recognized name authorization, usually assigned to the manufacturer by region to distinguish One or a group of network connections; in order to reduce the probability of occurrence of damaged strips, each strip corresponding to one strip corresponds to one hard disk, for example, one strip corresponds to four strip blocks, and the strip corresponds to four different hard disks. ;
  • the offset of the sliced block in the streaming object data can be understood as the offset of the streaming data stored in the data block in the streaming object data; by the offset, combined with the stripe block, it can be understood as a data block.
  • the offset in the hard disk can accurately read the streaming data stored in the data block;
  • the stream object data corresponding to the stripe block can be understood as the stream object data to which the stream data stored in the data block belongs.
  • the metadata of the streaming object data may be stored in the metadata server, where the metadata of the streaming object data may include: an identifier of a strip corresponding to the data block storing the streaming data, and a size of the streaming object data. Information, as shown in Table 2.
  • the set of stripe identifiers is a set of identifiers of strips corresponding to the data blocks storing the stream data.
  • the status of the stripe block can be understood as the storage state of the data block.
  • the storage state of the data block may include a normal state and a damaged state.
  • the storage state of the data block is set to a normal state, that is, the data block is marked as a normal block; when the data block is damaged, the data block is stored in the data block.
  • streaming data is lost, the storage state of the data block is set to a corrupted state, that is, the normal block is marked as a bad block.
  • the metadata server After receiving the identifier of the bad block sent by the storage server, the metadata server determines the stripe information of the normal stripe corresponding to the identifier of the bad block, and marks the normal block corresponding to the identifier of the bad block in the stripe information of the normal stripe.
  • a bad block that is, a "strip block" of the normal block corresponding to the identification of the bad block
  • the state is set to a damaged state; if the number of normal blocks marked as bad blocks in the normal strip is greater than the number of check blocks in the normal strip, the strip information of the normal strip is updated to Strip information for damaged strips.
  • the normal block corresponding to the identifier of the normal block included in the first stripe information is used as the first data block, and the first data block is grouped according to the streaming object data to which the streaming data stored in the normal block belongs.
  • strip information of four damaged strips is acquired, which are strip information of the strip 1, strip information of the strip 2, strip information of the strip 3, and strip 4 Stripe information, wherein the data block 1, the data block 3, the data block 6, the data block 8, the data block 9, the data block 10, the data block 13, and the data block 14 of the four strips are normal blocks, that is, there are 8
  • the stream data 9 is stored in the data block 9, the stream data 10 is stored in the data block 10, the stream data 13 is stored in the data block 13, and the stream data 14 is stored in the data block 14, where the stream data 1 is stored.
  • the streaming data 3, the streaming data 6, the streaming data 8, the streaming data 9 and the streaming data 10 belong to the streaming object data Object1, and the streaming data 13 and the streaming data 14 belong to the streaming object data Object2;
  • the data block 1 the data block 3, the data block 6, the data block 8, the data block 9, and the data may be used.
  • 10 grouped into a group the data blocks 13 and 14 is divided into a set of data blocks.
  • S203 Construct at least one new normal strip for each set of the first data block
  • the data block in order to facilitate data reading, after the plurality of first data blocks are grouped according to the streaming object data to which the streaming data stored in the normal block belongs, the data block may be stored in the data block.
  • the streamed data is sorted in the order of offset from the streamed object data, and the first block of data in each group is sorted, and a new normal strip is constructed according to the sorted first block.
  • the offset of the streaming data stored in the data block in the streaming object data may be included in the stripe information of the acquired corrupted stripe.
  • For each group of first data blocks after sorting in the first data block of the group, performing erasure coding calculation on the streaming data stored in each of the N first data blocks in order from the front to the back, and obtaining M check blocks.
  • Check data construct a new normal strip by each N first data blocks and M check blocks corresponding to the N first data blocks;
  • S is a positive integer, S ⁇ N
  • the preset data block can be an empty data block, that is, a data block that does not store any data, and can also be a data block of all 0s or other known data. This embodiment of the present application does not limit this.
  • N 4.
  • the order of the offset of the streaming data stored in each first data block from the small to the largest in the streaming object data Object1 is: Data 1 - Streaming Data 3 - Streaming Data 6 - Streaming Data 8 - Streaming Data 9 - Streaming Data 10
  • the ordering of the first data block is: Data Block 1 - Data Block 3 - Data Block 6 - Data block 8 - data block 9 - data block 10 if one strip corresponds to 4 data blocks and 1 check block, the flow data stored in every 4 first data blocks is corrected in order from front to back.
  • the code deletion calculation that is, the first four normal blocks of the data block 1, the data block 3, the data block 6, and the data block 8, are first obtained, and the erasure code calculation is performed on the stream data stored in the four normal blocks to obtain 1
  • One test data 1 the one check data 1 is stored in the check block 1, as shown in FIG. 5, and one block is composed of the data block 1, the data block 3, the data block 6, the data block 8, and the check block 1.
  • the -2 2 preset data blocks, which are composed of data block 9, data block 10 and the two preset data blocks, and form 4 normal blocks, and perform erasure code calculation on the data stored in the four data blocks to obtain 1 test data 2, the 1 check data 2 is stored in the check block 2, as shown in FIG. 5, the data block 9, the data block 10, the two preset data blocks and the check block 2 constitute one New normal strip 6.
  • the data in the N-S preset data blocks is refused to be stored to the storage server.
  • S204 Obtain stripe information of at least one new normal stripe, and use stripe information of at least one new normal stripe as the second stripe information.
  • S205 Send the second stripe information to the metadata server.
  • the metadata server adds the second stripe information. In this way, when the normal data block is damaged and the streaming data is lost, the streaming data stored in the corrupted normal data block can be repaired.
  • the metadata server may delete the first stripe information after receiving the second stripe information.
  • the new normal strip 7 will be used. 5.
  • the strip information of the new normal strip 6 and the new normal strip 7 is used as the second strip information and sent to the metadata server, and the new normal strip 5, the new normal strip 6 and the new normal are added to the metadata server.
  • the strip information of the strip 7 is additionally deleted for the strip information of the original strip 1, the strip information of the strip 2, the strip information of the strip 3, and the strip information of the strip 4.
  • the streaming data is read according to the metadata of the streaming object data. After the strip is reconstructed, the data block storing the streaming data and the corresponding stripe are changed. Therefore, in order to ensure the reading of the streaming data. After the metadata server obtains the second strip information, it should also update the metadata of the streaming object data.
  • the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs
  • the object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element.
  • the data server, the metadata server adds the second stripe information.
  • the normal block corresponding to the damaged strip when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again.
  • the streaming data stored in it can be repaired.
  • FIG. 6 is a schematic flowchart of another information processing method according to an embodiment of the present application, which is applied to a metadata server, and the method includes:
  • S601 Send the strip information of the at least one damaged strip to the storage server;
  • the storage server may use the stripe information of the at least one damaged stripe as the first stripe information, where the first stripe information includes: an identifier of the normal block and a streamed data stored in the normal block. Streaming object data; the normal block corresponding to the identifier of the normal block is used as the first data Blocking, grouping the first data blocks according to the streaming object data to which the streaming data stored in the normal block belongs; constructing at least one new normal strip for each set of the first data block.
  • the stripe information of the damaged stripe acquired by the storage server may be: stripe information of all damaged strips currently stored in the metadata server.
  • the metadata server may periodically send the stripe information of all the damaged strips currently stored to the storage server, or when the stripe information of the preset number of damaged strips is stored in the metadata server, The strip information of the preset number of damaged strips currently stored is sent to the storage server.
  • each strip corresponds to N data blocks and M check blocks, where N and M are positive integers, and at least one piece is damaged.
  • the information processing method may further include:
  • S701 Receive an identifier of a bad block sent by the storage server.
  • the storage server can periodically detect whether there is a bad block locally, and when the presence is detected, send the identifier of the bad block to the metadata server.
  • S702 Determine strip information of a normal stripe corresponding to the identifier of the bad block
  • S703 mark the normal block corresponding to the identifier of the bad block in the strip information of the normal stripe as a bad block;
  • S602 Receive second strip information sent by the storage server; the second strip information is strip information of at least one new normal stripe;
  • S603 Store the second strip information.
  • the metadata server may also receive the second strip information after receiving the second strip information. Delete the first strip information.
  • the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs
  • the object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the data blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal strip.
  • the metadata server adds the second strip information.
  • the normal block corresponding to the damaged strip when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the data block of the new normal strip is normal. Block, so according to the information of the new normal stripe, the stream data stored in the normal block that is damaged again can be repaired, thereby ensuring that the data block corresponding to the damaged strip is damaged again, and the damaged data block is stored. Repair of streaming data.
  • the embodiment of the present application further provides an information processing apparatus.
  • FIG. 8 is a schematic structural diagram of an information processing apparatus according to an embodiment of the present application, which is applied to a storage server, where the apparatus includes:
  • a first information acquiring unit 801 configured to acquire strip information of at least one damaged strip from the metadata server, and use strip information of the at least one damaged strip as first strip information; the first strip The information includes: an identifier of a normal block and streaming object data to which the streaming data stored in the normal block belongs;
  • a block grouping unit 802 configured to use the normal block corresponding to the identifier of the normal block as the first data block, and group the first data block according to the streaming object data to which the streaming data stored in the normal block belongs;
  • a strip construction unit 803 configured to construct at least one new normal strip for each set of first data blocks
  • a second information acquiring unit 804 configured to acquire stripe information of the at least one new normal stripe, and use stripe information of the at least one new normal stripe as the second stripe information;
  • the information sending unit 805 is configured to send the second stripe information to the metadata server, so that the metadata server adds the second stripe information.
  • the apparatus may further include:
  • the block detecting unit 901 is configured to detect whether a bad block exists locally before acquiring the stripe information of the at least one damaged stripe from the metadata server;
  • the identifier sending unit 902 is configured to send the identifier of the bad block to the metadata server if the detection result of the block detecting unit 901 is yes, so that the metadata server is configured according to the The identification of the bad block updates the strip information of the normal strip and the strip information of the damaged strip.
  • the first information acquiring unit 801 may be specifically configured to:
  • the first stripe information may further include: an offset of the streaming data stored in the normal block in the streaming object data;
  • the device may further include:
  • a block sorting unit configured to: after the first data block is grouped according to the streaming object data to which the streaming data stored in the normal block belongs, according to the streaming data stored in the normal block, in the streaming object data The order of the shifts from small to large, the first data block in each set of the first data block is sorted.
  • each strip corresponds to N data blocks and M check blocks, where N and M are positive integers;
  • the strip construction unit 803 can be specifically configured to:
  • the first data block in the first data block of the group is subjected to erasure coding calculation for the streaming data stored in each of the N first data blocks in order from the front to the back.
  • NS preset data blocks when the first first data blocks remain in the first data block of the group; and forming N normal blocks by the S first data blocks and the NS preset data blocks; And performing the erasure code calculation on the stream data stored in the S first data blocks and the data stored in the NS preset data blocks, and obtaining the verification data of the M check blocks, where the S first Constructing a new normal stripe, wherein S is a positive integer, the data block and the M check blocks corresponding to the NS preset data blocks, the S first data blocks, and the NS preset data blocks. S ⁇ N.
  • the apparatus may further include:
  • a storage rejecting unit configured to refuse to store data in the N-S preset data blocks to the storage Storage server.
  • the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs
  • the object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element.
  • the data server, the metadata server adds the second stripe information.
  • the normal block corresponding to the damaged strip when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again.
  • the streaming data stored in it can be repaired.
  • FIG. 10 is a schematic structural diagram of another information processing apparatus according to an embodiment of the present disclosure, which is applied to a metadata server, where the apparatus includes:
  • the information sending unit 1001 is configured to send the strip information of the at least one damaged strip to the storage server, so that the storage server uses the strip information of the at least one damaged strip as the first strip information, where the The information of the strip includes: the identifier of the normal block and the stream object data to which the stream data stored in the normal block belongs; the normal block corresponding to the identifier of the normal block is used as the first data block, and the stream stored in the normal block is used. Streaming object data to which the data belongs, grouping the first data block; constructing at least one new normal strip for each set of the first data block;
  • the information receiving unit 1002 is configured to receive second strip information sent by the storage server, where the second strip information is strip information of the at least one new normal stripe;
  • the information storage unit 1003 is configured to store the second strip information.
  • each strip corresponds to N data blocks and M check blocks, where N and M are positive integers;
  • the apparatus may further include:
  • the identifier receiving unit 1101 is configured to receive an identifier of the bad block sent by the storage server before sending the stripe information of the at least one damaged strip to the storage server;
  • the information determining unit 1102 is configured to determine strip information of a normal stripe corresponding to the identifier of the bad block;
  • a block marking unit 1103, configured to mark a normal block corresponding to the identifier of the bad block in the stripe information of the normal stripe as a bad block;
  • the information updating unit 1104 is configured to update the strip information of the normal stripe to a strip of the damaged strip when the number of normal blocks marked as bad blocks in the strip information of the normal strip is greater than M information.
  • the information sending unit 1001 may be specifically configured to:
  • the stripe information of the preset number of damaged strips is currently stored in the metadata server, the stripe information of all the damaged strips is sent to the storage server; or
  • the stripe information of all damaged strips currently stored is periodically sent to the storage server.
  • the apparatus further includes:
  • an information deleting unit configured to delete the first strip information after storing the second strip information.
  • the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs
  • the object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element.
  • the data server, the metadata server adds the second stripe information.
  • the normal block corresponding to the damaged strip when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again.
  • the streaming data stored in it can be repaired.
  • the embodiment of the present application further provides a storage server, including a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory complete communication with each other through a communication bus;
  • the information processing method includes:
  • the first stripe information includes: an identifier of the normal block and the storage in the normal block Streaming object data to which streaming data belongs;
  • the normal block corresponding to the identifier of the normal block is used as the first data block, and the first data block is grouped according to the streaming object data to which the streaming data stored in the normal block belongs;
  • the second stripe information is sent to the metadata server to cause the metadata server to add the second stripe information.
  • the method may further include:
  • the identifier of the bad block is sent to the metadata server, so that the metadata server updates the stripe information of the normal stripe and the stripe information of the damaged stripe according to the identifier of the bad block.
  • the step of acquiring strip information of at least one damaged strip from the metadata server includes:
  • the first stripe information may further include: an offset of the streaming data stored in the normal block in the streaming object data;
  • the method may further include:
  • the first data block in each set of the first data block is sorted in order of the offset of the streaming data stored in the normal block in the streaming object data from small to large.
  • each strip corresponds to N data blocks and M check blocks, where N and M are positive integers;
  • the step of constructing at least one new normal strip for each set of first data blocks includes:
  • the first data block in the first data block of the group is subjected to erasure coding calculation for the streaming data stored in each of the N first data blocks in order from the front to the back.
  • the NS preset data blocks are obtained; the S first data blocks and the NS preset data blocks form N normal blocks, and the first ones are S.
  • the stream data stored in the data block and the data stored in the NS preset data blocks are subjected to erasure code calculation, and the check data of the M check blocks is obtained, and the S first data blocks correspond to the NS preset data blocks.
  • M check blocks, S first data blocks, and NS preset data blocks are constructed to construct a new normal stripe, where S is a positive integer, S ⁇ N.
  • the information processing method may further include:
  • the data in the N-S preset data blocks is refused to be stored to the storage server.
  • the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs
  • the object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element.
  • the data server, the metadata server adds the second stripe information.
  • the normal block corresponding to the damaged strip when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again.
  • the streaming data stored in it can be repaired.
  • the communication bus mentioned by the above storage server may be PCI (Peripheral Component) Interconnect, peripheral component interconnect standard) bus or EISA (Extended Industry Standard Architecture) bus.
  • the communication bus can be divided into an address bus, a data bus, a control bus, and the like.
  • the communication interface is used for communication between the above storage server and other devices.
  • the memory may include a RAM (Random Access Memory), and may also include NVM (Non-Volatile Memory), such as at least one disk storage.
  • the memory may also be at least one storage device located away from the aforementioned processor.
  • the processor may be a general-purpose processor, including a CPU (Central Processing Unit), an NP (Network Processor), or the like; or a DSP (Digital Signal Processing) or an ASIC (Application) Specific Integrated Circuit, FPGA (Field-Programmable Gate Array) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components.
  • CPU Central Processing Unit
  • NP Network Processor
  • DSP Digital Signal Processing
  • ASIC Application) Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • other programmable logic device discrete gate or transistor logic device, discrete hardware components.
  • the embodiment of the present application further provides a metadata server, including a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;
  • the information processing method includes:
  • the storage server uses the stripe information of the at least one damaged strip as the first stripe information
  • the first stripe information includes: the identifier of the normal block and the normal block
  • the streaming object data to which the streaming data stored in the stream belongs the normal block corresponding to the identifier of the normal block is used as the first data block, and the first data block is performed according to the streaming object data to which the streaming data stored in the normal block belongs Grouping; constructing at least one new normal strip for each set of first data blocks;
  • the second strip information is strip information of at least one new normal stripe
  • each strip corresponds to N data blocks and M check blocks, where N and M are positive integers;
  • the method may further include:
  • the strip information of the normal strip is updated to the strip information of the damaged strip.
  • the step of transmitting the stripe information of the at least one damaged strip to the storage server includes:
  • the stripe information of the preset number of damaged strips is currently stored in the metadata server, the stripe information of all the damaged strips is sent to the storage server; or
  • the stripe information of all damaged strips currently stored is periodically sent to the storage server.
  • the method further includes:
  • the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs
  • the object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element.
  • the data server, the metadata server adds the second stripe information.
  • the normal block corresponding to the damaged strip when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so it can be stored in the normal block that is damaged again based on the information of the new normal strip.
  • the stored streaming data is repaired, so that when the normal data block corresponding to the damaged strip is damaged again, the streaming data stored in the corrupted normal data block can be repaired.
  • the communication bus mentioned by the above metadata server may be a PCI bus or an EISA bus or the like.
  • the communication bus can be divided into an address bus, a data bus, a control bus, and the like.
  • the communication interface is used for communication between the above metadata server and other devices.
  • the memory may include RAM, and may also include an NVM, such as at least one disk storage.
  • the memory may also be at least one storage device located away from the aforementioned processor.
  • the above processor may be a general-purpose processor, including a CPU, an NP, etc.; or may be a DSP, an ASIC, an FPGA, or other programmable logic device, a discrete gate or a transistor logic device, or a discrete hardware component.
  • the embodiment of the present application also provides an application program for implementing an information processing method at runtime.
  • the information processing method includes:
  • the first stripe information includes: an identifier of the normal block and the storage in the normal block Streaming object data to which streaming data belongs;
  • the normal block corresponding to the identifier of the normal block is used as the first data block, and the first data block is grouped according to the streaming object data to which the streaming data stored in the normal block belongs;
  • the second stripe information is sent to the metadata server to cause the metadata server to add the second stripe information.
  • the method may further include:
  • the identifier of the bad block is sent to the metadata server, so that the metadata server updates the stripe information of the normal stripe and the stripe information of the damaged stripe according to the identifier of the bad block.
  • the step of acquiring strip information of at least one damaged strip from the metadata server includes:
  • the first stripe information may further include: an offset of the streaming data stored in the normal block in the streaming object data;
  • the method may further include:
  • the first data block in each set of the first data block is sorted in order of the offset of the streaming data stored in the normal block in the streaming object data from small to large.
  • each strip corresponds to N data blocks and M check blocks, where N and M are positive integers;
  • the step of constructing at least one new normal strip for each set of first data blocks includes:
  • the first data block in the first data block of the group is subjected to erasure coding calculation for the streaming data stored in each of the N first data blocks in order from the front to the back.
  • the NS preset data blocks are obtained; the S first data blocks and the NS preset data blocks form N normal blocks, and the first ones are S.
  • the stream data stored in the data block and the data stored in the NS preset data blocks are subjected to erasure code calculation, and the check data of the M check blocks is obtained, and the S first data blocks correspond to the NS preset data blocks.
  • M check blocks, S first data blocks, and NS preset data blocks are constructed to construct a new normal stripe, where S is a positive integer, S ⁇ N.
  • the information processing method may further include:
  • the data in the N-S preset data blocks is refused to be stored to the storage server.
  • the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs
  • the object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element.
  • the data server, the metadata server adds the second stripe information.
  • the normal block corresponding to the damaged strip when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again.
  • the streaming data stored in it can be repaired.
  • the embodiment of the present application also provides an application program for implementing an information processing method at runtime.
  • the information processing method includes:
  • the storage server uses the stripe information of the at least one damaged strip as the first stripe information
  • the first stripe information includes: the identifier of the normal block and the normal block
  • the streaming object data to which the streaming data stored in the stream belongs the normal block corresponding to the identifier of the normal block is used as the first data block, and the first data block is performed according to the streaming object data to which the streaming data stored in the normal block belongs Grouping; constructing at least one new normal strip for each set of first data blocks;
  • the second strip information is strip information of at least one new normal stripe
  • each strip corresponds to N data blocks and M check blocks, where N and M are positive integers;
  • the method may further include:
  • the strip information of the normal strip is updated to the strip information of the damaged strip.
  • the step of transmitting the stripe information of the at least one damaged strip to the storage server includes:
  • the stripe information of the preset number of damaged strips is currently stored in the metadata server, the stripe information of all the damaged strips is sent to the storage server; or
  • the stripe information of all damaged strips currently stored is periodically sent to the storage server.
  • the method further includes:
  • the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs
  • the object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element.
  • the data server, the metadata server adds the second stripe information.
  • the normal block corresponding to the damaged strip when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again.
  • the streaming data stored in it can be repaired.
  • the embodiment of the present application further provides a readable storage medium for storing an application, and the application is implemented to implement an information processing method.
  • the information processing method includes:
  • the first stripe information includes: an identifier of the normal block and the storage in the normal block Streaming object data to which streaming data belongs;
  • the normal block corresponding to the identifier of the normal block is used as the first data block, and the flow stored in the normal block is used.
  • the second stripe information is sent to the metadata server to cause the metadata server to add the second stripe information.
  • the method may further include:
  • the identifier of the bad block is sent to the metadata server, so that the metadata server updates the stripe information of the normal stripe and the stripe information of the damaged stripe according to the identifier of the bad block.
  • the step of acquiring strip information of at least one damaged strip from the metadata server includes:
  • the first stripe information may further include: an offset of the streaming data stored in the normal block in the streaming object data;
  • the method may further include:
  • the first data block in each set of the first data block is sorted in order of the offset of the streaming data stored in the normal block in the streaming object data from small to large.
  • each strip corresponds to N data blocks and M check blocks, where N and M are positive integers;
  • the step of constructing at least one new normal strip for each set of first data blocks includes:
  • the first data block in the first data block of the group is subjected to erasure coding calculation for the streaming data stored in each of the N first data blocks in order from the front to the back.
  • the NS preset data blocks are obtained; the S first data blocks and the NS preset data blocks form N normal blocks, and the first ones are S.
  • the stream data stored in the data block and the data stored in the NS preset data blocks are subjected to erasure code calculation, and the check data of the M check blocks is obtained, and the S first data blocks correspond to the NS preset data blocks.
  • M check blocks, S first data blocks, and NS preset data blocks are constructed to construct a new normal stripe, where S is a positive integer, S ⁇ N.
  • the information processing method may further include:
  • the data in the N-S preset data blocks is refused to be stored to the storage server.
  • the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs
  • the object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element.
  • the data server, the metadata server adds the second stripe information.
  • the normal block corresponding to the damaged strip when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again.
  • the streaming data stored in it can be repaired.
  • the embodiment of the present application further provides a readable storage medium for storing an application, and the application is implemented to implement an information processing method.
  • the information processing method includes:
  • the storage server uses the stripe information of the at least one damaged strip as the first stripe information
  • the first stripe information includes: the identifier of the normal block and the normal block
  • the streaming object data to which the streaming data stored in the stream belongs the normal block corresponding to the identifier of the normal block is used as the first data block, and the first data block is performed according to the streaming object data to which the streaming data stored in the normal block belongs Grouping; building at least one new positive for each group of first data blocks Constant strip
  • the second strip information is strip information of at least one new normal stripe
  • each strip corresponds to N data blocks and M check blocks, where N and M are positive integers;
  • the method may further include:
  • the strip information of the normal strip is updated to the strip information of the damaged strip.
  • the step of transmitting the stripe information of the at least one damaged strip to the storage server includes:
  • the stripe information of the preset number of damaged strips is currently stored in the metadata server, the stripe information of all the damaged strips is sent to the storage server; or
  • the stripe information of all damaged strips currently stored is periodically sent to the storage server.
  • the method further includes:
  • the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs
  • the object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element.
  • Data server, metadata server adds second strip letter interest.
  • the normal block corresponding to the damaged strip when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again.
  • the streaming data stored in it can be repaired.
  • the storage server the metadata server, the application program, and the readable storage medium embodiment, since it is basically similar to the information processing method embodiment, the description is relatively simple, and the related information refers to the information processing method embodiment. Part of the description can be.

Abstract

A method and a device for information processing, which are applicable to a storage server. The method comprises: acquiring, from a metadata server, first stripe information of at least one damaged stripe, the first stripe information comprising: the identifiers of normal blocks, and streaming object data to which the streaming data stored in the normal blocks belongs (S201); according to the streaming object data to which the streaming data stored in the normal blocks belongs, grouping the normal blocks corresponding to the identifiers of the normal blocks (S202); constructing at least one new normal stripe for each group of data blocks (S203); acquiring second stripe information of the at least one new normal stripe (S204); and sending the second stripe information to the metadata server, so as to add the second stripe information into the metadata server (S205). In the use of the embodiments of the present application, if the normal data block corresponding to the damaged stripe is further damaged, the streaming data stored in the further damaged normal data block can be restored.

Description

一种信息处理方法及装置Information processing method and device
本申请要求于2016年12月13日提交中国专利局、申请号为201611143547.2发明名称为“一种信息处理方法及装置”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。The present application claims priority to Chinese Patent Application No. PCT Application No. No. No. No. No. No. No. No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No
技术领域Technical field
本申请涉及数据处理技术领域,特别是涉及一种信息处理方法及装置。The present application relates to the field of data processing technologies, and in particular, to an information processing method and apparatus.
背景技术Background technique
随着互联网的普及与发展,流式数据,如视频数据,在人类生活中起着越来越重要的作用。但流式数据在存储过程中,可能因为磁盘损坏、存储节点失效等原因,导致流式数据丢失。为了提高数据的可靠性和安全性,需要对流式数据进行修复。With the popularity and development of the Internet, streaming data, such as video data, plays an increasingly important role in human life. However, in the process of streaming data, streaming data may be lost due to disk corruption, storage node failure, and the like. In order to improve the reliability and security of data, it is necessary to repair the streaming data.
目前,流式数据修复一般是基于纠删码进行的,具体为,参考图1,一个条带中可以对应多个条带块,如图1中N+M个条带块,其中,一部分条带块,如图1中N个条带块作为数据块存储流式数据,另一部分条带块,如图1中M个条带块作为校验块存储校验数据。校验块中存储的校验数据是对数据块中存储的流式数据进行纠删码计算得到的,当条带对应的数据块损坏,下称为“坏块”,导致该坏块中的流式数据丢失时,可以根据校验块中的校验数据进行纠删码计算,修复该坏块中的流式数据。At present, the stream data repair is generally performed based on the erasure code. Specifically, referring to FIG. 1, a stripe may correspond to a plurality of stripe blocks, as shown in FIG. 1, N+M stripe blocks, wherein a part of the strips With a block, as shown in FIG. 1, N stripe blocks are used as data blocks to store streaming data, and another part of stripe blocks, such as M stripe blocks in FIG. 1, as check blocks to store parity data. The check data stored in the check block is obtained by performing erasure coding on the stream data stored in the data block. When the data block corresponding to the strip is damaged, it is referred to as a "bad block", resulting in the bad block. When the streaming data is lost, the erasure code calculation may be performed according to the check data in the check block, and the streaming data in the bad block is repaired.
现有技术中,基于纠删码修复条带对应的坏块中存储的流式数据,需要该条带对应的坏块的个数不大于该条带对应的校验块的个数;若该条带对应的坏块的个数大于该条带对应的校验块的个数,则再无法修复该条带对应的坏块中存储的流式数据,此时,该条带可以确定为损坏条带。这种情况下,若损坏条带对应的正常数据块,下称为“正常块”中,再出现损坏的正常块时,该再损坏的正常块中存储的流式数据也将再无法修复。In the prior art, the number of bad blocks corresponding to the stripe is not greater than the number of check blocks corresponding to the stripe, if the stream type data stored in the bad block corresponding to the stripe is repaired based on the erasure code; If the number of bad blocks corresponding to the strip is greater than the number of check blocks corresponding to the strip, the stream data stored in the bad block corresponding to the strip cannot be repaired. At this time, the strip can be determined to be damaged. Bands. In this case, if the normal data block corresponding to the strip is damaged, which is referred to as "normal block", and the damaged normal block occurs again, the streaming data stored in the re-damaged normal block will no longer be repaired.
发明内容Summary of the invention
本申请实施例的目的在于提供一种信息处理方法及装置,以使得损坏条带对应的正常数据块再损坏时,该再损坏的正常数据块中存储的流式数据能够被修复。具体技术方案如下:The purpose of the embodiment of the present application is to provide an information processing method and apparatus, so that when the normal data block corresponding to the damaged strip is damaged again, the streaming data stored in the corrupted normal data block can be repaired. The specific technical solutions are as follows:
一方面,本申请实施例公开了一种信息处理方法,应用于存储服务器, 所述方法包括:In one aspect, the embodiment of the present application discloses an information processing method, which is applied to a storage server. The method includes:
从元数据服务器中获取至少一条损坏条带的条带信息,将所述至少一条损坏条带的条带信息作为第一条带信息;所述第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;Obtaining strip information of at least one damaged strip from the metadata server, and using the strip information of the at least one damaged strip as the first strip information; the first strip information includes: normal block identifier and normal Streaming object data to which the streaming data stored in the block belongs;
将所述正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流式数据所属的流式对象数据,对所述第一数据块进行分组;The normal block corresponding to the identifier of the normal block is used as the first data block, and the first data block is grouped according to the streaming object data to which the streaming data stored in the normal block belongs;
构建针对每组第一数据块的至少一条新正常条带;Constructing at least one new normal strip for each set of first data blocks;
获取所述至少一条新正常条带的条带信息,将所述至少一条新正常条带的条带信息作为第二条带信息;Obtaining stripe information of the at least one new normal stripe, and using the stripe information of the at least one new normal stripe as the second stripe information;
将所述第二条带信息发送给所述元数据服务器,以使所述元数据服务器添加所述第二条带信息。Transmitting the second stripe information to the metadata server to cause the metadata server to add the second stripe information.
可选的,在所述从元数据服务器中获取至少一条损坏条带的条带信息的步骤之前,所述方法还包括:Optionally, before the step of acquiring the strip information of the at least one damaged strip from the metadata server, the method further includes:
检测本地是否存在坏块;Detect whether there is a bad block in the local area;
如果是,将所述坏块的标识发送给所述元数据服务器,以使所述元数据服务器根据所述坏块的标识更新正常条带的条带信息和损坏条带的条带信息。If yes, the identifier of the bad block is sent to the metadata server, so that the metadata server updates the stripe information of the normal stripe and the stripe information of the damaged strip according to the identifier of the bad block.
可选的,所述从元数据服务器中获取至少一条损坏条带的条带信息的步骤,包括:Optionally, the step of obtaining the strip information of the at least one damaged strip from the metadata server includes:
接收所述元数据服务器发送的当前存储的预设数量个损坏条带的条带信息;或,Receiving stripe information of a preset number of damaged strips currently stored by the metadata server; or
接收所述元数据服务器定时发送的当前存储的所有损坏条带的条带信息。Receiving stripe information of all currently stored corrupted strips periodically sent by the metadata server.
可选的,所述第一条带信息还包括:正常块中存储的流式数据在流式对象数据中偏移量;Optionally, the first strip information further includes: an offset of the streaming data stored in the normal block in the streaming object data;
在所述按照正常块中存储的流式数据所属的流式对象数据,对所述第一数据块进行分组的步骤之后,所述方法还包括:After the step of grouping the first data block according to the streaming object data to which the streaming data stored in the normal block belongs, the method further includes:
按照正常块中存储的流式数据在流式对象数据中偏移量从小到大的顺序, 对每组第一数据块中的第一数据块进行排序。According to the flow data stored in the normal block, the offsets in the streaming object data are in descending order. The first data block in each set of first data blocks is sorted.
可选的,每个条带对应N个数据块和M个校验块,其中,N和M为正整数;Optionally, each stripe corresponds to N data blocks and M check blocks, where N and M are positive integers;
所述构建针对每组第一数据块的至少一条新正常条带的步骤,包括:The step of constructing at least one new normal strip for each set of first data blocks includes:
对于每组第一数据块,对该组第一数据块中排序后的第一数据块,从前至后,依次对每N个第一数据块中存储的流式数据进行纠删码计算,获得M个校验块的检验数据;由每N个第一数据块和该N个第一数据块对应的M个校验块,构建新正常条带;For each first data block of the group, the first data block in the first data block of the group is subjected to erasure coding calculation for the streaming data stored in each of the N first data blocks in order from the front to the back. Check data of M check blocks; constructing a new normal strip by each N first data blocks and M check blocks corresponding to the N first data blocks;
当该组第一数据块中剩余S个第一数据块时,获取N-S个预设数据块;由所述S个第一数据块和所述N-S个预设数据块组成N个正常块,对所述S个第一数据块中存储的流式数据和所述N-S个预设数据块中存储的数据进行纠删码计算,获得M个校验块的检验数据,由所述S个第一数据块与所述N-S个预设数据块对应的M个校验块、所述S个第一数据块和所述N-S个预设数据块,构建新正常条带,其中,S为正整数,S<N。Obtaining NS preset data blocks when the first first data blocks remain in the first data block of the group; and forming the N normal blocks by the S first data blocks and the NS preset data blocks, And performing the erasure code calculation on the stream data stored in the S first data blocks and the data stored in the NS preset data blocks, and obtaining the verification data of the M check blocks, where the S first Constructing a new normal stripe, wherein S is a positive integer, the data block and the M check blocks corresponding to the NS preset data blocks, the S first data blocks, and the NS preset data blocks. S<N.
可选的,所述方法还包括:Optionally, the method further includes:
拒绝将所述N-S个预设数据块中的数据存储至所述存储服务器。Refusing to store data in the N-S preset data blocks to the storage server.
二方面,本申请实施例还公开了一种信息处理方法,应用于元数据服务器,所述方法包括:On the second aspect, the embodiment of the present application further discloses an information processing method, which is applied to a metadata server, and the method includes:
将至少一条损坏条带的条带信息发送给存储服务器,以使所述存储服务器将所述至少一条损坏条带的条带信息作为第一条带信息,所述第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;将所述正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流式数据所属的流式对象数据,对所述第一数据块进行分组;构建针对每组第一数据块的至少一条新正常条带;Transmitting the strip information of the at least one damaged strip to the storage server, so that the storage server uses the strip information of the at least one damaged strip as the first strip information, where the first strip information includes: The identifier of the block and the streaming object data to which the streaming data stored in the normal block belongs; the normal block corresponding to the identifier of the normal block is used as the first data block, and the streaming object belongs to the streaming data stored in the normal block Data, grouping the first data blocks; constructing at least one new normal strip for each set of first data blocks;
接收所述存储服务器发送的第二条带信息;所述第二条带信息为所述至少一条新正常条带的条带信息;Receiving, by the storage server, second strip information; the second strip information is stripe information of the at least one new normal stripe;
存储所述第二条带信息。The second strip information is stored.
可选的,每个条带对应N个数据块和M个校验块,其中,N和M为正整数; Optionally, each stripe corresponds to N data blocks and M check blocks, where N and M are positive integers;
在所述将至少一条损坏条带的条带信息发送给存储服务器的步骤之前,所述方法还包括:Before the step of transmitting the stripe information of the at least one damaged strip to the storage server, the method further includes:
接收所述存储服务器发送的坏块的标识;Receiving an identifier of a bad block sent by the storage server;
确定所述坏块的标识对应的正常条带的条带信息;Determining strip information of a normal strip corresponding to the identifier of the bad block;
将所述正常条带的条带信息中所述坏块的标识对应的正常块标记为坏块;Marking, in the strip information of the normal stripe, a normal block corresponding to the identifier of the bad block as a bad block;
当所述正常条带的条带信息中标记为坏块的正常块的个数大于M时,将所述正常条带的条带信息更新为损坏条带的条带信息。When the number of normal blocks marked as bad blocks in the strip information of the normal strip is greater than M, the strip information of the normal strip is updated to the strip information of the damaged strip.
可选的,所述将至少一条损坏条带的条带信息发送给存储服务器的步骤,包括:Optionally, the step of sending the stripe information of the at least one damaged strip to the storage server includes:
当所述元数据服务器中当前存储了预设数量个损坏条带的条带信息时,将所有所述损坏条带的条带信息发送给所述存储服务器;或,When the stripe information of the preset number of damaged strips is currently stored in the metadata server, the stripe information of all the damaged strips is sent to the storage server; or
定时将当前存储的所有损坏条带的条带信息发送给所述存储服务器。Timing information of all damaged strips currently stored is periodically sent to the storage server.
可选的,在所述存储所述第二条带信息的步骤之后,所述方法还包括:Optionally, after the step of storing the second strip information, the method further includes:
删除所述第一条带信息。Delete the first strip information.
三方面,本申请实施例还公开了一种信息处理装置,应用于存储服务器,所述装置包括:In one aspect, the embodiment of the present application further discloses an information processing apparatus, which is applied to a storage server, where the apparatus includes:
第一信息获取单元,用于从元数据服务器中获取至少一条损坏条带的条带信息,将所述至少一条损坏条带的条带信息作为第一条带信息;所述第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;a first information acquiring unit, configured to acquire strip information of at least one damaged strip from the metadata server, and use the strip information of the at least one damaged strip as the first strip information; the first strip information The method includes: an identifier of a normal block and streaming object data to which the streaming data stored in the normal block belongs;
块分组单元,用于将所述正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流式数据所属的流式对象数据,对所述第一数据块进行分组;a block grouping unit, configured to use the normal block corresponding to the identifier of the normal block as the first data block, and group the first data block according to the streaming object data to which the streaming data stored in the normal block belongs;
条带构建单元,用于构建针对每组第一数据块的至少一条新正常条带;a strip building unit for constructing at least one new normal strip for each set of first data blocks;
第二信息获取单元,用于获取所述至少一条新正常条带的条带信息,将所述至少一条新正常条带的条带信息作为第二条带信息; a second information acquiring unit, configured to acquire stripe information of the at least one new normal stripe, and use the stripe information of the at least one new normal stripe as the second stripe information;
信息发送单元,用于将所述第二条带信息发送给所述元数据服务器,以使所述元数据服务器添加所述第二条带信息。And an information sending unit, configured to send the second stripe information to the metadata server, so that the metadata server adds the second stripe information.
可选的,所述装置还包括:Optionally, the device further includes:
块检测单元,用于在从元数据服务器中获取至少一条损坏条带的条带信息之前,检测本地是否存在坏块;a block detecting unit, configured to detect whether a local bad block exists before acquiring the stripe information of the at least one damaged stripe from the metadata server;
标识发送单元,用于在所述块检测单元的检测结果为是的情况下,将所述坏块的标识发送给所述元数据服务器,以使所述元数据服务器根据所述坏块的标识更新正常条带的条带信息和损坏条带的条带信息。An identifier sending unit, configured to send, by the block detection unit, an identifier of the bad block to the metadata server, so that the metadata server is configured according to the identifier of the bad block Update the strip information of the normal strip and the strip information of the damaged strip.
可选的,所述第一信息获取单元,具体用于:Optionally, the first information acquiring unit is specifically configured to:
接收所述元数据服务器发送的当前存储的预设数量个损坏条带的条带信息;或,Receiving stripe information of a preset number of damaged strips currently stored by the metadata server; or
接收所述元数据服务器定时发送的当前存储的所有损坏条带的条带信息。Receiving stripe information of all currently stored corrupted strips periodically sent by the metadata server.
可选的,所述第一条带信息还包括:正常块中存储的流式数据在流式对象数据中偏移量;Optionally, the first strip information further includes: an offset of the streaming data stored in the normal block in the streaming object data;
所述装置还包括:The device also includes:
块排序单元,用于在按照正常块中存储的流式数据所属的流式对象数据,对所述第一数据块进行分组之后,按照正常块中存储的流式数据在流式对象数据中偏移量从小到大的顺序,对每组第一数据块中的第一数据块进行排序。a block sorting unit, configured to: after the first data block is grouped according to the streaming object data to which the streaming data stored in the normal block belongs, according to the streaming data stored in the normal block, in the streaming object data The order of the shifts from small to large, the first data block in each set of the first data block is sorted.
可选的,每个条带对应N个数据块和M个校验块,其中,N和M为正整数;Optionally, each stripe corresponds to N data blocks and M check blocks, where N and M are positive integers;
所述条带构建单元,具体用于:The strip building unit is specifically configured to:
对于每组第一数据块,对该组第一数据块中排序后的第一数据块,从前至后,依次对每N个第一数据块中存储的流式数据进行纠删码计算,获得M个校验块的检验数据;由每N个第一数据块和该N个第一数据块对应的M个校验块,构建新正常条带;For each first data block of the group, the first data block in the first data block of the group is subjected to erasure coding calculation for the streaming data stored in each of the N first data blocks in order from the front to the back. Check data of M check blocks; constructing a new normal strip by each N first data blocks and M check blocks corresponding to the N first data blocks;
当该组第一数据块中剩余S个第一数据块时,获取N-S个预设数据块;由所述S个第一数据块和所述N-S个预设数据块组成N个正常块;对所述S个第一 数据块中存储的流式数据和所述N-S个预设数据块中存储的数据进行纠删码计算,获得M个校验块的检验数据,由所述S个第一数据块与所述N-S个预设数据块对应的M个校验块、所述S个第一数据块和所述N-S个预设数据块,构建新正常条带,其中,S为正整数,S<N。Obtaining NS preset data blocks when the first first data blocks remain in the first data block of the group; and forming N normal blocks by the S first data blocks and the NS preset data blocks; The S first And performing the erasure code calculation on the stream data stored in the data block and the data stored in the NS preset data blocks, and obtaining the check data of the M check blocks, where the S first data blocks and the NS are The M parity blocks corresponding to the preset data blocks, the S first data blocks, and the NS preset data blocks construct a new normal stripe, where S is a positive integer, S<N.
可选的,所述装置还包括:Optionally, the device further includes:
存储拒绝单元,用于拒绝将所述N-S个预设数据块中的数据存储至所述存储服务器。And a storage rejecting unit, configured to refuse to store data in the N-S preset data blocks to the storage server.
四方面,本申请实施例还公开了一种信息处理装置,应用于元数据服务器,所述装置包括:In an aspect, the embodiment of the present application further discloses an information processing apparatus, which is applied to a metadata server, where the apparatus includes:
信息发送单元,用于将至少一条损坏条带的条带信息发送给存储服务器,以使所述存储服务器将所述至少一条损坏条带的条带信息作为第一条带信息,所述第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;将所述正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流式数据所属的流式对象数据,对所述第一数据块进行分组;构建针对每组第一数据块的至少一条新正常条带;An information sending unit, configured to send the strip information of the at least one damaged strip to the storage server, so that the storage server uses the strip information of the at least one damaged strip as the first strip information, the first The stripe information includes: an identifier of a normal block and streaming object data to which the streaming data stored in the normal block belongs; a normal block corresponding to the identifier of the normal block is used as the first data block, and the flow is stored in the normal block. Streaming object data to which the data belongs, grouping the first data block; constructing at least one new normal strip for each set of the first data block;
信息接收单元,用于接收所述存储服务器发送的第二条带信息;所述第二条带信息为所述至少一条新正常条带的条带信息;An information receiving unit, configured to receive second strip information sent by the storage server; the second strip information is stripe information of the at least one new normal stripe;
信息存储单元,用于存储所述第二条带信息。An information storage unit, configured to store the second strip information.
可选的,每个条带对应N个数据块和M个校验块,其中,N和M为正整数;Optionally, each stripe corresponds to N data blocks and M check blocks, where N and M are positive integers;
所述装置还包括:The device also includes:
标识接收单元,用于在将至少一条损坏条带的条带信息发送给存储服务器之前,接收所述存储服务器发送的坏块的标识;An identifier receiving unit, configured to receive an identifier of a bad block sent by the storage server before sending the stripe information of the at least one damaged strip to the storage server;
信息确定单元,用于确定所述坏块的标识对应的正常条带的条带信息;An information determining unit, configured to determine strip information of a normal stripe corresponding to the identifier of the bad block;
块标记单元,用于将所述正常条带的条带信息中所述坏块的标识对应的正常块标记为坏块;a block marking unit, configured to mark a normal block corresponding to the identifier of the bad block in the strip information of the normal stripe as a bad block;
信息更新单元,用于当所述正常条带的条带信息中标记为坏块的正常块 的个数大于M时,将所述正常条带的条带信息更新为损坏条带的条带信息。An information updating unit, configured to: mark a normal block marked as a bad block in the strip information of the normal stripe When the number of the numbers is greater than M, the strip information of the normal strip is updated to the strip information of the damaged strip.
可选的,所述信息发送单元,具体用于:Optionally, the information sending unit is specifically configured to:
当所述元数据服务器中当前存储了预设数量个损坏条带的条带信息时,将所有所述损坏条带的条带信息发送给所述存储服务器;或,When the stripe information of the preset number of damaged strips is currently stored in the metadata server, the stripe information of all the damaged strips is sent to the storage server; or
定时将当前存储的所有损坏条带的条带信息发送给所述存储服务器。Timing information of all damaged strips currently stored is periodically sent to the storage server.
可选的,所述装置还包括:Optionally, the device further includes:
信息删除单元,用于在存储所述第二条带信息之后,删除所述第一条带信息。And an information deleting unit, configured to delete the first strip information after storing the second strip information.
五方面,本申请实施例公开一种存储服务器,包括处理器、通信接口、存储器和通信总线;其中,所述处理器、所述通信接口、所述存储器通过所述通信总线完成相互间的通信;In five aspects, the embodiment of the present application discloses a storage server, including a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus. ;
所述存储器,用于存放应用程序;The memory is configured to store an application;
所述处理器,用于执行所述存储器上所存放的应用程序时,实现上述应用于存储服务器的信息处理方法步骤。The processor, when used to execute an application stored on the memory, implements the above-mentioned information processing method steps applied to the storage server.
六方面,本申请实施例公开了一种元数据服务器,包括处理器、通信接口、存储器和通信总线;其中,所述处理器、所述通信接口、所述存储器通过所述通信总线完成相互间的通信;In one aspect, the embodiment of the present application discloses a metadata server, including a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory complete each other through the communication bus. Communication;
所述存储器,用于存放应用程序;The memory is configured to store an application;
所述处理器,用于执行所述存储器上所存放的应用程序时,实现上述应用于元数据服务器的信息处理方法步骤。The processor is configured to implement the information processing method step applied to the metadata server when the application stored on the memory is executed.
七方面,本申请实施例公开了一种应用程序,所述应用程序用于在运行时实现上述应用于存储服务器的信息处理方法步骤。。In the seventh aspect, the embodiment of the present application discloses an application program, which is used to implement the above-mentioned information processing method step applied to the storage server at runtime. .
八方面,本申请实施例公开了一种应用程序,所述应用程序用于在运行时实现上述应用于元数据服务器的信息处理方法步骤。In one aspect, the embodiment of the present application discloses an application program for implementing the above-mentioned information processing method step applied to the metadata server at runtime.
九方面,本申请实施例公开了一种可读存储介质,所述可读存储介质用 于存储应用程序,所述应用程序被运行时实现上述应用于存储服务器的信息处理方法步骤。In nine aspects, the embodiment of the present application discloses a readable storage medium, and the readable storage medium is used. In the storage application, the application is implemented to implement the above-described information processing method steps applied to the storage server.
十方面,本申请实施例公开了一种可读存储介质,所述可读存储介质用于存储应用程序,所述应用程序被运行时实现上述应用于元数据服务器的信息处理方法步骤。In ten aspects, the embodiment of the present application discloses a readable storage medium for storing an application, and the application is implemented to implement the above-mentioned information processing method step applied to the metadata server.
本申请实施例提供了一种信息处理方法及装置,存储服务器从元数据服务器中获取至少一条损坏条带的第一条带信息,该第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据,按照正常块中存储的流式数据所属的流式对象数据,对正常块的标识对应的正常块进行分组,进而构建至少一条新正常条带,并将第二条带信息发送给元数据服务器,元数据服务器添加第二条带信息。可见,本申请实施例中,当损坏条带对应的正常块再损坏时,因该损坏条带对应的正常块再次重新构建了新正常条带,而该新正常条带对应的数据块都为正常块,因此可以根据该新正常条带的信息,对再损坏的正常块中存储的流式数据进行修复,进而使得损坏条带对应的正常数据块再损坏时,该再损坏的正常数据块中存储的流式数据能够被修复。当然,实施本申请的任一产品或方法必不一定需要同时达到以上所述的所有优点。An embodiment of the present application provides an information processing method and apparatus, where a storage server obtains first strip information of at least one damaged strip from a metadata server, where the first strip information includes: an identifier of a normal block and a normal block. The streamed object data to which the stored stream data belongs is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and The second stripe information is sent to the metadata server, and the metadata server adds the second stripe information. It can be seen that, in the embodiment of the present application, when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again. The streaming data stored in it can be repaired. Of course, implementing any of the products or methods of the present application necessarily does not necessarily require all of the advantages described above to be achieved at the same time.
附图说明DRAWINGS
为了更清楚地说明本申请实施例或相关技术中的技术方案,下面将对实施例或相关技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings to be used in the embodiments or the related art description will be briefly described below. Obviously, the drawings in the following description are only Some embodiments of the application may also be used to obtain other figures from those of ordinary skill in the art without departing from the scope of the invention.
图1为条带结构示意图;Figure 1 is a schematic view of a strip structure;
图2为本申请实施例提供的一种信息处理方法的一种流程示意图;2 is a schematic flowchart of an information processing method according to an embodiment of the present application;
图3为本申请实施例提供的一种信息处理方法的另一种流程示意图;FIG. 3 is another schematic flowchart of an information processing method according to an embodiment of the present disclosure;
图4为本申请实施例中获取的损坏条带的示意图;4 is a schematic diagram of a damaged strip obtained in an embodiment of the present application;
图5为本申请实施例中重新构建的新正常条带的示意图;FIG. 5 is a schematic diagram of a new normal strip reconstructed in the embodiment of the present application; FIG.
图6为本申请实施例提供的另一种信息处理方法的一种流程示意图; FIG. 6 is a schematic flowchart diagram of another information processing method according to an embodiment of the present disclosure;
图7为本申请实施例提供的另一种信息处理方法的另一种流程示意图;FIG. 7 is another schematic flowchart of another information processing method according to an embodiment of the present disclosure;
图8为本申请实施例提供的一种信息处理装置的一种结构示意图;FIG. 8 is a schematic structural diagram of an information processing apparatus according to an embodiment of the present disclosure;
图9为本申请实施例提供的一种信息处理装置的另一种结构示意图;FIG. 9 is another schematic structural diagram of an information processing apparatus according to an embodiment of the present disclosure;
图10为本申请实施例提供的另一种信息处理装置的一种结构示意图;FIG. 10 is a schematic structural diagram of another information processing apparatus according to an embodiment of the present disclosure;
图11为本申请实施例提供的另一种信息处理装置的另一种结构示意图。FIG. 11 is another schematic structural diagram of another information processing apparatus according to an embodiment of the present application.
具体实施方式detailed description
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the drawings in the embodiments of the present application. It is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.
为便于说明本申请,下面对本申请实施例中出现的词进行定义说明:For convenience of description of the present application, the definitions of the words appearing in the embodiments of the present application are described below:
条带块可以作为数据块存储流式数据,也可以作为校验块存储校验数据;The stripe block can store streaming data as a data block, or can store parity data as a check block;
正常块为正常数据块,存储服务器可以正常地从正常块中读取流式数据;The normal block is a normal data block, and the storage server can normally read the streaming data from the normal block;
坏块为损坏数据块,也就是损坏的正常块,存储服务器无法从坏块中读取到流式数据;The bad block is a corrupted data block, that is, a damaged normal block, and the storage server cannot read the streaming data from the bad block;
其中,正常块和坏块统称为数据块。Among them, normal blocks and bad blocks are collectively referred to as data blocks.
下面通过具体实施例,对本申请实施例进行详细说明。The embodiments of the present application are described in detail below through specific embodiments.
参考图2,图2为本申请实施例提供的一种信息处理方法的一种流程示意图,应用于存储服务器,该方法包括:Referring to FIG. 2, FIG. 2 is a schematic flowchart of an information processing method according to an embodiment of the present disclosure, which is applied to a storage server, and the method includes:
S201:从元数据服务器中获取至少一条损坏条带的条带信息,将该至少一条损坏条带的条带信息作为第一条带信息;S201: Obtain at least one stripe information of the damaged strip from the metadata server, and use the stripe information of the at least one damaged strip as the first stripe information;
其中,第一条带信息可以包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据。这里,流式对象数据可以为来自同一终端的所有流式数据,也可以为来自同一时间段的所有流式数据,也可以为用户划分出的一些流式数据,本申请实施例对此不进行限定。流式对象数据中的流式数据丢失部分,用户仍然可以读取该流式对象数据中的其他流式数据。 The first strip information may include: an identifier of the normal block and streaming object data to which the streaming data stored in the normal block belongs. Here, the streaming object data may be all the streaming data from the same terminal, or all the streaming data from the same time period, or some streaming data divided by the user, which is not performed by the embodiment of the present application. limited. In the streaming data loss part of the streaming object data, the user can still read other streaming data in the streaming object data.
在本申请的一个实施例中,存储服务器获取的损坏条带的条带信息可以为:元数据服务器中当前存储的所有损坏条带的条带信息。一般的,元数据服务器可以定时地将当前存储的所有损坏条带的条带信息发送给存储服务器,也可以在该元数据服务器中存储了预设数量个损坏条带的条带信息时,将当前存储的这预设数量个损坏条带的条带信息都发送给存储服务器。In an embodiment of the present application, the stripe information of the damaged stripe acquired by the storage server may be: stripe information of all damaged strips currently stored in the metadata server. Generally, the metadata server may periodically send the stripe information of all the damaged strips currently stored to the storage server, or when the stripe information of the preset number of damaged strips is stored in the metadata server, The stripe information of the preset number of damaged strips currently stored is sent to the storage server.
在本申请的一个实施例中,参考图3,在图2的基础上,在从元数据服务器中获取至少一条损坏条带的条带信息之前,上述信息处理方法还可以包括:In an embodiment of the present application, referring to FIG. 3, on the basis of FIG. 2, before the stripe information of the at least one damaged stripe is obtained from the metadata server, the information processing method may further include:
S301:检测本地是否存在坏块;如果是,S302;S301: Detect whether there is a bad block locally; if yes, S302;
一般的,存储服务器可以定时地检测本地是否存在坏块。在本申请的一个实施例中,若读取一个正常块中的流式数据失败,则再次读取该正常块中的流式数据,若还是失败,则可以将该正常块确定为坏块。In general, the storage server can periodically detect whether there is a bad block locally. In an embodiment of the present application, if the streaming data in a normal block fails to be read, the streaming data in the normal block is read again, and if it still fails, the normal block can be determined as a bad block.
S302:将坏块的标识发送给元数据服务器。S302: Send the identifier of the bad block to the metadata server.
元数据服务器可以存储条带的元数据,该条带的元数据即为条带信息,可以包括:硬盘的标识、数据块在硬盘中的偏移量、数据块的标识、数据块中存储的流式数据所属的流式对象数据、数据块中存储的流式数据在流式对象数据中的偏移量、数据块的存储状态等信息,如表1所示。The metadata server may store the metadata of the stripe, and the metadata of the stripe is stripe information, which may include: an identifier of the hard disk, an offset of the data block in the hard disk, an identifier of the data block, and a storage in the data block. The flow object data to which the streaming data belongs, the offset of the streaming data stored in the data block in the streaming object data, and the storage state of the data block are as shown in Table 1.
表1Table 1
Figure PCTCN2017101221-appb-000001
Figure PCTCN2017101221-appb-000001
条带块分为数据块和校验块,上述表1中,条带块可以理解为数据块,条 带块集合为一个条带对应的数据块的集合;The stripe block is divided into a data block and a check block. In the above table 1, the strip block can be understood as a data block and a strip. A set of strips is a set of data blocks corresponding to one stripe;
其中,硬盘的标识wwn(World Wide Name,全球唯一名字),是由权威的组织分配的唯一的48位或64位数字,专门制定为公认的名称授权,通常通过区域分配给制造商,以区分一个或一组网络连接;为了减少出现损坏条带的概率,一个条带对应的每一条带块对应一个硬盘,例如一个条带对应4个条带块,则该条带对应4个不同的硬盘;The hard disk identifier wwn (World Wide Name) is a unique 48-bit or 64-bit number assigned by an authoritative organization. It is specifically defined as a recognized name authorization, usually assigned to the manufacturer by region to distinguish One or a group of network connections; in order to reduce the probability of occurrence of damaged strips, each strip corresponding to one strip corresponds to one hard disk, for example, one strip corresponds to four strip blocks, and the strip corresponds to four different hard disks. ;
条带块在流式对象数据中偏移量可以理解为数据块中存储的流式数据在流式对象数据中偏移量;通过该偏移量,结合条带块,这里可以理解为数据块在硬盘中的偏移量,可以准确地读取到数据块中存储的流式数据;The offset of the sliced block in the streaming object data can be understood as the offset of the streaming data stored in the data block in the streaming object data; by the offset, combined with the stripe block, it can be understood as a data block. The offset in the hard disk can accurately read the streaming data stored in the data block;
条带块对应的流式对象数据可以理解为数据块中存储的流式数据所属的流式对象数据。一般的,元数据服务器中还可以存储流式对象数据的元数据,该流式对象数据的元数据可以包括:存储流式数据的数据块对应的条带的标识和流式对象数据的大小等信息,如表2所示。The stream object data corresponding to the stripe block can be understood as the stream object data to which the stream data stored in the data block belongs. Generally, the metadata of the streaming object data may be stored in the metadata server, where the metadata of the streaming object data may include: an identifier of a strip corresponding to the data block storing the streaming data, and a size of the streaming object data. Information, as shown in Table 2.
表2Table 2
字段Field Key(键)Key (key) 备注Remarks
object_keyObject_key Primary Key(主键)Primary Key 流式对象数据Streaming object data
strip_id集合Strip_id collection   条带的标识的集合Collection of stripe logos
object_sizeObject_size   流式对象数据的大小Streaming object data size
其中,条带标识的集合为:存储流式数据的数据块对应的条带的标识的集合。The set of stripe identifiers is a set of identifiers of strips corresponding to the data blocks storing the stream data.
条带块状态可以理解为数据块的存储状态,这里,数据块的存储状态,可以包括正常状态和损坏状态。具体地,当数据块中存储了流式数据时,该数据块的存储状态设置为正常状态,也就是,将该数据块标记为正常块;当数据块出现损坏,导致该数据块中存储的流式数据丢失时,该数据块的存储状态设置为损坏状态,也就是,将该正常块标记为坏块。The status of the stripe block can be understood as the storage state of the data block. Here, the storage state of the data block may include a normal state and a damaged state. Specifically, when streaming data is stored in the data block, the storage state of the data block is set to a normal state, that is, the data block is marked as a normal block; when the data block is damaged, the data block is stored in the data block. When streaming data is lost, the storage state of the data block is set to a corrupted state, that is, the normal block is marked as a bad block.
当元数据服务器接收到存储服务器发送的坏块的标识后,确定坏块的标识对应的正常条带的条带信息,将该正常条带的条带信息中坏块的标识对应的正常块标记为坏块,也就是,将该坏块的标识对应的正常块的“条带块状 态”设置为损坏状态;若该正常条带中的被标记为坏块的正常块的个数大于该正常条带中校验块的个数,则将该正常条带的条带信息更新为损坏条带的条带信息。After receiving the identifier of the bad block sent by the storage server, the metadata server determines the stripe information of the normal stripe corresponding to the identifier of the bad block, and marks the normal block corresponding to the identifier of the bad block in the stripe information of the normal stripe. a bad block, that is, a "strip block" of the normal block corresponding to the identification of the bad block The state is set to a damaged state; if the number of normal blocks marked as bad blocks in the normal strip is greater than the number of check blocks in the normal strip, the strip information of the normal strip is updated to Strip information for damaged strips.
S202:将第一条带信息中包括的正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流式数据所属的流式对象数据,对第一数据块进行分组;S202: The normal block corresponding to the identifier of the normal block included in the first stripe information is used as the first data block, and the first data block is grouped according to the streaming object data to which the streaming data stored in the normal block belongs.
例如,如图4所示,获取了4条损坏条带的条带信息,分别为条带1的条带信息、条带2的条带信息、条带3的条带信息和条带4的条带信息,其中,4条条带中的数据块1、数据块3、数据块6、数据块8、数据块9、数据块10、数据块13和数据块14为正常块,即存在8个第一数据块,数据块1中存储了流式数据1、数据块3中存储了流式数据3、数据块6中存储了流式数据6、数据块8中存储了流式数据8、数据块9中存储了流式数据9、数据块10中存储了流式数据10、数据块13中存储了流式数据13、数据块14中存储了流式数据14,这里,流式数据1、流式数据3、流式数据6、流式数据8、流式数据9和流式数据10属于流式对象数据Object1,流式数据13和流式数据14属于流式对象数据Object2;这种情况下,对第一数据块进行分组时,可以将数据块1、数据块3、数据块6、数据块8、数据块9和数据块10分为一组,将数据块13和数据块14分为一组。For example, as shown in FIG. 4, strip information of four damaged strips is acquired, which are strip information of the strip 1, strip information of the strip 2, strip information of the strip 3, and strip 4 Stripe information, wherein the data block 1, the data block 3, the data block 6, the data block 8, the data block 9, the data block 10, the data block 13, and the data block 14 of the four strips are normal blocks, that is, there are 8 The first data block, the streaming data 1 in the data block 1, the streaming data 3 in the data block 3, the streaming data 6 in the data block 6, and the streaming data 8 in the data block 8, The stream data 9 is stored in the data block 9, the stream data 10 is stored in the data block 10, the stream data 13 is stored in the data block 13, and the stream data 14 is stored in the data block 14, where the stream data 1 is stored. The streaming data 3, the streaming data 6, the streaming data 8, the streaming data 9 and the streaming data 10 belong to the streaming object data Object1, and the streaming data 13 and the streaming data 14 belong to the streaming object data Object2; In the case, when the first data block is grouped, the data block 1, the data block 3, the data block 6, the data block 8, the data block 9, and the data may be used. 10 grouped into a group, the data blocks 13 and 14 is divided into a set of data blocks.
S203:构建针对每组第一数据块的至少一条新正常条带;S203: Construct at least one new normal strip for each set of the first data block;
在本申请的一个实施例中,为了便于数据的读取,在按照正常块中存储的流式数据所属的流式对象数据,对多个第一数据块进行分组之后,可以按照数据块中存储的流式数据在流式对象数据中偏移量从小到大的顺序,对每组中的第一数据块进行排序,根据排序后的第一数据块构建新正常条带。这里,数据块中存储的流式数据在流式对象数据中偏移量可以包括在获取的损坏条带的条带信息中。In an embodiment of the present application, in order to facilitate data reading, after the plurality of first data blocks are grouped according to the streaming object data to which the streaming data stored in the normal block belongs, the data block may be stored in the data block. The streamed data is sorted in the order of offset from the streamed object data, and the first block of data in each group is sorted, and a new normal strip is constructed according to the sorted first block. Here, the offset of the streaming data stored in the data block in the streaming object data may be included in the stripe information of the acquired corrupted stripe.
对于每组第一数据块,对该组第一数据块中排序后,从前至后,依次对每N个第一数据块中存储的流式数据进行纠删码计算,获得M个校验块的检验数据;由每N个第一数据块和该N个第一数据块对应的M个校验块,构建新正常条带; For each group of first data blocks, after sorting in the first data block of the group, performing erasure coding calculation on the streaming data stored in each of the N first data blocks in order from the front to the back, and obtaining M check blocks. Check data; construct a new normal strip by each N first data blocks and M check blocks corresponding to the N first data blocks;
当该组第一数据块中剩余S个第一数据块时,则获取N-S个预设数据块;由这S个第一数据块和这N-S个预设数据块组成N个正常块;对这S个第一数据块中存储的流式数据和这N-S个预设数据块中存储的数据进行纠删码计算,获得M个校验块的检验数据,由这S个第一数据块与这N-S个预设数据块对应的M个校验块、这S个第一数据块和这N-S个预设数据块,构建新正常条带。需要说明的是,S为正整数,S<N,预设数据块可以为空的数据块,也就是,未存储任何数据的数据块,也可以为全0或其他已知数据的数据块,本申请实施例对此不进行限定。When S first data blocks remain in the first data block of the group, acquiring NS preset data blocks; and forming the N normal blocks by the S first data blocks and the NS preset data blocks; The stream data stored in the S first data blocks and the data stored in the NS preset data blocks are subjected to erasure code calculation, and the check data of the M check blocks is obtained, and the S first data blocks are used. The M parity blocks corresponding to the NS preset data blocks, the S first data blocks, and the NS preset data blocks construct a new normal stripe. It should be noted that S is a positive integer, S<N, and the preset data block can be an empty data block, that is, a data block that does not store any data, and can also be a data block of all 0s or other known data. This embodiment of the present application does not limit this.
仍以S202中的例子进行说明,N=4,对于Object1组第一数据块,各个第一数据块中存储的流式数据在流式对象数据Object1中偏移量从小到大的顺序为:流式数据1-流式数据3-流式数据6-流式数据8-流式数据9-流式数据10,则第一数据块的排序为:数据块1-数据块3-数据块6-数据块8-数据块9-数据块10,若一个条带对应4个数据块和1个校验块,则从前至后,依次对每4个第一数据块中存储的流式数据进行纠删码计算,也就是,先获取数据块1、数据块3、数据块6、数据块8这4个正常块,对这4个正常块中存储的流式数据进行纠删码计算,获得1个检验数据1,将这1个校验数据1存储在校验块1中,如图5所示,由数据块1、数据块3、数据块6、数据块8和校验块1构成一个新正常条带5;此时,Object1组第一数据块只剩下数据块9和数据块10这2个正常块,即S=2,此时,2<4,则获取N-S=4-2=2个预设数据块,由数据块9、数据块10和这2个预设数据块,构成4个正常块,对这4个数据块中存储的数据进行纠删码计算,获得1个检验数据2,将这1个校验数据2存储在校验块2中,如图5所示,由数据块9、数据块10、2个预设数据块和校验块2构成一个新正常条带6。Still speaking, the example in S202 is described, N=4. For the first data block of the Object1 group, the order of the offset of the streaming data stored in each first data block from the small to the largest in the streaming object data Object1 is: Data 1 - Streaming Data 3 - Streaming Data 6 - Streaming Data 8 - Streaming Data 9 - Streaming Data 10, the ordering of the first data block is: Data Block 1 - Data Block 3 - Data Block 6 - Data block 8 - data block 9 - data block 10, if one strip corresponds to 4 data blocks and 1 check block, the flow data stored in every 4 first data blocks is corrected in order from front to back. The code deletion calculation, that is, the first four normal blocks of the data block 1, the data block 3, the data block 6, and the data block 8, are first obtained, and the erasure code calculation is performed on the stream data stored in the four normal blocks to obtain 1 One test data 1, the one check data 1 is stored in the check block 1, as shown in FIG. 5, and one block is composed of the data block 1, the data block 3, the data block 6, the data block 8, and the check block 1. The new normal strip 5; at this time, the first data block of the Object1 group only has two normal blocks of the data block 9 and the data block 10, that is, S=2. At this time, 2<4, the NS=4 is obtained. -2=2 preset data blocks, which are composed of data block 9, data block 10 and the two preset data blocks, and form 4 normal blocks, and perform erasure code calculation on the data stored in the four data blocks to obtain 1 test data 2, the 1 check data 2 is stored in the check block 2, as shown in FIG. 5, the data block 9, the data block 10, the two preset data blocks and the check block 2 constitute one New normal strip 6.
在本申请的一个实施例中,为了节约存储服务器的存储空间,拒绝将N-S个预设数据块中的数据存储至存储服务器。In an embodiment of the present application, in order to save the storage space of the storage server, the data in the N-S preset data blocks is refused to be stored to the storage server.
S204:获取至少一条新正常条带的条带信息,将至少一条新正常条带的条带信息作为第二条带信息;S204: Obtain stripe information of at least one new normal stripe, and use stripe information of at least one new normal stripe as the second stripe information.
S205:将第二条带信息发送给元数据服务器。S205: Send the second stripe information to the metadata server.
元数据服务器添加第二条带信息。这样,再有正常数据块损坏导致流式数据丢失时,该再损坏的正常数据块中存储的流式数据能够被修复。 The metadata server adds the second stripe information. In this way, when the normal data block is damaged and the streaming data is lost, the streaming data stored in the corrupted normal data block can be repaired.
另外,为了保证该再损坏的正常数据块中存储的流式数据能够被准确地修复,元数据服务器在接收到第二条带信息后,还可以删除第一条带信息。In addition, in order to ensure that the streamed data stored in the normal data block that is re-damaged can be accurately repaired, the metadata server may delete the first stripe information after receiving the second stripe information.
以S203中的例子进行说明,若对于Object2组第一数据块,由数据块13、数据块14、2个预设数据块和校验块3构成一个新正常条带7,将新正常条带5、新正常条带6和新正常条带7的条带信息作为第二条带信息,并发送给元数据服务器,元数据服务器中添加新正常条带5、新正常条带6和新正常条带7的条带信息,另外,删除原条带1的条带信息、条带2的条带信息、条带3的条带信息和条带4的条带信息。As an example in S203, if a new normal strip 7 is formed by the data block 13, the data block 14, the two preset data blocks, and the check block 3 for the first data block of the Object2 group, the new normal strip will be used. 5. The strip information of the new normal strip 6 and the new normal strip 7 is used as the second strip information and sent to the metadata server, and the new normal strip 5, the new normal strip 6 and the new normal are added to the metadata server. The strip information of the strip 7 is additionally deleted for the strip information of the original strip 1, the strip information of the strip 2, the strip information of the strip 3, and the strip information of the strip 4.
另外,流式数据依据流式对象数据的元数据进行读取,在重新构建了条带后,存储流式数据的数据块和对应的条带发生了变化,因此,为了保证对流式数据的读取,元数据服务器在获取到第二条带信息后,还应更新流式对象数据的元数据。In addition, the streaming data is read according to the metadata of the streaming object data. After the strip is reconstructed, the data block storing the streaming data and the corresponding stripe are changed. Therefore, in order to ensure the reading of the streaming data. After the metadata server obtains the second strip information, it should also update the metadata of the streaming object data.
应用上述实施例,存储服务器从元数据服务器中获取至少一条损坏条带的第一条带信息,该第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据,按照正常块中存储的流式数据所属的流式对象数据,对正常块的标识对应的正常块进行分组,进而构建至少一条新正常条带,并将第二条带信息发送给元数据服务器,元数据服务器添加第二条带信息。可见,本申请实施例中,当损坏条带对应的正常块再损坏时,因该损坏条带对应的正常块再次重新构建了新正常条带,而该新正常条带对应的数据块都为正常块,因此可以根据该新正常条带的信息,对再损坏的正常块中存储的流式数据进行修复,进而使得损坏条带对应的正常数据块再损坏时,该再损坏的正常数据块中存储的流式数据能够被修复。Applying the foregoing embodiment, the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs The object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element. The data server, the metadata server adds the second stripe information. It can be seen that, in the embodiment of the present application, when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again. The streaming data stored in it can be repaired.
参考图6,图6为本申请实施例提供的另一种信息处理方法的一种流程示意图,应用于元数据服务器,该方法包括:Referring to FIG. 6, FIG. 6 is a schematic flowchart of another information processing method according to an embodiment of the present application, which is applied to a metadata server, and the method includes:
S601:将至少一条损坏条带的条带信息发送给存储服务器;S601: Send the strip information of the at least one damaged strip to the storage server;
这种情况下,存储服务器就可以将这至少一条损坏条带的条带信息作为第一条带信息,其中,第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;将正常块的标识对应的正常块作为第一数据 块,按照正常块中存储的流式数据所属的流式对象数据,对第一数据块进行分组;构建针对每组第一数据块的至少一条新正常条带。In this case, the storage server may use the stripe information of the at least one damaged stripe as the first stripe information, where the first stripe information includes: an identifier of the normal block and a streamed data stored in the normal block. Streaming object data; the normal block corresponding to the identifier of the normal block is used as the first data Blocking, grouping the first data blocks according to the streaming object data to which the streaming data stored in the normal block belongs; constructing at least one new normal strip for each set of the first data block.
在本申请的一个实施例中,存储服务器获取的损坏条带的条带信息可以为:元数据服务器中当前存储的所有损坏条带的条带信息。一般的,元数据服务器可以定时地将当前存储的所有损坏条带的条带信息发送给存储服务器,也可以在该元数据服务器中存储了预设数量个损坏条带的条带信息时,则将当前存储的这预设数量个损坏条带的条带信息都发送给存储服务器。In an embodiment of the present application, the stripe information of the damaged stripe acquired by the storage server may be: stripe information of all damaged strips currently stored in the metadata server. Generally, the metadata server may periodically send the stripe information of all the damaged strips currently stored to the storage server, or when the stripe information of the preset number of damaged strips is stored in the metadata server, The strip information of the preset number of damaged strips currently stored is sent to the storage server.
在本申请的一个实施例中,参考图7,在图6的基础上,每个条带对应N个数据块和M个校验块,其中,N和M为正整数,在将至少一条损坏条带的条带信息发送给存储服务器之前,上述信息处理方法还可以包括:In an embodiment of the present application, referring to FIG. 7, on the basis of FIG. 6, each strip corresponds to N data blocks and M check blocks, where N and M are positive integers, and at least one piece is damaged. Before the stripe information of the stripe is sent to the storage server, the information processing method may further include:
S701:接收存储服务器发送的坏块的标识;S701: Receive an identifier of a bad block sent by the storage server.
一般的,存储服务器可以定时地检测本地是否存在坏块,当检测到存在时,将坏块的标识发送给元数据服务器。Generally, the storage server can periodically detect whether there is a bad block locally, and when the presence is detected, send the identifier of the bad block to the metadata server.
S702:确定坏块的标识对应的正常条带的条带信息;S702: Determine strip information of a normal stripe corresponding to the identifier of the bad block;
S703:将正常条带的条带信息中坏块的标识对应的正常块标记为坏块;S703: mark the normal block corresponding to the identifier of the bad block in the strip information of the normal stripe as a bad block;
S704:当正常条带的条带信息中标记为坏块的正常块的个数大于M时,将正常条带的条带信息更新为损坏条带的条带信息。S704: When the number of normal blocks marked as bad blocks in the strip information of the normal strip is greater than M, the strip information of the normal strip is updated to the strip information of the damaged strip.
S602:接收存储服务器发送的第二条带信息;该第二条带信息为至少一条新正常条带的条带信息;S602: Receive second strip information sent by the storage server; the second strip information is strip information of at least one new normal stripe;
S603:存储第二条带信息。S603: Store the second strip information.
另外,为了保证该再损坏的正常数据块中存储的流式数据能够被准确地修复,避免一个正常数据块对应多个条带信息,元数据服务器在接收到第二条带信息后,还可以删除第一条带信息。In addition, in order to ensure that the stream data stored in the normal data block that is damaged again can be accurately repaired, and avoiding a normal data block corresponding to multiple stripe information, the metadata server may also receive the second strip information after receiving the second strip information. Delete the first strip information.
应用上述实施例,存储服务器从元数据服务器中获取至少一条损坏条带的第一条带信息,该第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据,按照正常块中存储的流式数据所属的流式对象数据,对正常块的标识对应的数据块进行分组,进而构建至少一条新正常条 带,并将第二条带信息发送给元数据服务器,元数据服务器添加第二条带信息。可见,本申请实施例中,当损坏条带对应的正常块再损坏时,因该损坏条带对应的正常块再次重新构建了新正常条带,而该新正常条带的数据块都为正常块,因此可以根据该新正常条带的信息,对再损坏的正常块中存储的流式数据进行修复,进而保证了损坏条带对应的数据块再损坏时,该再损坏的数据块中存储的流式数据的修复。Applying the foregoing embodiment, the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs The object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the data blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal strip. Band, and send the second strip information to the metadata server, the metadata server adds the second strip information. It can be seen that, in the embodiment of the present application, when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the data block of the new normal strip is normal. Block, so according to the information of the new normal stripe, the stream data stored in the normal block that is damaged again can be repaired, thereby ensuring that the data block corresponding to the damaged strip is damaged again, and the damaged data block is stored. Repair of streaming data.
与方法实施例对应,本申请实施例还提供了一种信息处理装置。Corresponding to the method embodiment, the embodiment of the present application further provides an information processing apparatus.
参考图8,图8为本申请实施例提供的一种信息处理装置的一种结构示意图,应用于存储服务器,所述装置包括:Referring to FIG. 8, FIG. 8 is a schematic structural diagram of an information processing apparatus according to an embodiment of the present application, which is applied to a storage server, where the apparatus includes:
第一信息获取单元801,用于从元数据服务器中获取至少一条损坏条带的条带信息,将所述至少一条损坏条带的条带信息作为第一条带信息;所述第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;a first information acquiring unit 801, configured to acquire strip information of at least one damaged strip from the metadata server, and use strip information of the at least one damaged strip as first strip information; the first strip The information includes: an identifier of a normal block and streaming object data to which the streaming data stored in the normal block belongs;
块分组单元802,用于将所述正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流式数据所属的流式对象数据,对所述第一数据块进行分组;a block grouping unit 802, configured to use the normal block corresponding to the identifier of the normal block as the first data block, and group the first data block according to the streaming object data to which the streaming data stored in the normal block belongs;
条带构建单元803,用于构建针对每组第一数据块的至少一条新正常条带;a strip construction unit 803, configured to construct at least one new normal strip for each set of first data blocks;
第二信息获取单元804,用于获取所述至少一条新正常条带的条带信息,将所述至少一条新正常条带的条带信息作为第二条带信息;a second information acquiring unit 804, configured to acquire stripe information of the at least one new normal stripe, and use stripe information of the at least one new normal stripe as the second stripe information;
信息发送单元805,用于将所述第二条带信息发送给所述元数据服务器,以使所述元数据服务器添加所述第二条带信息。The information sending unit 805 is configured to send the second stripe information to the metadata server, so that the metadata server adds the second stripe information.
在本申请的其他实施例中,参考图9,在图8的基础上,所述装置还可以包括:In other embodiments of the present application, referring to FIG. 9, on the basis of FIG. 8, the apparatus may further include:
块检测单元901,用于在从元数据服务器中获取至少一条损坏条带的条带信息之前,检测本地是否存在坏块;The block detecting unit 901 is configured to detect whether a bad block exists locally before acquiring the stripe information of the at least one damaged stripe from the metadata server;
标识发送单元902,用于在所述块检测单元901的检测结果为是的情况下,将所述坏块的标识发送给所述元数据服务器,以使所述元数据服务器根据所 述坏块的标识更新正常条带的条带信息和损坏条带的条带信息。The identifier sending unit 902 is configured to send the identifier of the bad block to the metadata server if the detection result of the block detecting unit 901 is yes, so that the metadata server is configured according to the The identification of the bad block updates the strip information of the normal strip and the strip information of the damaged strip.
在本申请的其他实施例中,所述第一信息获取单元801,具体可以用于:In other embodiments of the present application, the first information acquiring unit 801 may be specifically configured to:
接收所述元数据服务器发送的当前存储的预设数量个损坏条带的条带信息;或,Receiving stripe information of a preset number of damaged strips currently stored by the metadata server; or
接收所述元数据服务器定时发送的当前存储的所有损坏条带的条带信息。Receiving stripe information of all currently stored corrupted strips periodically sent by the metadata server.
在本申请的其他实施例中,所述第一条带信息还可以包括:正常块中存储的流式数据在流式对象数据中偏移量;In other embodiments of the present application, the first stripe information may further include: an offset of the streaming data stored in the normal block in the streaming object data;
这种情况下,所述装置还可以包括:In this case, the device may further include:
块排序单元,用于在按照正常块中存储的流式数据所属的流式对象数据,对所述第一数据块进行分组之后,按照正常块中存储的流式数据在流式对象数据中偏移量从小到大的顺序,对每组第一数据块中的第一数据块进行排序。a block sorting unit, configured to: after the first data block is grouped according to the streaming object data to which the streaming data stored in the normal block belongs, according to the streaming data stored in the normal block, in the streaming object data The order of the shifts from small to large, the first data block in each set of the first data block is sorted.
在本申请的其他实施例中,每个条带对应N个数据块和M个校验块,其中,N和M为正整数;In other embodiments of the present application, each strip corresponds to N data blocks and M check blocks, where N and M are positive integers;
所述条带构建单元803,具体可以用于:The strip construction unit 803 can be specifically configured to:
对于每组第一数据块,对该组第一数据块中排序后的第一数据块,从前至后,依次对每N个第一数据块中存储的流式数据进行纠删码计算,获得M个校验块的检验数据;由每N个第一数据块和该N个第一数据块对应的M个校验块,构建新正常条带;For each first data block of the group, the first data block in the first data block of the group is subjected to erasure coding calculation for the streaming data stored in each of the N first data blocks in order from the front to the back. Check data of M check blocks; constructing a new normal strip by each N first data blocks and M check blocks corresponding to the N first data blocks;
当该组第一数据块中剩余S个第一数据块时,获取N-S个预设数据块;由所述S个第一数据块和所述N-S个预设数据块组成N个正常块;对所述S个第一数据块中存储的流式数据和所述N-S个预设数据块中存储的数据进行纠删码计算,获得M个校验块的检验数据,由所述S个第一数据块与所述N-S个预设数据块对应的M个校验块、所述S个第一数据块和所述N-S个预设数据块,构建新正常条带,其中,S为正整数,S<N。Obtaining NS preset data blocks when the first first data blocks remain in the first data block of the group; and forming N normal blocks by the S first data blocks and the NS preset data blocks; And performing the erasure code calculation on the stream data stored in the S first data blocks and the data stored in the NS preset data blocks, and obtaining the verification data of the M check blocks, where the S first Constructing a new normal stripe, wherein S is a positive integer, the data block and the M check blocks corresponding to the NS preset data blocks, the S first data blocks, and the NS preset data blocks. S<N.
在本申请的其他实施例中,所述装置还可以包括:In other embodiments of the present application, the apparatus may further include:
存储拒绝单元,用于拒绝将所述N-S个预设数据块中的数据存储至所述存 储服务器。a storage rejecting unit, configured to refuse to store data in the N-S preset data blocks to the storage Storage server.
应用上述实施例,存储服务器从元数据服务器中获取至少一条损坏条带的第一条带信息,该第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据,按照正常块中存储的流式数据所属的流式对象数据,对正常块的标识对应的正常块进行分组,进而构建至少一条新正常条带,并将第二条带信息发送给元数据服务器,元数据服务器添加第二条带信息。可见,本申请实施例中,当损坏条带对应的正常块再损坏时,因该损坏条带对应的正常块再次重新构建了新正常条带,而该新正常条带对应的数据块都为正常块,因此可以根据该新正常条带的信息,对再损坏的正常块中存储的流式数据进行修复,进而使得损坏条带对应的正常数据块再损坏时,该再损坏的正常数据块中存储的流式数据能够被修复。Applying the foregoing embodiment, the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs The object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element. The data server, the metadata server adds the second stripe information. It can be seen that, in the embodiment of the present application, when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again. The streaming data stored in it can be repaired.
参考图10,图10为本申请实施例提供的另一种信息处理装置的一种结构示意图,应用于元数据服务器,所述装置包括:Referring to FIG. 10, FIG. 10 is a schematic structural diagram of another information processing apparatus according to an embodiment of the present disclosure, which is applied to a metadata server, where the apparatus includes:
信息发送单元1001,用于将至少一条损坏条带的条带信息发送给存储服务器,以使所述存储服务器将所述至少一条损坏条带的条带信息作为第一条带信息,所述第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;将所述正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流式数据所属的流式对象数据,对所述第一数据块进行分组;构建针对每组第一数据块的至少一条新正常条带;The information sending unit 1001 is configured to send the strip information of the at least one damaged strip to the storage server, so that the storage server uses the strip information of the at least one damaged strip as the first strip information, where the The information of the strip includes: the identifier of the normal block and the stream object data to which the stream data stored in the normal block belongs; the normal block corresponding to the identifier of the normal block is used as the first data block, and the stream stored in the normal block is used. Streaming object data to which the data belongs, grouping the first data block; constructing at least one new normal strip for each set of the first data block;
信息接收单元1002,用于接收所述存储服务器发送的第二条带信息;所述第二条带信息为所述至少一条新正常条带的条带信息;The information receiving unit 1002 is configured to receive second strip information sent by the storage server, where the second strip information is strip information of the at least one new normal stripe;
信息存储单元1003,用于存储所述第二条带信息。The information storage unit 1003 is configured to store the second strip information.
在本申请的其他实施例中,每个条带对应N个数据块和M个校验块,其中,N和M为正整数;In other embodiments of the present application, each strip corresponds to N data blocks and M check blocks, where N and M are positive integers;
这种情况下,参考图11,在图10的基础上,所述装置还可以包括:In this case, referring to FIG. 11, on the basis of FIG. 10, the apparatus may further include:
标识接收单元1101,用于在将至少一条损坏条带的条带信息发送给存储服务器之前,接收所述存储服务器发送的坏块的标识; The identifier receiving unit 1101 is configured to receive an identifier of the bad block sent by the storage server before sending the stripe information of the at least one damaged strip to the storage server;
信息确定单元1102,用于确定所述坏块的标识对应的正常条带的条带信息;The information determining unit 1102 is configured to determine strip information of a normal stripe corresponding to the identifier of the bad block;
块标记单元1103,用于将所述正常条带的条带信息中所述坏块的标识对应的正常块标记为坏块;a block marking unit 1103, configured to mark a normal block corresponding to the identifier of the bad block in the stripe information of the normal stripe as a bad block;
信息更新单元1104,用于当所述正常条带的条带信息中标记为坏块的正常块的个数大于M时,将所述正常条带的条带信息更新为损坏条带的条带信息。The information updating unit 1104 is configured to update the strip information of the normal stripe to a strip of the damaged strip when the number of normal blocks marked as bad blocks in the strip information of the normal strip is greater than M information.
在本申请的其他实施例中,所述信息发送单元1001,具体可以用于:In other embodiments of the present application, the information sending unit 1001 may be specifically configured to:
当所述元数据服务器中当前存储了预设数量个损坏条带的条带信息时,将所有所述损坏条带的条带信息发送给存储服务器;或,When the stripe information of the preset number of damaged strips is currently stored in the metadata server, the stripe information of all the damaged strips is sent to the storage server; or
定时将当前存储的所有损坏条带的条带信息发送给存储服务器。The stripe information of all damaged strips currently stored is periodically sent to the storage server.
在本申请的其他实施例中,所述装置还包括:In other embodiments of the present application, the apparatus further includes:
信息删除单元,用于在存储所述第二条带信息之后,删除所述第一条带信息。And an information deleting unit, configured to delete the first strip information after storing the second strip information.
应用上述实施例,存储服务器从元数据服务器中获取至少一条损坏条带的第一条带信息,该第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据,按照正常块中存储的流式数据所属的流式对象数据,对正常块的标识对应的正常块进行分组,进而构建至少一条新正常条带,并将第二条带信息发送给元数据服务器,元数据服务器添加第二条带信息。可见,本申请实施例中,当损坏条带对应的正常块再损坏时,因该损坏条带对应的正常块再次重新构建了新正常条带,而该新正常条带对应的数据块都为正常块,因此可以根据该新正常条带的信息,对再损坏的正常块中存储的流式数据进行修复,进而使得损坏条带对应的正常数据块再损坏时,该再损坏的正常数据块中存储的流式数据能够被修复。Applying the foregoing embodiment, the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs The object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element. The data server, the metadata server adds the second stripe information. It can be seen that, in the embodiment of the present application, when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again. The streaming data stored in it can be repaired.
本申请实施例还提供了一种存储服务器,包括处理器、通信接口、存储器和通信总线;其中,处理器、通信接口、存储器通过通信总线完成相互间的通信;The embodiment of the present application further provides a storage server, including a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory complete communication with each other through a communication bus;
存储器,用于存放应用程序; Memory for storing applications;
处理器,用于执行存储器上所存放的应用程序时,实现信息处理方法。其中,信息处理方法,包括:A processor that implements an information processing method when executing an application stored on the memory. The information processing method includes:
从元数据服务器中获取至少一条损坏条带的条带信息,将至少一条损坏条带的条带信息作为第一条带信息;第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;Obtaining at least one stripe information of the damaged stripe from the metadata server, and using the stripe information of the at least one damaged stripe as the first stripe information; the first stripe information includes: an identifier of the normal block and the storage in the normal block Streaming object data to which streaming data belongs;
将正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流式数据所属的流式对象数据,对第一数据块进行分组;The normal block corresponding to the identifier of the normal block is used as the first data block, and the first data block is grouped according to the streaming object data to which the streaming data stored in the normal block belongs;
构建针对每组第一数据块的至少一条新正常条带;Constructing at least one new normal strip for each set of first data blocks;
获取至少一条新正常条带的条带信息,将至少一条新正常条带的条带信息作为第二条带信息;Obtaining strip information of at least one new normal stripe, and using stripe information of at least one new normal stripe as the second stripe information;
将第二条带信息发送给元数据服务器,以使元数据服务器添加第二条带信息。The second stripe information is sent to the metadata server to cause the metadata server to add the second stripe information.
在本申请的一个实施例中,在从元数据服务器中获取至少一条损坏条带的条带信息的步骤之前,还可以包括:In an embodiment of the present application, before the step of acquiring the stripe information of the at least one damaged strip from the metadata server, the method may further include:
检测本地是否存在坏块;Detect whether there is a bad block in the local area;
如果是,将坏块的标识发送给元数据服务器,以使元数据服务器根据坏块的标识更新正常条带的条带信息和损坏条带的条带信息。If so, the identifier of the bad block is sent to the metadata server, so that the metadata server updates the stripe information of the normal stripe and the stripe information of the damaged stripe according to the identifier of the bad block.
在本申请的一个实施例中,从元数据服务器中获取至少一条损坏条带的条带信息的步骤,包括:In an embodiment of the present application, the step of acquiring strip information of at least one damaged strip from the metadata server includes:
接收元数据服务器发送的当前存储的预设数量个损坏条带的条带信息;或,Receiving stripe information of a preset number of damaged strips currently stored by the metadata server; or
接收元数据服务器定时发送的当前存储的所有损坏条带的条带信息。Receives stripe information of all corrupted strips currently stored by the metadata server.
在本申请的一个实施例中,第一条带信息还可以包括:正常块中存储的流式数据在流式对象数据中偏移量;In an embodiment of the present application, the first stripe information may further include: an offset of the streaming data stored in the normal block in the streaming object data;
这种情况下,在按照正常块中存储的流式数据所属的流式对象数据,对第一数据块进行分组的步骤之后,还可以包括: In this case, after the step of grouping the first data block according to the streaming object data to which the streaming data stored in the normal block belongs, the method may further include:
按照正常块中存储的流式数据在流式对象数据中偏移量从小到大的顺序,对每组第一数据块中的第一数据块进行排序。The first data block in each set of the first data block is sorted in order of the offset of the streaming data stored in the normal block in the streaming object data from small to large.
在本申请的一个实施例中,每个条带对应N个数据块和M个校验块,其中,N和M为正整数;In an embodiment of the present application, each strip corresponds to N data blocks and M check blocks, where N and M are positive integers;
这种情况下,构建针对每组第一数据块的至少一条新正常条带的步骤,包括:In this case, the step of constructing at least one new normal strip for each set of first data blocks includes:
对于每组第一数据块,对该组第一数据块中排序后的第一数据块,从前至后,依次对每N个第一数据块中存储的流式数据进行纠删码计算,获得M个校验块的检验数据;由每N个第一数据块和该N个第一数据块对应的M个校验块,构建新正常条带;For each first data block of the group, the first data block in the first data block of the group is subjected to erasure coding calculation for the streaming data stored in each of the N first data blocks in order from the front to the back. Check data of M check blocks; constructing a new normal strip by each N first data blocks and M check blocks corresponding to the N first data blocks;
当该组第一数据块中剩余S个第一数据块时,获取N-S个预设数据块;由S个第一数据块和N-S个预设数据块组成N个正常块,对S个第一数据块中存储的流式数据和N-S个预设数据块中存储的数据进行纠删码计算,获得M个校验块的检验数据,由S个第一数据块与N-S个预设数据块对应的M个校验块、S个第一数据块和N-S个预设数据块,构建新正常条带,其中,S为正整数,S<N。When the first data blocks are stored in the first data block of the group, the NS preset data blocks are obtained; the S first data blocks and the NS preset data blocks form N normal blocks, and the first ones are S. The stream data stored in the data block and the data stored in the NS preset data blocks are subjected to erasure code calculation, and the check data of the M check blocks is obtained, and the S first data blocks correspond to the NS preset data blocks. M check blocks, S first data blocks, and NS preset data blocks are constructed to construct a new normal stripe, where S is a positive integer, S < N.
在本申请的一个实施例中,上述信息处理方法还可以包括:In an embodiment of the present application, the information processing method may further include:
拒绝将N-S个预设数据块中的数据存储至存储服务器。The data in the N-S preset data blocks is refused to be stored to the storage server.
应用上述实施例,存储服务器从元数据服务器中获取至少一条损坏条带的第一条带信息,该第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据,按照正常块中存储的流式数据所属的流式对象数据,对正常块的标识对应的正常块进行分组,进而构建至少一条新正常条带,并将第二条带信息发送给元数据服务器,元数据服务器添加第二条带信息。可见,本申请实施例中,当损坏条带对应的正常块再损坏时,因该损坏条带对应的正常块再次重新构建了新正常条带,而该新正常条带对应的数据块都为正常块,因此可以根据该新正常条带的信息,对再损坏的正常块中存储的流式数据进行修复,进而使得损坏条带对应的正常数据块再损坏时,该再损坏的正常数据块中存储的流式数据能够被修复。Applying the foregoing embodiment, the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs The object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element. The data server, the metadata server adds the second stripe information. It can be seen that, in the embodiment of the present application, when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again. The streaming data stored in it can be repaired.
上述存储服务器提到的通信总线可以是PCI(Peripheral Component  Interconnect,外设部件互连标准)总线或EISA(Extended Industry Standard Architecture,扩展工业标准结构)总线等。该通信总线可以分为地址总线、数据总线、控制总线等。The communication bus mentioned by the above storage server may be PCI (Peripheral Component) Interconnect, peripheral component interconnect standard) bus or EISA (Extended Industry Standard Architecture) bus. The communication bus can be divided into an address bus, a data bus, a control bus, and the like.
通信接口用于上述存储服务器与其他设备之间的通信。The communication interface is used for communication between the above storage server and other devices.
存储器可以包括RAM(Random Access Memory,随机存取存储器),也可以包括NVM(Non-Volatile Memory,非易失性存储器),例如至少一个磁盘存储器。可选的,存储器还可以是至少一个位于远离前述处理器的存储装置。The memory may include a RAM (Random Access Memory), and may also include NVM (Non-Volatile Memory), such as at least one disk storage. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
上述处理器可以是通用处理器,包括CPU(Central Processing Unit,中央处理器)、NP(Network Processor,网络处理器)等;还可以是DSP(Digital Signal Processing,数字信号处理器)、ASIC(Application Specific Integrated Circuit,专用集成电路)、FPGA(Field-Programmable Gate Array,现场可编程门阵列)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。The processor may be a general-purpose processor, including a CPU (Central Processing Unit), an NP (Network Processor), or the like; or a DSP (Digital Signal Processing) or an ASIC (Application) Specific Integrated Circuit, FPGA (Field-Programmable Gate Array) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components.
本申请实施例还提供了一种元数据服务器,包括处理器、通信接口、存储器和通信总线;其中,处理器、通信接口、存储器通过通信总线完成相互间的通信;The embodiment of the present application further provides a metadata server, including a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;
存储器,用于存放应用程序;Memory for storing applications;
处理器,用于执行存储器上所存放的应用程序时,实现信息处理方法。其中,信息处理方法,包括:A processor that implements an information processing method when executing an application stored on the memory. The information processing method includes:
将至少一条损坏条带的条带信息发送给存储服务器,以使存储服务器将至少一条损坏条带的条带信息作为第一条带信息,第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;将正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流式数据所属的流式对象数据,对第一数据块进行分组;构建针对每组第一数据块的至少一条新正常条带;Sending the stripe information of the at least one damaged strip to the storage server, so that the storage server uses the stripe information of the at least one damaged strip as the first stripe information, where the first stripe information includes: the identifier of the normal block and the normal block The streaming object data to which the streaming data stored in the stream belongs; the normal block corresponding to the identifier of the normal block is used as the first data block, and the first data block is performed according to the streaming object data to which the streaming data stored in the normal block belongs Grouping; constructing at least one new normal strip for each set of first data blocks;
接收存储服务器发送的第二条带信息;第二条带信息为至少一条新正常条带的条带信息; Receiving, by the storage server, second strip information; the second strip information is strip information of at least one new normal stripe;
存储第二条带信息。Store the second strip information.
在本申请的一个实施例中,每个条带对应N个数据块和M个校验块,其中,N和M为正整数;In an embodiment of the present application, each strip corresponds to N data blocks and M check blocks, where N and M are positive integers;
这种情况下,在将至少一条损坏条带的条带信息发送给存储服务器的步骤之前,还可以包括:In this case, before the step of sending the stripe information of the at least one damaged strip to the storage server, the method may further include:
接收存储服务器发送的坏块的标识;Receiving an identifier of a bad block sent by the storage server;
确定坏块的标识对应的正常条带的条带信息;Determining strip information of a normal strip corresponding to the identifier of the bad block;
将正常条带的条带信息中坏块的标识对应的正常块标记为坏块;Marking the normal block corresponding to the identifier of the bad block in the strip information of the normal stripe as a bad block;
当正常条带的条带信息中标记为坏块的正常块的个数大于M时,将正常条带的条带信息更新为损坏条带的条带信息。When the number of normal blocks marked as bad blocks in the strip information of the normal strip is greater than M, the strip information of the normal strip is updated to the strip information of the damaged strip.
在本申请的一个实施例中,将至少一条损坏条带的条带信息发送给存储服务器的步骤,包括:In an embodiment of the present application, the step of transmitting the stripe information of the at least one damaged strip to the storage server includes:
当元数据服务器中当前存储了预设数量个损坏条带的条带信息时,将所有损坏条带的条带信息发送给存储服务器;或,When the stripe information of the preset number of damaged strips is currently stored in the metadata server, the stripe information of all the damaged strips is sent to the storage server; or
定时将当前存储的所有损坏条带的条带信息发送给存储服务器。The stripe information of all damaged strips currently stored is periodically sent to the storage server.
在本申请的一个实施例中,在存储第二条带信息的步骤之后,还可以包括:In an embodiment of the present application, after the step of storing the second strip information, the method further includes:
删除第一条带信息。Delete the first strip information.
应用上述实施例,存储服务器从元数据服务器中获取至少一条损坏条带的第一条带信息,该第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据,按照正常块中存储的流式数据所属的流式对象数据,对正常块的标识对应的正常块进行分组,进而构建至少一条新正常条带,并将第二条带信息发送给元数据服务器,元数据服务器添加第二条带信息。可见,本申请实施例中,当损坏条带对应的正常块再损坏时,因该损坏条带对应的正常块再次重新构建了新正常条带,而该新正常条带对应的数据块都为正常块,因此可以根据该新正常条带的信息,对再损坏的正常块中存 储的流式数据进行修复,进而使得损坏条带对应的正常数据块再损坏时,该再损坏的正常数据块中存储的流式数据能够被修复。Applying the foregoing embodiment, the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs The object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element. The data server, the metadata server adds the second stripe information. It can be seen that, in the embodiment of the present application, when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so it can be stored in the normal block that is damaged again based on the information of the new normal strip. The stored streaming data is repaired, so that when the normal data block corresponding to the damaged strip is damaged again, the streaming data stored in the corrupted normal data block can be repaired.
上述元数据服务器提到的通信总线可以是PCI总线或EISA总线等。该通信总线可以分为地址总线、数据总线、控制总线等。The communication bus mentioned by the above metadata server may be a PCI bus or an EISA bus or the like. The communication bus can be divided into an address bus, a data bus, a control bus, and the like.
通信接口用于上述元数据服务器与其他设备之间的通信。The communication interface is used for communication between the above metadata server and other devices.
存储器可以包括RAM,也可以包括NVM,例如至少一个磁盘存储器。可选的,存储器还可以是至少一个位于远离前述处理器的存储装置。The memory may include RAM, and may also include an NVM, such as at least one disk storage. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
上述处理器可以是通用处理器,包括CPU、NP等;还可以是DSP、ASIC、FPGA或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。The above processor may be a general-purpose processor, including a CPU, an NP, etc.; or may be a DSP, an ASIC, an FPGA, or other programmable logic device, a discrete gate or a transistor logic device, or a discrete hardware component.
本申请实施例还提供了一种应用程序,应用程序用于在运行时实现信息处理方法。其中,信息处理方法,包括:The embodiment of the present application also provides an application program for implementing an information processing method at runtime. The information processing method includes:
从元数据服务器中获取至少一条损坏条带的条带信息,将至少一条损坏条带的条带信息作为第一条带信息;第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;Obtaining at least one stripe information of the damaged stripe from the metadata server, and using the stripe information of the at least one damaged stripe as the first stripe information; the first stripe information includes: an identifier of the normal block and the storage in the normal block Streaming object data to which streaming data belongs;
将正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流式数据所属的流式对象数据,对第一数据块进行分组;The normal block corresponding to the identifier of the normal block is used as the first data block, and the first data block is grouped according to the streaming object data to which the streaming data stored in the normal block belongs;
构建针对每组第一数据块的至少一条新正常条带;Constructing at least one new normal strip for each set of first data blocks;
获取至少一条新正常条带的条带信息,将至少一条新正常条带的条带信息作为第二条带信息;Obtaining strip information of at least one new normal stripe, and using stripe information of at least one new normal stripe as the second stripe information;
将第二条带信息发送给元数据服务器,以使元数据服务器添加第二条带信息。The second stripe information is sent to the metadata server to cause the metadata server to add the second stripe information.
在本申请的一个实施例中,在从元数据服务器中获取至少一条损坏条带的条带信息的步骤之前,还可以包括:In an embodiment of the present application, before the step of acquiring the stripe information of the at least one damaged strip from the metadata server, the method may further include:
检测本地是否存在坏块;Detect whether there is a bad block in the local area;
如果是,将坏块的标识发送给元数据服务器,以使元数据服务器根据坏块的标识更新正常条带的条带信息和损坏条带的条带信息。 If so, the identifier of the bad block is sent to the metadata server, so that the metadata server updates the stripe information of the normal stripe and the stripe information of the damaged stripe according to the identifier of the bad block.
在本申请的一个实施例中,从元数据服务器中获取至少一条损坏条带的条带信息的步骤,包括:In an embodiment of the present application, the step of acquiring strip information of at least one damaged strip from the metadata server includes:
接收元数据服务器发送的当前存储的预设数量个损坏条带的条带信息;或,Receiving stripe information of a preset number of damaged strips currently stored by the metadata server; or
接收元数据服务器定时发送的当前存储的所有损坏条带的条带信息。Receives stripe information of all corrupted strips currently stored by the metadata server.
在本申请的一个实施例中,第一条带信息还可以包括:正常块中存储的流式数据在流式对象数据中偏移量;In an embodiment of the present application, the first stripe information may further include: an offset of the streaming data stored in the normal block in the streaming object data;
这种情况下,在按照正常块中存储的流式数据所属的流式对象数据,对第一数据块进行分组的步骤之后,还可以包括:In this case, after the step of grouping the first data block according to the streaming object data to which the streaming data stored in the normal block belongs, the method may further include:
按照正常块中存储的流式数据在流式对象数据中偏移量从小到大的顺序,对每组第一数据块中的第一数据块进行排序。The first data block in each set of the first data block is sorted in order of the offset of the streaming data stored in the normal block in the streaming object data from small to large.
在本申请的一个实施例中,每个条带对应N个数据块和M个校验块,其中,N和M为正整数;In an embodiment of the present application, each strip corresponds to N data blocks and M check blocks, where N and M are positive integers;
这种情况下,构建针对每组第一数据块的至少一条新正常条带的步骤,包括:In this case, the step of constructing at least one new normal strip for each set of first data blocks includes:
对于每组第一数据块,对该组第一数据块中排序后的第一数据块,从前至后,依次对每N个第一数据块中存储的流式数据进行纠删码计算,获得M个校验块的检验数据;由每N个第一数据块和该N个第一数据块对应的M个校验块,构建新正常条带;For each first data block of the group, the first data block in the first data block of the group is subjected to erasure coding calculation for the streaming data stored in each of the N first data blocks in order from the front to the back. Check data of M check blocks; constructing a new normal strip by each N first data blocks and M check blocks corresponding to the N first data blocks;
当该组第一数据块中剩余S个第一数据块时,获取N-S个预设数据块;由S个第一数据块和N-S个预设数据块组成N个正常块,对S个第一数据块中存储的流式数据和N-S个预设数据块中存储的数据进行纠删码计算,获得M个校验块的检验数据,由S个第一数据块与N-S个预设数据块对应的M个校验块、S个第一数据块和N-S个预设数据块,构建新正常条带,其中,S为正整数,S<N。When the first data blocks are stored in the first data block of the group, the NS preset data blocks are obtained; the S first data blocks and the NS preset data blocks form N normal blocks, and the first ones are S. The stream data stored in the data block and the data stored in the NS preset data blocks are subjected to erasure code calculation, and the check data of the M check blocks is obtained, and the S first data blocks correspond to the NS preset data blocks. M check blocks, S first data blocks, and NS preset data blocks are constructed to construct a new normal stripe, where S is a positive integer, S < N.
在本申请的一个实施例中,上述信息处理方法还可以包括:In an embodiment of the present application, the information processing method may further include:
拒绝将N-S个预设数据块中的数据存储至存储服务器。 The data in the N-S preset data blocks is refused to be stored to the storage server.
应用上述实施例,存储服务器从元数据服务器中获取至少一条损坏条带的第一条带信息,该第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据,按照正常块中存储的流式数据所属的流式对象数据,对正常块的标识对应的正常块进行分组,进而构建至少一条新正常条带,并将第二条带信息发送给元数据服务器,元数据服务器添加第二条带信息。可见,本申请实施例中,当损坏条带对应的正常块再损坏时,因该损坏条带对应的正常块再次重新构建了新正常条带,而该新正常条带对应的数据块都为正常块,因此可以根据该新正常条带的信息,对再损坏的正常块中存储的流式数据进行修复,进而使得损坏条带对应的正常数据块再损坏时,该再损坏的正常数据块中存储的流式数据能够被修复。Applying the foregoing embodiment, the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs The object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element. The data server, the metadata server adds the second stripe information. It can be seen that, in the embodiment of the present application, when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again. The streaming data stored in it can be repaired.
本申请实施例还提供了一种应用程序,应用程序用于在运行时实现信息处理方法。其中,信息处理方法,包括:The embodiment of the present application also provides an application program for implementing an information processing method at runtime. The information processing method includes:
将至少一条损坏条带的条带信息发送给存储服务器,以使存储服务器将至少一条损坏条带的条带信息作为第一条带信息,第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;将正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流式数据所属的流式对象数据,对第一数据块进行分组;构建针对每组第一数据块的至少一条新正常条带;Sending the stripe information of the at least one damaged strip to the storage server, so that the storage server uses the stripe information of the at least one damaged strip as the first stripe information, where the first stripe information includes: the identifier of the normal block and the normal block The streaming object data to which the streaming data stored in the stream belongs; the normal block corresponding to the identifier of the normal block is used as the first data block, and the first data block is performed according to the streaming object data to which the streaming data stored in the normal block belongs Grouping; constructing at least one new normal strip for each set of first data blocks;
接收存储服务器发送的第二条带信息;第二条带信息为至少一条新正常条带的条带信息;Receiving, by the storage server, second strip information; the second strip information is strip information of at least one new normal stripe;
存储第二条带信息。Store the second strip information.
在本申请的一个实施例中,每个条带对应N个数据块和M个校验块,其中,N和M为正整数;In an embodiment of the present application, each strip corresponds to N data blocks and M check blocks, where N and M are positive integers;
这种情况下,在将至少一条损坏条带的条带信息发送给存储服务器的步骤之前,还可以包括:In this case, before the step of sending the stripe information of the at least one damaged strip to the storage server, the method may further include:
接收存储服务器发送的坏块的标识;Receiving an identifier of a bad block sent by the storage server;
确定坏块的标识对应的正常条带的条带信息; Determining strip information of a normal strip corresponding to the identifier of the bad block;
将正常条带的条带信息中坏块的标识对应的正常块标记为坏块;Marking the normal block corresponding to the identifier of the bad block in the strip information of the normal stripe as a bad block;
当正常条带的条带信息中标记为坏块的正常块的个数大于M时,将正常条带的条带信息更新为损坏条带的条带信息。When the number of normal blocks marked as bad blocks in the strip information of the normal strip is greater than M, the strip information of the normal strip is updated to the strip information of the damaged strip.
在本申请的一个实施例中,将至少一条损坏条带的条带信息发送给存储服务器的步骤,包括:In an embodiment of the present application, the step of transmitting the stripe information of the at least one damaged strip to the storage server includes:
当元数据服务器中当前存储了预设数量个损坏条带的条带信息时,将所有损坏条带的条带信息发送给存储服务器;或,When the stripe information of the preset number of damaged strips is currently stored in the metadata server, the stripe information of all the damaged strips is sent to the storage server; or
定时将当前存储的所有损坏条带的条带信息发送给存储服务器。The stripe information of all damaged strips currently stored is periodically sent to the storage server.
在本申请的一个实施例中,在存储第二条带信息的步骤之后,还可以包括:In an embodiment of the present application, after the step of storing the second strip information, the method further includes:
删除第一条带信息。Delete the first strip information.
应用上述实施例,存储服务器从元数据服务器中获取至少一条损坏条带的第一条带信息,该第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据,按照正常块中存储的流式数据所属的流式对象数据,对正常块的标识对应的正常块进行分组,进而构建至少一条新正常条带,并将第二条带信息发送给元数据服务器,元数据服务器添加第二条带信息。可见,本申请实施例中,当损坏条带对应的正常块再损坏时,因该损坏条带对应的正常块再次重新构建了新正常条带,而该新正常条带对应的数据块都为正常块,因此可以根据该新正常条带的信息,对再损坏的正常块中存储的流式数据进行修复,进而使得损坏条带对应的正常数据块再损坏时,该再损坏的正常数据块中存储的流式数据能够被修复。Applying the foregoing embodiment, the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs The object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element. The data server, the metadata server adds the second stripe information. It can be seen that, in the embodiment of the present application, when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again. The streaming data stored in it can be repaired.
本申请实施例还提供了一种可读存储介质,可读存储介质用于存储应用程序,应用程序被运行时实现信息处理方法。其中,信息处理方法,包括:The embodiment of the present application further provides a readable storage medium for storing an application, and the application is implemented to implement an information processing method. The information processing method includes:
从元数据服务器中获取至少一条损坏条带的条带信息,将至少一条损坏条带的条带信息作为第一条带信息;第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;Obtaining at least one stripe information of the damaged stripe from the metadata server, and using the stripe information of the at least one damaged stripe as the first stripe information; the first stripe information includes: an identifier of the normal block and the storage in the normal block Streaming object data to which streaming data belongs;
将正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流 式数据所属的流式对象数据,对第一数据块进行分组;The normal block corresponding to the identifier of the normal block is used as the first data block, and the flow stored in the normal block is used. Streaming object data to which the data belongs, grouping the first data block;
构建针对每组第一数据块的至少一条新正常条带;Constructing at least one new normal strip for each set of first data blocks;
获取至少一条新正常条带的条带信息,将至少一条新正常条带的条带信息作为第二条带信息;Obtaining strip information of at least one new normal stripe, and using stripe information of at least one new normal stripe as the second stripe information;
将第二条带信息发送给元数据服务器,以使元数据服务器添加第二条带信息。The second stripe information is sent to the metadata server to cause the metadata server to add the second stripe information.
在本申请的一个实施例中,在从元数据服务器中获取至少一条损坏条带的条带信息的步骤之前,还可以包括:In an embodiment of the present application, before the step of acquiring the stripe information of the at least one damaged strip from the metadata server, the method may further include:
检测本地是否存在坏块;Detect whether there is a bad block in the local area;
如果是,将坏块的标识发送给元数据服务器,以使元数据服务器根据坏块的标识更新正常条带的条带信息和损坏条带的条带信息。If so, the identifier of the bad block is sent to the metadata server, so that the metadata server updates the stripe information of the normal stripe and the stripe information of the damaged stripe according to the identifier of the bad block.
在本申请的一个实施例中,从元数据服务器中获取至少一条损坏条带的条带信息的步骤,包括:In an embodiment of the present application, the step of acquiring strip information of at least one damaged strip from the metadata server includes:
接收元数据服务器发送的当前存储的预设数量个损坏条带的条带信息;或,Receiving stripe information of a preset number of damaged strips currently stored by the metadata server; or
接收元数据服务器定时发送的当前存储的所有损坏条带的条带信息。Receives stripe information of all corrupted strips currently stored by the metadata server.
在本申请的一个实施例中,第一条带信息还可以包括:正常块中存储的流式数据在流式对象数据中偏移量;In an embodiment of the present application, the first stripe information may further include: an offset of the streaming data stored in the normal block in the streaming object data;
这种情况下,在按照正常块中存储的流式数据所属的流式对象数据,对第一数据块进行分组的步骤之后,还可以包括:In this case, after the step of grouping the first data block according to the streaming object data to which the streaming data stored in the normal block belongs, the method may further include:
按照正常块中存储的流式数据在流式对象数据中偏移量从小到大的顺序,对每组第一数据块中的第一数据块进行排序。The first data block in each set of the first data block is sorted in order of the offset of the streaming data stored in the normal block in the streaming object data from small to large.
在本申请的一个实施例中,每个条带对应N个数据块和M个校验块,其中,N和M为正整数;In an embodiment of the present application, each strip corresponds to N data blocks and M check blocks, where N and M are positive integers;
这种情况下,构建针对每组第一数据块的至少一条新正常条带的步骤,包括: In this case, the step of constructing at least one new normal strip for each set of first data blocks includes:
对于每组第一数据块,对该组第一数据块中排序后的第一数据块,从前至后,依次对每N个第一数据块中存储的流式数据进行纠删码计算,获得M个校验块的检验数据;由每N个第一数据块和该N个第一数据块对应的M个校验块,构建新正常条带;For each first data block of the group, the first data block in the first data block of the group is subjected to erasure coding calculation for the streaming data stored in each of the N first data blocks in order from the front to the back. Check data of M check blocks; constructing a new normal strip by each N first data blocks and M check blocks corresponding to the N first data blocks;
当该组第一数据块中剩余S个第一数据块时,获取N-S个预设数据块;由S个第一数据块和N-S个预设数据块组成N个正常块,对S个第一数据块中存储的流式数据和N-S个预设数据块中存储的数据进行纠删码计算,获得M个校验块的检验数据,由S个第一数据块与N-S个预设数据块对应的M个校验块、S个第一数据块和N-S个预设数据块,构建新正常条带,其中,S为正整数,S<N。When the first data blocks are stored in the first data block of the group, the NS preset data blocks are obtained; the S first data blocks and the NS preset data blocks form N normal blocks, and the first ones are S. The stream data stored in the data block and the data stored in the NS preset data blocks are subjected to erasure code calculation, and the check data of the M check blocks is obtained, and the S first data blocks correspond to the NS preset data blocks. M check blocks, S first data blocks, and NS preset data blocks are constructed to construct a new normal stripe, where S is a positive integer, S < N.
在本申请的一个实施例中,上述信息处理方法还可以包括:In an embodiment of the present application, the information processing method may further include:
拒绝将N-S个预设数据块中的数据存储至存储服务器。The data in the N-S preset data blocks is refused to be stored to the storage server.
应用上述实施例,存储服务器从元数据服务器中获取至少一条损坏条带的第一条带信息,该第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据,按照正常块中存储的流式数据所属的流式对象数据,对正常块的标识对应的正常块进行分组,进而构建至少一条新正常条带,并将第二条带信息发送给元数据服务器,元数据服务器添加第二条带信息。可见,本申请实施例中,当损坏条带对应的正常块再损坏时,因该损坏条带对应的正常块再次重新构建了新正常条带,而该新正常条带对应的数据块都为正常块,因此可以根据该新正常条带的信息,对再损坏的正常块中存储的流式数据进行修复,进而使得损坏条带对应的正常数据块再损坏时,该再损坏的正常数据块中存储的流式数据能够被修复。Applying the foregoing embodiment, the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs The object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element. The data server, the metadata server adds the second stripe information. It can be seen that, in the embodiment of the present application, when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again. The streaming data stored in it can be repaired.
本申请实施例还提供了一种可读存储介质,可读存储介质用于存储应用程序,应用程序被运行时实现信息处理方法。其中,信息处理方法,包括:The embodiment of the present application further provides a readable storage medium for storing an application, and the application is implemented to implement an information processing method. The information processing method includes:
将至少一条损坏条带的条带信息发送给存储服务器,以使存储服务器将至少一条损坏条带的条带信息作为第一条带信息,第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;将正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流式数据所属的流式对象数据,对第一数据块进行分组;构建针对每组第一数据块的至少一条新正 常条带;Sending the stripe information of the at least one damaged strip to the storage server, so that the storage server uses the stripe information of the at least one damaged strip as the first stripe information, where the first stripe information includes: the identifier of the normal block and the normal block The streaming object data to which the streaming data stored in the stream belongs; the normal block corresponding to the identifier of the normal block is used as the first data block, and the first data block is performed according to the streaming object data to which the streaming data stored in the normal block belongs Grouping; building at least one new positive for each group of first data blocks Constant strip
接收存储服务器发送的第二条带信息;第二条带信息为至少一条新正常条带的条带信息;Receiving, by the storage server, second strip information; the second strip information is strip information of at least one new normal stripe;
存储第二条带信息。Store the second strip information.
在本申请的一个实施例中,每个条带对应N个数据块和M个校验块,其中,N和M为正整数;In an embodiment of the present application, each strip corresponds to N data blocks and M check blocks, where N and M are positive integers;
这种情况下,在将至少一条损坏条带的条带信息发送给存储服务器的步骤之前,还可以包括:In this case, before the step of sending the stripe information of the at least one damaged strip to the storage server, the method may further include:
接收存储服务器发送的坏块的标识;Receiving an identifier of a bad block sent by the storage server;
确定坏块的标识对应的正常条带的条带信息;Determining strip information of a normal strip corresponding to the identifier of the bad block;
将正常条带的条带信息中坏块的标识对应的正常块标记为坏块;Marking the normal block corresponding to the identifier of the bad block in the strip information of the normal stripe as a bad block;
当正常条带的条带信息中标记为坏块的正常块的个数大于M时,将正常条带的条带信息更新为损坏条带的条带信息。When the number of normal blocks marked as bad blocks in the strip information of the normal strip is greater than M, the strip information of the normal strip is updated to the strip information of the damaged strip.
在本申请的一个实施例中,将至少一条损坏条带的条带信息发送给存储服务器的步骤,包括:In an embodiment of the present application, the step of transmitting the stripe information of the at least one damaged strip to the storage server includes:
当元数据服务器中当前存储了预设数量个损坏条带的条带信息时,将所有损坏条带的条带信息发送给存储服务器;或,When the stripe information of the preset number of damaged strips is currently stored in the metadata server, the stripe information of all the damaged strips is sent to the storage server; or
定时将当前存储的所有损坏条带的条带信息发送给存储服务器。The stripe information of all damaged strips currently stored is periodically sent to the storage server.
在本申请的一个实施例中,在存储第二条带信息的步骤之后,还可以包括:In an embodiment of the present application, after the step of storing the second strip information, the method further includes:
删除第一条带信息。Delete the first strip information.
应用上述实施例,存储服务器从元数据服务器中获取至少一条损坏条带的第一条带信息,该第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据,按照正常块中存储的流式数据所属的流式对象数据,对正常块的标识对应的正常块进行分组,进而构建至少一条新正常条带,并将第二条带信息发送给元数据服务器,元数据服务器添加第二条带信 息。可见,本申请实施例中,当损坏条带对应的正常块再损坏时,因该损坏条带对应的正常块再次重新构建了新正常条带,而该新正常条带对应的数据块都为正常块,因此可以根据该新正常条带的信息,对再损坏的正常块中存储的流式数据进行修复,进而使得损坏条带对应的正常数据块再损坏时,该再损坏的正常数据块中存储的流式数据能够被修复。Applying the foregoing embodiment, the storage server acquires, from the metadata server, the first stripe information of the at least one damaged stripe, where the first stripe information includes: an identifier of the normal block and a stream to which the streaming data stored in the normal block belongs The object data is grouped according to the stream object data to which the stream data stored in the normal block belongs, and the normal blocks corresponding to the identifiers of the normal blocks are grouped, thereby constructing at least one new normal stripe, and transmitting the second stripe information to the element. Data server, metadata server adds second strip letter interest. It can be seen that, in the embodiment of the present application, when the normal block corresponding to the damaged strip is damaged again, the normal block corresponding to the damaged strip re-constructs the new normal strip, and the corresponding data block of the new normal strip is Normal block, so the stream data stored in the normal block that is damaged again can be repaired according to the information of the new normal stripe, so that the normal data block corresponding to the damaged strip is damaged again, and the normal data block is damaged again. The streaming data stored in it can be repaired.
对于信息处理装置、存储服务器、元数据服务器、应用程序和可读存储介质实施例而言,由于其基本相似于信息处理方法实施例,所以描述的比较简单,相关之处参见信息处理方法实施例的部分说明即可。For the information processing apparatus, the storage server, the metadata server, the application program, and the readable storage medium embodiment, since it is basically similar to the information processing method embodiment, the description is relatively simple, and the related information refers to the information processing method embodiment. Part of the description can be.
需要说明的是,在本文中,诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来,而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。It should be noted that, in this context, relational terms such as first and second are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply such entities or operations. There is any such actual relationship or order between them. Furthermore, the term "comprises" or "comprises" or "comprises" or any other variations thereof is intended to encompass a non-exclusive inclusion, such that a process, method, article, or device that comprises a plurality of elements includes not only those elements but also Other elements, or elements that are inherent to such a process, method, item, or device. An element that is defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device that comprises the element.
本说明书中的各个实施例均采用相关的方式描述,各个实施例之间相同相似的部分互相参见即可,每个实施例重点说明的都是与其他实施例的不同之处。尤其,对于信息处理装置、存储服务器、元数据服务器和可读存储介质实施例而言,由于其基本相似于信息处理方法实施例,所以描述的比较简单,相关之处参见信息处理方法实施例的部分说明即可。The various embodiments in the present specification are described in a related manner, and the same or similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the information processing apparatus, the storage server, the metadata server, and the readable storage medium embodiment, since it is basically similar to the information processing method embodiment, the description is relatively simple, and the relevant places are referred to the information processing method embodiment. Part of the description is fine.
以上所述仅为本申请的较佳实施例而已,并非用于限定本申请的保护范围。凡在本申请的精神和原则之内所作的任何修改、等同替换、改进等,均包含在本申请的保护范围内。 The above description is only the preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application are included in the scope of the present application.

Claims (26)

  1. 一种信息处理方法,其特征在于,应用于存储服务器,所述方法包括:An information processing method is applied to a storage server, and the method includes:
    从元数据服务器中获取至少一条损坏条带的条带信息,将所述至少一条损坏条带的条带信息作为第一条带信息;所述第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;Obtaining strip information of at least one damaged strip from the metadata server, and using the strip information of the at least one damaged strip as the first strip information; the first strip information includes: normal block identifier and normal Streaming object data to which the streaming data stored in the block belongs;
    将所述正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流式数据所属的流式对象数据,对所述第一数据块进行分组;The normal block corresponding to the identifier of the normal block is used as the first data block, and the first data block is grouped according to the streaming object data to which the streaming data stored in the normal block belongs;
    构建针对每组第一数据块的至少一条新正常条带;Constructing at least one new normal strip for each set of first data blocks;
    获取所述至少一条新正常条带的条带信息,将所述至少一条新正常条带的条带信息作为第二条带信息;Obtaining stripe information of the at least one new normal stripe, and using the stripe information of the at least one new normal stripe as the second stripe information;
    将所述第二条带信息发送给所述元数据服务器,以使所述元数据服务器添加所述第二条带信息。Transmitting the second stripe information to the metadata server to cause the metadata server to add the second stripe information.
  2. 根据权利要求1所述的方法,其特征在于,在所述从元数据服务器中获取至少一条损坏条带的条带信息的步骤之前,所述方法还包括:The method according to claim 1, wherein before the step of acquiring the stripe information of the at least one damaged stripe from the metadata server, the method further comprises:
    检测本地是否存在坏块;Detect whether there is a bad block in the local area;
    如果是,将所述坏块的标识发送给所述元数据服务器,以使所述元数据服务器根据所述坏块的标识更新正常条带的条带信息和损坏条带的条带信息。If yes, the identifier of the bad block is sent to the metadata server, so that the metadata server updates the stripe information of the normal stripe and the stripe information of the damaged strip according to the identifier of the bad block.
  3. 根据权利要求1或2所述的方法,其特征在于,所述从元数据服务器中获取至少一条损坏条带的条带信息的步骤,包括:The method according to claim 1 or 2, wherein the step of acquiring strip information of at least one damaged strip from the metadata server comprises:
    接收所述元数据服务器发送的当前存储的预设数量个损坏条带的条带信息;或,Receiving stripe information of a preset number of damaged strips currently stored by the metadata server; or
    接收所述元数据服务器定时发送的当前存储的所有损坏条带的条带信息。Receiving stripe information of all currently stored corrupted strips periodically sent by the metadata server.
  4. 根据权利要求1所述的方法,其特征在于,所述第一条带信息还包括:正常块中存储的流式数据在流式对象数据中偏移量;The method according to claim 1, wherein the first stripe information further comprises: an offset of the streaming data stored in the normal block in the streaming object data;
    在所述按照正常块中存储的流式数据所属的流式对象数据,对所述第一数据块进行分组的步骤之后,所述方法还包括: After the step of grouping the first data block according to the streaming object data to which the streaming data stored in the normal block belongs, the method further includes:
    按照正常块中存储的流式数据在流式对象数据中偏移量从小到大的顺序,对每组第一数据块中的第一数据块进行排序。The first data block in each set of the first data block is sorted in order of the offset of the streaming data stored in the normal block in the streaming object data from small to large.
  5. 根据权利要求4所述的方法,其特征在于,每个条带对应N个数据块和M个校验块,其中,N和M为正整数;The method according to claim 4, wherein each strip corresponds to N data blocks and M check blocks, wherein N and M are positive integers;
    所述构建针对每组第一数据块的至少一条新正常条带的步骤,包括:The step of constructing at least one new normal strip for each set of first data blocks includes:
    对于每组第一数据块,对该组第一数据块中排序后的第一数据块,从前至后,依次对每N个第一数据块中存储的流式数据进行纠删码计算,获得M个校验块的检验数据;由每N个第一数据块和该N个第一数据块对应的M个校验块,构建新正常条带;For each first data block of the group, the first data block in the first data block of the group is subjected to erasure coding calculation for the streaming data stored in each of the N first data blocks in order from the front to the back. Check data of M check blocks; constructing a new normal strip by each N first data blocks and M check blocks corresponding to the N first data blocks;
    当该组第一数据块中剩余S个第一数据块时,获取N-S个预设数据块;由所述S个第一数据块和所述N-S个预设数据块组成N个正常块,对所述S个第一数据块中存储的流式数据和所述N-S个预设数据块中存储的数据进行纠删码计算,获得M个校验块的检验数据,由所述S个第一数据块与所述N-S个预设数据块对应的M个校验块、所述S个第一数据块和所述N-S个预设数据块,构建新正常条带,其中,S为正整数,S<N。Obtaining NS preset data blocks when the first first data blocks remain in the first data block of the group; and forming the N normal blocks by the S first data blocks and the NS preset data blocks, And performing the erasure code calculation on the stream data stored in the S first data blocks and the data stored in the NS preset data blocks, and obtaining the verification data of the M check blocks, where the S first Constructing a new normal stripe, wherein S is a positive integer, the data block and the M check blocks corresponding to the NS preset data blocks, the S first data blocks, and the NS preset data blocks. S<N.
  6. 根据权利要求5所述的方法,其特征在于,所述方法还包括:The method of claim 5, wherein the method further comprises:
    拒绝将所述N-S个预设数据块中的数据存储至所述存储服务器。Refusing to store data in the N-S preset data blocks to the storage server.
  7. 一种信息处理方法,其特征在于,应用于元数据服务器,所述方法包括:An information processing method, which is applied to a metadata server, the method comprising:
    将至少一条损坏条带的条带信息发送给存储服务器,以使所述存储服务器将所述至少一条损坏条带的条带信息作为第一条带信息,所述第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;将所述正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流式数据所属的流式对象数据,对所述第一数据块进行分组;构建针对每组第一数据块的至少一条新正常条带;Transmitting the strip information of the at least one damaged strip to the storage server, so that the storage server uses the strip information of the at least one damaged strip as the first strip information, where the first strip information includes: The identifier of the block and the streaming object data to which the streaming data stored in the normal block belongs; the normal block corresponding to the identifier of the normal block is used as the first data block, and the streaming object belongs to the streaming data stored in the normal block Data, grouping the first data blocks; constructing at least one new normal strip for each set of first data blocks;
    接收所述存储服务器发送的第二条带信息;所述第二条带信息为所述至少一条新正常条带的条带信息; Receiving, by the storage server, second strip information; the second strip information is stripe information of the at least one new normal stripe;
    存储所述第二条带信息。The second strip information is stored.
  8. 根据权利要求7所述的方法,其特征在于,每个条带对应N个数据块和M个校验块,其中,N和M为正整数;The method according to claim 7, wherein each strip corresponds to N data blocks and M check blocks, wherein N and M are positive integers;
    在所述将至少一条损坏条带的条带信息发送给存储服务器的步骤之前,所述方法还包括:Before the step of transmitting the stripe information of the at least one damaged strip to the storage server, the method further includes:
    接收所述存储服务器发送的坏块的标识;Receiving an identifier of a bad block sent by the storage server;
    确定所述坏块的标识对应的正常条带的条带信息;Determining strip information of a normal strip corresponding to the identifier of the bad block;
    将所述正常条带的条带信息中所述坏块的标识对应的正常块标记为坏块;Marking, in the strip information of the normal stripe, a normal block corresponding to the identifier of the bad block as a bad block;
    当所述正常条带的条带信息中标记为坏块的正常块的个数大于M时,将所述正常条带的条带信息更新为损坏条带的条带信息。When the number of normal blocks marked as bad blocks in the strip information of the normal strip is greater than M, the strip information of the normal strip is updated to the strip information of the damaged strip.
  9. 根据权利要求7所述的方法,其特征在于,所述将至少一条损坏条带的条带信息发送给存储服务器的步骤,包括:The method according to claim 7, wherein the step of transmitting the stripe information of the at least one damaged strip to the storage server comprises:
    当所述元数据服务器中当前存储了预设数量个损坏条带的条带信息时,将所有所述损坏条带的条带信息发送给所述存储服务器;或,When the stripe information of the preset number of damaged strips is currently stored in the metadata server, the stripe information of all the damaged strips is sent to the storage server; or
    定时将当前存储的所有损坏条带的条带信息发送给所述存储服务器。Timing information of all damaged strips currently stored is periodically sent to the storage server.
  10. 根据权利要求7-9任一项所述的方法,其特征在于,在所述存储所述第二条带信息的步骤之后,所述方法还包括:The method according to any one of claims 7-9, wherein after the step of storing the second strip information, the method further comprises:
    删除所述第一条带信息。Delete the first strip information.
  11. 一种信息处理装置,其特征在于,应用于存储服务器,所述装置包括:An information processing apparatus is applied to a storage server, and the apparatus includes:
    第一信息获取单元,用于从元数据服务器中获取至少一条损坏条带的条带信息,将所述至少一条损坏条带的条带信息作为第一条带信息;所述第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;a first information acquiring unit, configured to acquire strip information of at least one damaged strip from the metadata server, and use the strip information of the at least one damaged strip as the first strip information; the first strip information The method includes: an identifier of a normal block and streaming object data to which the streaming data stored in the normal block belongs;
    块分组单元,用于将所述正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流式数据所属的流式对象数据,对所述第一数据块进行 分组;a block grouping unit, configured to use the normal block corresponding to the identifier of the normal block as the first data block, and perform the first data block according to the streaming object data to which the streaming data stored in the normal block belongs Grouping
    条带构建单元,用于构建针对每组第一数据块的至少一条新正常条带;a strip building unit for constructing at least one new normal strip for each set of first data blocks;
    第二信息获取单元,用于获取所述至少一条新正常条带的条带信息,将所述至少一条新正常条带的条带信息作为第二条带信息;a second information acquiring unit, configured to acquire stripe information of the at least one new normal stripe, and use the stripe information of the at least one new normal stripe as the second stripe information;
    信息发送单元,用于将所述第二条带信息发送给所述元数据服务器,以使所述元数据服务器添加所述第二条带信息。And an information sending unit, configured to send the second stripe information to the metadata server, so that the metadata server adds the second stripe information.
  12. 根据权利要求11所述的装置,其特征在于,所述装置还包括:The device according to claim 11, wherein the device further comprises:
    块检测单元,用于在从元数据服务器中获取至少一条损坏条带的条带信息之前,检测本地是否存在坏块;a block detecting unit, configured to detect whether a local bad block exists before acquiring the stripe information of the at least one damaged stripe from the metadata server;
    标识发送单元,用于在所述块检测单元的检测结果为是的情况下,将所述坏块的标识发送给所述元数据服务器,以使所述元数据服务器根据所述坏块的标识更新正常条带的条带信息和损坏条带的条带信息。An identifier sending unit, configured to send, by the block detection unit, an identifier of the bad block to the metadata server, so that the metadata server is configured according to the identifier of the bad block Update the strip information of the normal strip and the strip information of the damaged strip.
  13. 根据权利要求11或12所述的装置,其特征在于,所述第一信息获取单元,具体用于:The device according to claim 11 or 12, wherein the first information acquiring unit is specifically configured to:
    接收所述元数据服务器发送的当前存储的预设数量个损坏条带的条带信息;或,Receiving stripe information of a preset number of damaged strips currently stored by the metadata server; or
    接收所述元数据服务器定时发送的当前存储的所有损坏条带的条带信息。Receiving stripe information of all currently stored corrupted strips periodically sent by the metadata server.
  14. 根据权利要求11所述的装置,其特征在于,所述第一条带信息还包括:正常块中存储的流式数据在流式对象数据中偏移量;The apparatus according to claim 11, wherein the first stripe information further comprises: an offset of the streaming data stored in the normal block in the streaming object data;
    所述装置还包括:The device also includes:
    块排序单元,用于在按照正常块中存储的流式数据所属的流式对象数据,对所述第一数据块进行分组之后,按照正常块中存储的流式数据在流式对象数据中偏移量从小到大的顺序,对每组第一数据块中的第一数据块进行排序。a block sorting unit, configured to: after the first data block is grouped according to the streaming object data to which the streaming data stored in the normal block belongs, according to the streaming data stored in the normal block, in the streaming object data The order of the shifts from small to large, the first data block in each set of the first data block is sorted.
  15. 根据权利要求14所述的装置,其特征在于,每个条带对应N个数据块和M个校验块,其中,N和M为正整数;The apparatus according to claim 14, wherein each strip corresponds to N data blocks and M check blocks, wherein N and M are positive integers;
    所述条带构建单元,具体用于: The strip building unit is specifically configured to:
    对于每组第一数据块,对该组第一数据块中排序后的第一数据块,从前至后,依次对每N个第一数据块中存储的流式数据进行纠删码计算,获得M个校验块的检验数据;由每N个第一数据块和该N个第一数据块对应的M个校验块,构建新正常条带;For each first data block of the group, the first data block in the first data block of the group is subjected to erasure coding calculation for the streaming data stored in each of the N first data blocks in order from the front to the back. Check data of M check blocks; constructing a new normal strip by each N first data blocks and M check blocks corresponding to the N first data blocks;
    当该组第一数据块中剩余S个第一数据块时,获取N-S个预设数据块;由所述S个第一数据块和所述N-S个预设数据块组成N个正常块;对所述S个第一数据块中存储的流式数据和所述N-S个预设数据块中存储的数据进行纠删码计算,获得M个校验块的检验数据,由所述S个第一数据块与所述N-S个预设数据块对应的M个校验块、所述S个第一数据块和所述N-S个预设数据块,构建新正常条带,其中,S为正整数,S<N。Obtaining NS preset data blocks when the first first data blocks remain in the first data block of the group; and forming N normal blocks by the S first data blocks and the NS preset data blocks; And performing the erasure code calculation on the stream data stored in the S first data blocks and the data stored in the NS preset data blocks, and obtaining the verification data of the M check blocks, where the S first Constructing a new normal stripe, wherein S is a positive integer, the data block and the M check blocks corresponding to the NS preset data blocks, the S first data blocks, and the NS preset data blocks. S<N.
  16. 根据权利要求15所述的装置,其特征在于,所述装置还包括:The device according to claim 15, wherein the device further comprises:
    存储拒绝单元,用于拒绝将所述N-S个预设数据块中的数据存储至所述存储服务器。And a storage rejecting unit, configured to refuse to store data in the N-S preset data blocks to the storage server.
  17. 一种信息处理装置,其特征在于,应用于元数据服务器,所述装置包括:An information processing apparatus is applied to a metadata server, the apparatus comprising:
    信息发送单元,用于将至少一条损坏条带的条带信息发送给存储服务器,以使所述存储服务器将所述至少一条损坏条带的条带信息作为第一条带信息,所述第一条带信息包括:正常块的标识和正常块中存储的流式数据所属的流式对象数据;将所述正常块的标识对应的正常块作为第一数据块,按照正常块中存储的流式数据所属的流式对象数据,对所述第一数据块进行分组;构建针对每组第一数据块的至少一条新正常条带;An information sending unit, configured to send the strip information of the at least one damaged strip to the storage server, so that the storage server uses the strip information of the at least one damaged strip as the first strip information, the first The stripe information includes: an identifier of a normal block and streaming object data to which the streaming data stored in the normal block belongs; a normal block corresponding to the identifier of the normal block is used as the first data block, and the flow is stored in the normal block. Streaming object data to which the data belongs, grouping the first data block; constructing at least one new normal strip for each set of the first data block;
    信息接收单元,用于接收所述存储服务器发送的第二条带信息;所述第二条带信息为所述至少一条新正常条带的条带信息;An information receiving unit, configured to receive second strip information sent by the storage server; the second strip information is stripe information of the at least one new normal stripe;
    信息存储单元,用于存储所述第二条带信息。An information storage unit, configured to store the second strip information.
  18. 根据权利要求17所述的装置,其特征在于,每个条带对应N个数据块和M个校验块,其中,N和M为正整数;The apparatus according to claim 17, wherein each strip corresponds to N data blocks and M check blocks, wherein N and M are positive integers;
    所述装置还包括: The device also includes:
    标识接收单元,用于在将至少一条损坏条带的条带信息发送给存储服务器之前,接收所述存储服务器发送的坏块的标识;An identifier receiving unit, configured to receive an identifier of a bad block sent by the storage server before sending the stripe information of the at least one damaged strip to the storage server;
    信息确定单元,用于确定所述坏块的标识对应的正常条带的条带信息;An information determining unit, configured to determine strip information of a normal stripe corresponding to the identifier of the bad block;
    块标记单元,用于将所述正常条带的条带信息中所述坏块的标识对应的正常块标记为坏块;a block marking unit, configured to mark a normal block corresponding to the identifier of the bad block in the strip information of the normal stripe as a bad block;
    信息更新单元,用于当所述正常条带的条带信息中标记为坏块的正常块的个数大于M时,将所述正常条带的条带信息更新为损坏条带的条带信息。An information updating unit, configured to update strip information of the normal stripe to stripe information of the damaged stripe when the number of normal blocks marked as bad blocks in the stripe information of the normal stripe is greater than M .
  19. 根据权利要求17所述的装置,其特征在于,所述信息发送单元,具体用于:The device according to claim 17, wherein the information sending unit is specifically configured to:
    当所述元数据服务器中当前存储了预设数量个损坏条带的条带信息时,将所有所述损坏条带的条带信息发送给所述存储服务器;或,When the stripe information of the preset number of damaged strips is currently stored in the metadata server, the stripe information of all the damaged strips is sent to the storage server; or
    定时将当前存储的所有损坏条带的条带信息发送给所述存储服务器。Timing information of all damaged strips currently stored is periodically sent to the storage server.
  20. 根据权利要求17-19任一项所述的装置,其特征在于,所述装置还包括:The device according to any one of claims 17 to 19, wherein the device further comprises:
    信息删除单元,用于在存储所述第二条带信息之后,删除所述第一条带信息。And an information deleting unit, configured to delete the first strip information after storing the second strip information.
  21. 一种存储服务器,其特征在于,包括处理器、通信接口、存储器和通信总线;其中,所述处理器、所述通信接口、所述存储器通过所述通信总线完成相互间的通信;A storage server, comprising: a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;
    所述存储器,用于存放应用程序;The memory is configured to store an application;
    所述处理器,用于执行所述存储器上所存放的应用程序时,实现权利要求1-6任一所述的方法步骤。The processor, when used to execute an application stored on the memory, implements the method steps of any of claims 1-6.
  22. 一种元数据服务器,其特征在于,包括处理器、通信接口、存储器和通信总线;其中,所述处理器、所述通信接口、所述存储器通过所述通信总线完成相互间的通信;A metadata server, comprising: a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;
    所述存储器,用于存放应用程序; The memory is configured to store an application;
    所述处理器,用于执行所述存储器上所存放的应用程序时,实现权利要求7-10任一所述的方法步骤。The processor, when executed to execute an application stored on the memory, implements the method steps of any one of claims 7-10.
  23. 一种应用程序,其特征在于,所述应用程序用于在运行时实现权利要求1-6任一所述的方法步骤。An application, characterized in that the application is operative to implement the method steps of any of claims 1-6 at runtime.
  24. 一种应用程序,其特征在于,所述应用程序用于在运行时实现权利要求7-10任一所述的方法步骤。An application, characterized in that the application is adapted to implement the method steps of any of claims 7-10 at runtime.
  25. 一种可读存储介质,其特征在于,所述可读存储介质用于存储应用程序,所述应用程序被运行时实现权利要求1-6任一所述的方法步骤。A readable storage medium, characterized in that the readable storage medium is for storing an application, the application being executed to implement the method steps of any of claims 1-6.
  26. 一种可读存储介质,其特征在于,所述可读存储介质用于存储应用程序,所述应用程序被运行时实现权利要求7-10任一所述的方法步骤。 A readable storage medium, characterized in that the readable storage medium is for storing an application, the application being executed to implement the method steps of any of claims 7-10.
PCT/CN2017/101221 2016-12-13 2017-09-11 Method and device for information processing WO2018107828A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201611143547.2 2016-12-13
CN201611143547.2A CN108614670B (en) 2016-12-13 2016-12-13 Information processing method and device

Publications (1)

Publication Number Publication Date
WO2018107828A1 true WO2018107828A1 (en) 2018-06-21

Family

ID=62558011

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/101221 WO2018107828A1 (en) 2016-12-13 2017-09-11 Method and device for information processing

Country Status (2)

Country Link
CN (1) CN108614670B (en)
WO (1) WO2018107828A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102722340A (en) * 2012-04-27 2012-10-10 华为技术有限公司 Data processing method, apparatus and system
US8706695B1 (en) * 2012-01-26 2014-04-22 Amazon Technologies, Inc. Reducing head and tail duplication in stored data
CN104932953A (en) * 2015-06-04 2015-09-23 华为技术有限公司 Data distribution method, data storage method, and relevant device and system
CN105095013A (en) * 2015-06-04 2015-11-25 华为技术有限公司 Data storage method, data recovery method, and relevant device and system
CN105404469A (en) * 2015-10-22 2016-03-16 浙江宇视科技有限公司 Video data storage method and system

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103593253A (en) * 2013-11-22 2014-02-19 华中科技大学 Vertical RAID-6 coding method based on exclusive or
US9529670B2 (en) * 2014-05-16 2016-12-27 International Business Machines Corporation Storage element polymorphism to reduce performance degradation during error recovery
CN103970487A (en) * 2014-05-23 2014-08-06 浙江宇视科技有限公司 Redundant array of independent disks and implementation method
CN105630417B (en) * 2015-12-24 2018-07-20 创新科软件技术(深圳)有限公司 A kind of RAID5 systems and in the subsequent method for continuing data of RAID5 thrashings

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8706695B1 (en) * 2012-01-26 2014-04-22 Amazon Technologies, Inc. Reducing head and tail duplication in stored data
CN102722340A (en) * 2012-04-27 2012-10-10 华为技术有限公司 Data processing method, apparatus and system
CN104932953A (en) * 2015-06-04 2015-09-23 华为技术有限公司 Data distribution method, data storage method, and relevant device and system
CN105095013A (en) * 2015-06-04 2015-11-25 华为技术有限公司 Data storage method, data recovery method, and relevant device and system
CN105404469A (en) * 2015-10-22 2016-03-16 浙江宇视科技有限公司 Video data storage method and system

Also Published As

Publication number Publication date
CN108614670B (en) 2020-07-03
CN108614670A (en) 2018-10-02

Similar Documents

Publication Publication Date Title
US20210160053A1 (en) Merkle tree construction methods and apparatuses and simplified payment verification methods and apparatuses
US10222987B2 (en) Data deduplication with augmented cuckoo filters
WO2020151323A1 (en) Data slicing-based data storage method, device, and medium
US20130246378A1 (en) Partial hash system, method, and computer program product
WO2015051638A1 (en) Fault location method and device
WO2020211146A1 (en) Identifier association method and device, and electronic apparatus
CN105224600B (en) A kind of detection method and device of Sample Similarity
WO2020124960A1 (en) Data storage detection method and apparatus, and storage medium and electronic apparatus
US9369474B2 (en) Analytics data validation
CN108573007A (en) Method, apparatus, electronic equipment and the storage medium of data consistency detection
US7966358B2 (en) Determining an approximate number of instances of an item for an organization
US10664349B2 (en) Method and device for file storage
CN109002253B (en) Method and device for processing damaged strip
WO2016165542A1 (en) Method for analyzing cache hit rate, and device
CN108875018B (en) News influence evaluation method and device and electronic equipment
CN109597566B (en) Data reading and storing method and device
WO2018107828A1 (en) Method and device for information processing
CN111835871A (en) Method and device for transmitting data file and method and device for receiving data file
CN107612946A (en) Detection method, detection means and the electronic equipment of IP address
WO2020207008A1 (en) Data verification method, apparatus, electronic device and storage medium
JP6958241B2 (en) Change detection program, change detection method and change detection device
US20180365261A1 (en) Fingerprinting data for more aggressive de-duplication
US9374474B1 (en) System, method, and computer program for detecting duplicated telecommunications events in a consumer telecommunications network
CN108037950A (en) A kind of information deletion method, device, electronic equipment and readable storage medium storing program for executing
WO2018000825A1 (en) Data processing method and apparatus

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17880159

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17880159

Country of ref document: EP

Kind code of ref document: A1