WO2014012246A1 - Procédés de codage, de reconstruction et de récupération utilisés pour exécuter un code de réparation automatique enregistré par un réseau distribué - Google Patents

Procédés de codage, de reconstruction et de récupération utilisés pour exécuter un code de réparation automatique enregistré par un réseau distribué Download PDF

Info

Publication number
WO2014012246A1
WO2014012246A1 PCT/CN2012/078927 CN2012078927W WO2014012246A1 WO 2014012246 A1 WO2014012246 A1 WO 2014012246A1 CN 2012078927 W CN2012078927 W CN 2012078927W WO 2014012246 A1 WO2014012246 A1 WO 2014012246A1
Authority
WO
WIPO (PCT)
Prior art keywords
storage
finite field
storage node
data
node
Prior art date
Application number
PCT/CN2012/078927
Other languages
English (en)
Chinese (zh)
Inventor
李挥
吉书龙
侯韩旭
张华宇
韩元波
张卫
姜军
Original Assignee
北京大学深圳研究生院
深圳报业集团
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京大学深圳研究生院, 深圳报业集团 filed Critical 北京大学深圳研究生院
Priority to PCT/CN2012/078927 priority Critical patent/WO2014012246A1/fr
Priority to CN201280074817.2A priority patent/CN104782101B/zh
Publication of WO2014012246A1 publication Critical patent/WO2014012246A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/37—Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35
    • H03M13/3761—Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes

Definitions

  • the present invention relates to the field of distributed network storage, and more particularly to a method for encoding, reconstructing and recovering self-healing codes for distributed network storage.
  • Network storage systems have received much attention in recent years, and storage systems contain different types: such as P2P-based distributed storage systems and dedicated infrastructure systems based on data center and storage area networks. Since storage node failure or file transmission loss often occurs in distributed storage systems, redundancy must be present in the network storage system. Redundancy can be achieved by copying data from a single cartridge, but the storage effect is not high, and the error correction code provides an efficient storage scheme different from previous replication.
  • An (n, k) MDS (Maximum Distance Separable) error correcting code needs to divide an original file into k equal-sized modules, and generate n mutually uncorrelated encoding modules by linear encoding, by n nodes. Store different modules and meet the MDS properties (any k of the n encoding modules can reconstruct the original file). This coding technology plays an important role in providing effective network storage redundancy, and is particularly suitable for storing large files and archive data backup applications.
  • the EC code was originally designed to make the communication robust, that is, the failure of some modules can be tolerated in one communication channel.
  • Network storage treats the EC code as a black box, providing an efficient distributed data storage and a data recovery device via EC code.
  • the different challenges that are not addressed in the EC code faced in network storage, especially the repair problem.
  • nodes may fail or go online frequently. There must be new nodes to provide coding modules to compensate for the situation when a node leaves the system (failure) and ensure system redundancy (in order to Tolerate additional node failures afterwards).
  • any two module information can be used to repair the third module, in the literature [A. Duminuco, E. Biersack, “ Hierarchical Codes: How to Make Erasure Codes Attractive for Peer-to-Peer Storage Systems", Peer-to-Peer Computing (P2P), 2008.] proposes a HC code (Hierarchical Codes).
  • the HC code is an iterative construct that gradually forms a large code starting from a small EC code, generated by a submodule constructed by an EC code or by an EC code.
  • the local redundancy module can be used to repair the failure of the nodes in the subgroup, so only need to access the module with less than the entire file size to repair; and the global redundancy module provides further repair guarantee, that is, the module that fails in a subgroup It can be fixed by the global redundancy module when there are too many to fix itself.
  • the status of some modules may be higher than that of other modules, making it difficult to make an in-depth resilience analysis (affecting the understanding of coding effectiveness); Encoding requires more complex algorithms (whether refactoring or repairing); different encoding modules have different status in HC code, so the number of modules needed to repair a lost module depends not only on the number of modules lost, but also on which Module loss is related; likewise, the number of modules needed to reconstruct the original file may also be different The missing module is different.
  • RGC code Regenerating Codes
  • PCT/CN2012/071177 An RGC code (Regenerating Codes) is proposed in the patent PCT/CN2012/071177, which requires only a small amount of data to be repaired for a lost coding module without first reconstructing the entire file.
  • the RGC code uses a linear network coding technique to improve the overhead required to repair an encoding module through the NC (Network Coding Network Coding) attribute (ie, maximum stream minimum cut).
  • NC Network Coding Network Coding
  • the network information theory can prove the same amount of data as the lost module.
  • Network overhead can repair the original lost module.
  • the main idea of the RGC code is to use the MDS attribute. When some storage nodes fail, it is equivalent to storing data loss. It is necessary to download information from the existing valid nodes to regenerate the lost data and store it on the new node.
  • the regeneration process needs to ensure two points: 1) The failed nodes are independent of each other, and the regeneration process can be cyclically recursive; 2) Any k nodes are enough to recover the original file.
  • Figure 3 depicts the regeneration process when a node fails.
  • n storage nodes each store "data.
  • the download amount of each node is a pair for each storage node i.
  • the nodes X ' «, X '.w indicate that the pair of nodes are connected by an edge whose capacity is the storage amount of the node (ie, " ).
  • the regeneration process is described by an information flow graph, from any of the available nodes in the system. Collect each of the beta data by ⁇ « ⁇ .”'in ⁇ . "" stores a data, any one of the recipients can access X. "'.
  • the maximum information flow from the source to the sink is determined by the minimum cut set in the graph. When the sink is to reconstruct the original file, the size of the stream cannot be smaller than the size of the original file.
  • MSR Minimum-bandwidth Regenerating
  • MSR Minimum-Storage Regenerating
  • the repair bandwidth ⁇ is the smallest, ie MSS_T ' ⁇ .
  • exact repair the failed module needs to be constructed correctly, the recovered information is the same as the lost one (the core technology is interference queue and NC); function repair: the newly generated module can contain different from the missing Node data, as long as the repaired system supports MDS code attributes (core technology is NC); system part exact repair: is a hybrid repair model between exact repair and function repair, in this hybrid model, for system nodes ( The storage of unencoded data requires accurate recovery, ie the recovered information is the same as the information stored by the failed node. For non-system nodes (storage encoding module), no exact repair is required, only functional repair is required to make the recovered information full. MDS code attributes (core technology is interference queue and NC).
  • the RGC code In order to apply the RGC code to the actual distributed system, even if it is not optimal, at least the data needs to be downloaded from the k nodes to repair the lost module. Therefore, even if the data transmission amount required for the repair process is relatively low, the RGC code needs to be high.
  • the protocol load and system design (NC technology) complexity is achieved.
  • engineering solutions are not considered in the RGC code, such as the lazy repair process, so the repair load caused by temporary failure cannot be avoided.
  • the computational cost of the codec implementation of the NC-based RGC code is relatively large, which is one order higher than the traditional EC code.
  • the HSRC code mainly has the following two attributes: 1) The missing encoding module can download less than the entire file data from other encoding modules for repair; 2) The missing encoding module is repaired from a given number of modules, the given number is only It is related to how many modules are lost, and it is not related to which modules are lost. These attributes make the load of repairing a lost module relatively low. In addition, because the nodes in the system have the same status and load balancing, different lost modules can be repaired independently and concurrently in different locations of the network.
  • the codeword has the following characteristics: 1) When a node fails, there may be (n-1)/2 pairs of repair nodes available for selection; 2) when there are (n-1)/2 nodes At the same time, we can still use the remaining 2 nodes of (n+1) / nodes to repair the failed nodes.
  • HSRC codes require computational polynomials to be relatively complex.
  • the coding modules are not subdividable, so the repair coding modules must also be inseparable; in addition, in order to reproduce a specific storage Node, once a node is randomly selected as a help node, and for the HSRC code, There is only one node left to choose from.
  • the technical problem to be solved by the present invention is to provide a computing operation with a relatively simple operation and a small overhead for distributed network storage in view of the above-mentioned problems of repairing data or reconstructing data in the prior art.
  • Self-healing code encoding, reconstruction and recovery methods are provided.
  • a generator element which is a multiplicative group of the first finite field
  • step B) further includes the following steps:
  • the elements in the second finite field F m are represented as m-tuples. Further, the division of the second finite field multiplicative group / represents the second finite field F as a form of multiplication of elements of the first finite field multiplicative group / and the second finite field multiplicative group / ⁇ .
  • step C) further includes:
  • C2 were selected coding vector t + 1 th sub-space as the corresponding storage node of each of the elements in a sub-space.
  • step D) further includes:
  • each storage node stores t+1 stored coded data blocks determined by t+1 coding modules.
  • the present invention also relates to a method of reconstructing data in a storage system employing the self-healing code encoding method described above, comprising the steps of:
  • step J) further comprises: respectively obtaining, by the server, a coding vector of the selected storage node or obtaining the coding vector by the selected storage node.
  • the present invention also relates to a method for repairing a failed storage node in a storage system employing the self-healing code encoding method described above, comprising the steps of:
  • the coding vector of the relevant node is obtained by performing operation on the coding vector of the failed storage node and the selected storage node, and then the related node is found; or.
  • the data stored by the failed storage node is obtained by reorganizing data stored by the selected storage node and the associated storage node.
  • the method for encoding, reconstructing and restoring the self-repairing code for distributed network storage of the present invention has the following beneficial effects: the second finite field obtained according to the number of stored file encoding modules is divided into a plurality of subspaces, and Each subspace corresponds to a storage node, and determines the location of the encoded data module stored by the storage node.
  • the failed node is repaired, only one storage node is selected, and the storage node corresponding to the selected storage node is found, and the storage is downloaded.
  • the data of the node is reorganized to obtain the data stored by the failed storage node. Therefore, the calculation is relatively simple and the overhead is small.
  • FIG. 1 is a schematic diagram of data reconstruction of an EC code in the prior art
  • FIG. 2 is a schematic diagram of data repair of an EC code in the prior art
  • FIG. 3 is a schematic diagram of data reconstruction of an RGC code in the prior art
  • FIG. 4 is a coding flow chart of an embodiment of a method for encoding, reconstructing, and recovering a self-healing code for distributed network storage according to the present invention
  • Figure 5 is a flow chart of a method for data reconstruction in the embodiment
  • FIG. 6 is a flow chart of a method for data repair in the embodiment
  • Figure 7 is a graphical illustration of the comparison of the static restoring force encoded in the embodiment and the static restoring force of the EC code. detailed description
  • the encoding process includes the following steps:
  • Step S41 setting a basic finite field having an inclusion relationship, a first finite field, and a second finite field: In this step, first setting a basic finite field of q order, and then dividing the projective space of the basic finite field into t Dimension space, that is, do t-stretch; then get a first finite field whose order is 1 .
  • q is the power of a prime p
  • m-dimensional vector on the finite field is represented as PG ( m-1 , q ), which is called the projective space.
  • the storage file to be stored in each storage node is composed of a plurality of encoded data modules, and Q ⁇ 7 ⁇ is actually the number of encoded data modules included in the storage file.
  • the vectors in this implementation are all row vectors.
  • Projective space is the most unique type of geometric object in algebraic geometry. It is defined as: in the n-dimensional affine space k n on the field k, the set of all the straight lines of the origin is called the projective space on the field k. .
  • the field k can take the complex field and so on. From a basic mathematical concept, a coordinate system corresponds to an affine space (Affine Space). When a vector is transformed from one coordinate system to another, linear transformation is performed.
  • P the projective space
  • the t-stretch of the projective space P is the t-dimensional subspace of the projective space P
  • the set of t-dimensional subspaces is S, which divides the projective space P into several t
  • each point in the projective space P belongs to only one t-dimensional subspace in the set S.
  • the relationship between finite fields is
  • Step S42 divides the second finite field into a plurality of subspaces in a coset manner:
  • the second finite field F2 is divided into a plurality of subspaces in a coset manner.
  • the finite field Fl is the (t+1)-dimensional subspace of the space V, that is, the t-dimensional shadow space of the projective space P.
  • the coset in the finite field is a special case of the projective space.
  • the coset is ⁇ 1, 3 2 (ie, a is the element of the second finite field F2 ), the coset divides the multiplicative group in the second finite field F2 into parts. This constitutes a t-extension of the space P.
  • the step specifically includes: obtaining a first finite field multiplicative group F +1 , let V be a generator of the first finite field multiplicative group F +1 ; obtaining a second finite field multiplicative group /, and let w be Generator of the second finite field multiplicative group; using cosets
  • Step S43 Obtain a basic vector of each subspace, and select t+1 of the coding vectors as the storage nodes corresponding to the subspace:
  • this step respectively obtain the basics of each of the subspaces (ie, one of the cosets) A vector, and in which a linearly independent t+1 basic vector is selected as the coding vector of the storage node.
  • the one subspace corresponds to a storage node, and the basic vector selected by the subspace is used as the coding vector of the storage node.
  • this step comprises: each subspace separately acquire the element 1; t + 1 respectively take any storage node as a sub-space corresponding to the m elements in each of the subspace Coding vector.
  • Step S44 Obtain an encoding module of the file according to the encoding vector of each storage node and store:
  • the encoding data module of each storage node obtained according to the above steps obtains the corresponding encoded data module, and is stored on the storage node. .
  • the storage file coded data blocks corresponding to the positions of the elements having the element of 1 in each coding vector are sequentially obtained and added as the storage determined by the coding vector.
  • the coded data block is stored in the storage node; each storage node stores t+1 stored coded data blocks determined by t+1 coding modules.
  • the second finite field F2 can be directly decomposed into several basic finite field F0 addition forms:
  • the elements in the field F2 can be written as a 4-tuple.
  • the size of the file is B, and the file B needs to be stored in n storage nodes.
  • the storage size of each storage node is ", when there is a storage node failure, the remaining connections need to be connected (n -1) d of the storage nodes and download data from each of the d nodes, represented by PSRC(n, k) as a projective self-repair code, where parameters n and k are parameters in the stretch of the construct.
  • VF; ⁇ (0100), (0011), (0111 ) ⁇
  • V 2 F: ⁇ (0010), ( 1101 ), ( 1111 ) ⁇
  • the data stored by each storage node is as follows: Node base vector stores data
  • the present embodiment also relates to a method of reconstructing data from a storage module obtained by storing the above method. Including the following steps:
  • Step S51 Select k among n storage nodes: In this step, k are randomly selected from n storage nodes storing stored file encoded data, where k ⁇ mt + V) , where m and t It has the same meaning as in the encoding step described above.
  • Step S52 Download the data in the selected storage node and reconstruct the data:
  • the data of the selected storage node is separately downloaded and the storage file is reconstructed according to the coding vectors of the storage nodes.
  • the server obtains the code vector of the selected storage node.
  • the code vector can also be obtained by the selected storage node.
  • Step S53 Is the reconstruction completed? It is judged whether the file reconstruction is completed, that is, whether the file is reconstructed, and if so, step S54 is executed to exit the file data reconstruction; otherwise, the process goes to step S55.
  • Step S54 Exit this data reconstruction: In this step, the stored file has been obtained and exited.
  • Step S55 Select one of the unselected storage nodes: In this step, since the data downloaded by the selected storage node does not reconstruct the file data, one of the unselected storage nodes is selected, so that The number of selected storage nodes is increased by one, and the flow jumps to step S52.
  • the vector stored by the two storage nodes N and N is arbitrarily selected as (Vi, . . . , v a ), and the vector stored by the node N′ is ( Ul , ... , M J . It is assumed that there is a vector in the storage node N V , vector V is linearly related to some vectors in node N', that is, V can be written as:
  • the method further relates to a data recovery method for recovering the code obtained by the foregoing method, including the following Steps:
  • Step S61 Confirming that the storage node is invalid and obtaining the coding vector of the storage node: In this step, it is confirmed that one storage node has failed, and the stored data needs to be repaired and stored on another storage node; meanwhile, obtained by the server The encoding vector of the storage node.
  • Step S62 selects an un-failed storage node and obtains its coding vector: arbitrarily selects a node among the non-failed storage nodes, and obtains the coding vector of the storage node from the server.
  • Step S63 Finding a storage node associated with the selected storage node: In this step, performing at least one storage node related to the selected storage node by performing operation on the coded vector of the failed storage node and the selected storage node The node coding vector, and then the storage node corresponding to the coding vector is found on the server; in this step, the operation taken is an exclusive OR operation.
  • Step S64 downloading the selected storage node and its associated storage node data, and obtaining data stored by the failed node and saving: In this step, downloading data stored by the selected storage node and its associated storage node, and according to the data Corresponding coding vector (including the coding vector of the failed storage node, selecting the coding vector of the storage node and the coding vector of the above-mentioned storage node), reorganizing the data, obtaining the data stored by the failed node, and storing it on a new storage node .
  • the data Corresponding coding vector including the coding vector of the failed storage node, selecting the coding vector of the storage node and the coding vector of the above-mentioned storage node
  • each storage node stores "the amount of encoded data.
  • at least one storage node Nj can recover the data stored by the failed node by downloading the data of the storage node and Nj.
  • a new node replaces the failed node Ni.
  • the new node selects any storage node.
  • the data stored by the N l storage node is ⁇ .
  • V ' F 2 U VF ; can repair the data stored in the node Ni.
  • the self-repair capability of the PSRC code is stronger than the self-repair capability of the HSRC code.
  • the new node can repair the data stored by the failed node (N 4 , N 12 ), (N 4 , N 10 ) by connecting and downloading any of the following three pairs of nodes (the three pairs of nodes all contain the node N 4 ), ( N 4 , N 5 ).
  • static resilience refers to the probability that once the data is stored in the system, the stored original data can be recovered without further repairing the failed node.
  • p n be .
  • De is the effective probability of any given node. Since there are no two different data modules stored in the same node in the system, we can assume that the validity of the module stored by any node is p n . De .
  • Figure 7 compares the probability of static restoring forces for PSRC (21, 3) codes and MDS (21, 3) codes.
  • the values in the figure are calculated by the computer by evaluating the value of ⁇ . From this figure we can see that the MDS code may not have any (n, k) characteristics. More importantly, although the PSRC (21, 3) code loses a bit of static resilience, it has more self-healing capabilities than the MDS code. It is not to be understood as limiting the scope of the invention. It should be noted that a number of variations and modifications may be made by those skilled in the art without departing from the spirit and scope of the invention. Therefore, the scope of the invention should be determined by the appended claims.

Landscapes

  • Physics & Mathematics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

La présente invention se rapporte à un procédé de codage utilisé pour exécuter un code de réparation automatique enregistré par un réseau distribué. Le procédé selon l'invention comprend les étapes suivantes consistant : à définir un champ fini de base Fq, et à obtenir un premier champ fini qui correspond à l'équation (I) ; à obtenir un second champ fini qui correspond à l'équation (II), Fq⊆Equation (I)⊆Equation (II) ; à diviser, sous la forme d'une classe latérale qui correspond à l'équation (III), un espace représenté par le second champ fini qui correspond à l'équation (II), de sorte à obtenir des sous-espaces avec le nombre qui correspond à l'équation (IV) ; à sélectionner respectivement t+1 vecteurs de base parmi des vecteurs de base qui sont représentés par des éléments du champ fini de base de chacun des sous-espaces devant être utilisés en tant que des vecteurs de codage d'un nœud de stockage, le nœud de stockage correspondant au sous-espace, et le vecteur de codage correspondant à une position d'un bloc de données de codage dans un fichier de stockage ; et à obtenir le bloc de données de codage à la position correspondante dans le fichier de stockage sur la base du vecteur de codage de chaque nœud de stockage ; enfin, à enregistrer le bloc de données de codage dans le nœud de stockage. La présente invention se rapporte d'autre part à des procédés adaptés pour reconstruire des données et pour retrouver des données dans un système de stockage dans lequel le procédé de codage susmentionné est mis en œuvre. Les procédés de codage, de reconstruction et de récupération utilisés pour exécuter un code de réparation automatique enregistré par un réseau distribué, selon l'invention, offrent les avantages suivants : leur fonctionnement est relativement simple, et la charge de données utiles est relativement faible.
PCT/CN2012/078927 2012-07-20 2012-07-20 Procédés de codage, de reconstruction et de récupération utilisés pour exécuter un code de réparation automatique enregistré par un réseau distribué WO2014012246A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2012/078927 WO2014012246A1 (fr) 2012-07-20 2012-07-20 Procédés de codage, de reconstruction et de récupération utilisés pour exécuter un code de réparation automatique enregistré par un réseau distribué
CN201280074817.2A CN104782101B (zh) 2012-07-20 2012-07-20 用于分布式网络存储的自修复码的编码、重构和恢复方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2012/078927 WO2014012246A1 (fr) 2012-07-20 2012-07-20 Procédés de codage, de reconstruction et de récupération utilisés pour exécuter un code de réparation automatique enregistré par un réseau distribué

Publications (1)

Publication Number Publication Date
WO2014012246A1 true WO2014012246A1 (fr) 2014-01-23

Family

ID=49948183

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2012/078927 WO2014012246A1 (fr) 2012-07-20 2012-07-20 Procédés de codage, de reconstruction et de récupération utilisés pour exécuter un code de réparation automatique enregistré par un réseau distribué

Country Status (2)

Country Link
CN (1) CN104782101B (fr)
WO (1) WO2014012246A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017041233A1 (fr) * 2015-09-08 2017-03-16 广东超算数据安全技术有限公司 Procédé de réparation de nœud de codage et de stockage pour un code de régénération de réparation fonctionnelle
CN108182235A (zh) * 2017-12-27 2018-06-19 北京奇虎科技有限公司 一种用于对用户特征进行分布式编码的方法和系统
CN109038575A (zh) * 2018-09-05 2018-12-18 东北大学 基于改进物种生灭算法的含分布式电源配电网重构方法

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105681425B (zh) * 2016-01-22 2019-01-22 广东顺德中山大学卡内基梅隆大学国际联合研究院 基于分布式存储系统的多节点修复方法及其系统
CN113505021B (zh) * 2021-05-26 2023-07-18 南京大学 基于多主节点主从分布式架构的容错方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101415016A (zh) * 2007-10-17 2009-04-22 深圳市亚贝电气技术有限公司 一种数据复制方法、系统和存储服务器
CN100579016C (zh) * 2006-01-24 2010-01-06 华为技术有限公司 一种网络数据的分布式存储下载系统、设备及方法
CN101923558A (zh) * 2009-05-20 2010-12-22 中国科学院声学研究所 基于(d,k)摩尔图的存储网络结构及数据读写方法
CN102279777A (zh) * 2011-08-18 2011-12-14 成都市华为赛门铁克科技有限公司 数据冗余处理方法、装置和分布式存储系统

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8631269B2 (en) * 2010-05-21 2014-01-14 Indian Institute Of Science Methods and system for replacing a failed node in a distributed storage network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100579016C (zh) * 2006-01-24 2010-01-06 华为技术有限公司 一种网络数据的分布式存储下载系统、设备及方法
CN101415016A (zh) * 2007-10-17 2009-04-22 深圳市亚贝电气技术有限公司 一种数据复制方法、系统和存储服务器
CN101923558A (zh) * 2009-05-20 2010-12-22 中国科学院声学研究所 基于(d,k)摩尔图的存储网络结构及数据读写方法
CN102279777A (zh) * 2011-08-18 2011-12-14 成都市华为赛门铁克科技有限公司 数据冗余处理方法、装置和分布式存储系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
WANG, YU: "Research on Data Redundancy and Maintenance Technology in Distributed Storage System.", DOCTORAL DISSERTATION OF SOUTH CHINA UNIVERSITY OF TECHNOLOGY., 4 May 2012 (2012-05-04) *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017041233A1 (fr) * 2015-09-08 2017-03-16 广东超算数据安全技术有限公司 Procédé de réparation de nœud de codage et de stockage pour un code de régénération de réparation fonctionnelle
CN108182235A (zh) * 2017-12-27 2018-06-19 北京奇虎科技有限公司 一种用于对用户特征进行分布式编码的方法和系统
CN109038575A (zh) * 2018-09-05 2018-12-18 东北大学 基于改进物种生灭算法的含分布式电源配电网重构方法

Also Published As

Publication number Publication date
CN104782101A (zh) 2015-07-15
CN104782101B (zh) 2018-02-13

Similar Documents

Publication Publication Date Title
Oggier et al. Self-repairing homomorphic codes for distributed storage systems
US9961142B2 (en) Data storage method, device and distributed network storage system
US9647698B2 (en) Method for encoding MSR (minimum-storage regenerating) codes and repairing storage nodes
Dimakis et al. A survey on network codes for distributed storage
Olmez et al. Fractional repetition codes with flexible repair from combinatorial designs
Cadambe et al. Optimal repair of MDS codes in distributed storage via subspace interference alignment
Sung et al. A ZigZag-decodable code with the MDS property for distributed storage systems
US20140152476A1 (en) Data encoding methods, data decoding methods, data reconstruction methods, data encoding devices, data decoding devices, and data reconstruction devices
Oggier et al. Byzantine fault tolerance of regenerating codes
WO2014012246A1 (fr) Procédés de codage, de reconstruction et de récupération utilisés pour exécuter un code de réparation automatique enregistré par un réseau distribué
Shahabinejad et al. A class of binary locally repairable codes
Li et al. Beyond the MDS bound in distributed cloud storage
Chiu et al. A new diskless checkpointing approach for multiple processor failures
Hollmann Storage codes—Coding rate and repair locality
WO2015180038A1 (fr) Procédé et dispositif de construction de code de réplique partiel, et procédé de récupération de données correspondant
Mahdaviani et al. Product matrix MSR codes with bandwidth adaptive exact repair
WO2013159341A1 (fr) Procédé de codage, de décodage et de réparation de données reposant sur un code d'auto-réparation homomorphique et système de stockage associé
WO2014059651A1 (fr) Procédé de codage, de restructuration de données et de correction de codes auto-correcteurs projectifs
Mahdaviani et al. Bandwidth adaptive & error resilient MBR exact repair regenerating codes
WO2017041232A1 (fr) Cadre de codage et de décodage pour code cyclique binaire
WO2017041233A1 (fr) Procédé de réparation de nœud de codage et de stockage pour un code de régénération de réparation fonctionnelle
Chen et al. A new Zigzag MDS code with optimal encoding and efficient decoding
CN108199720B (zh) 一种减小存储开销和提高修复效率的节点修复方法及系统
Mahdaviani et al. Bandwidth adaptive & error resilient regenerating codes with minimum repair bandwidth
Calis et al. Architecture-aware coding for distributed storage: Repairable block failure resilient codes

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12881204

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12881204

Country of ref document: EP

Kind code of ref document: A1