CN109828723A - A kind of distributed memory system and its precise information restorative procedure and device - Google Patents
A kind of distributed memory system and its precise information restorative procedure and device Download PDFInfo
- Publication number
- CN109828723A CN109828723A CN201910113215.7A CN201910113215A CN109828723A CN 109828723 A CN109828723 A CN 109828723A CN 201910113215 A CN201910113215 A CN 201910113215A CN 109828723 A CN109828723 A CN 109828723A
- Authority
- CN
- China
- Prior art keywords
- node
- memory node
- parent
- data
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
Present embodiment discloses a kind of distributed memory system and its precise information restorative procedures and device, it is combined using regeneration code and copy mode, when the failure of the parent memory node described in the distributed memory system, data identical with fail data are obtained from coupled parent memory node and/or filial generation memory node, carry out precise information reparation.The computation complexity for not only reducing the reparation of distributed memory system precise information repairs node convenient and efficient, and improves the number for itself tolerating node failure, that is, improves reliability and safety.
Description
Technical field
The disclosure belongs to the technical field of distributed storage, is related to a kind of distributed memory system and its precise information reparation
Method and device.
Background technique
Only there is provided background technical informations relevant to the disclosure for the statement of this part, it is not necessary to so constitute first skill
Art.
Currently, with the sharp increase of mass storage data, traditional document storage system is unable to satisfy high capacity, highly reliable
The demand of property, high-performance etc., distributed memory system are gradually used widely, but in distributed memory system
At this moment the increase of memory node number will repair node data to guarantee data so that the probability of node failure increases
Integrality.The type of data reparation is divided into the accurate reparation three types of accurate reparation, functional reparation and components of system as directed,
In the characteristics of accurately repairing type be that data before data and failure after repairing are just the same, so the most practical reliable.
In data reparation, duplication is simplest redundancy structure, is exactly that former data are replicated multiple copies, although this
Method maintains the reliability of system, but since amount of redundant information is too big, causes cost prohibitive；It is M that correcting and eleting codes, which are by size,
Initial data be divided into k block, then these k blocks are encoded again, generate n encoding block (n > k), each piece of size is M/
k.Therefore, k sub-block in n encoding block is arbitrarily taken, it will be able to restore initial data.Compared to duplication, the use of correcting and eleting codes
Redundant storage amount can be effectively reduced while guaranteeing reliability.After network code theory introduces distributed memory system,
Dimakis etc. proposes the concept of regeneration code, and in regeneration code theory, single storage node not only has storage and forwarding function
Can, but also can be carried out operation.From 2010, scholars proposed MBR (the Minimum Bandwidth of different properties
Regenerating) code, MSR (Minimum Storage Regenerating) code.Data reparation based on correcting and eleting codes is compared
It is small in replica code storage overhead but biggish computation complexity can be generated simultaneously, so Rouayheb et al. proposes part duplication
The concept of code is widely studied because it is able to achieve the advantage of the characteristic and low complex degree accurately repaired.
The design and realization of part replica code are based on chart, and the data symbol of each node storage is to design
, so being repaired when carrying out node reparation using stationary nodes, and code is commonly regenerated when repairing node
It is to randomly select d effectively nodes in remaining node to be repaired.Because part replica code belongs to no cataloged procedure, institute
Low with complexity, the reparation bandwidth in repair process is low, is easy to implement, and combines different charts, and part replica code is various
Various kinds, the characteristic shown also have difference.Rouyheb et al., which is proposed, utilizes the methods of complete graph construction portion replica code, institute
The coding of construction simple and fast, computation complexity 0 when node is repaired, while method of the required amount of storage than directly replicating
It is small, but its tolerance very little, i.e., patient failure node number it is seldom.
Summary of the invention
For the deficiencies in the prior art, one or more other embodiments of the present disclosure provide a kind of distributed storage
System and its precise information restorative procedure and device are combined using regeneration code and copy mode, not only reduce distributed storage
The computation complexity of system precise information reparation repairs node convenient and efficient, and improves tolerance node failure itself
Number improves reliability and safety.
According to the one aspect of one or more other embodiments of the present disclosure, a kind of distributed memory system is provided.
A kind of distributed memory system, the system include: parent memory node set and filial generation memory node set；
The parent memory node set includes several sequentially connected parent memory nodes, for storing initial data；
The filial generation memory node set includes several child nodes, and each parent memory node is connect with two child nodes, for certainly
The copy of body data content stores；
When the parent memory node fails, obtained from coupled parent memory node and/or filial generation memory node
Data identical with fail data carry out precise information reparation.
Further, the initial data in the distributed memory system constitutes a part according to graphtheoretic concept and replicates
Code, the part replica code are the set of data element set and several memory nodes several different, and the part is multiple
The set of several memory nodes is the parent memory node set in code processed.
Further, the storing data size of the parent memory node is the filial generation storage node storing data size
Twice；The parent memory node number is the half of the filial generation storage node number.
Further, data downloading cost is identical between the parent memory node, number between the filial generation memory node
Identical according to downloading cost, data are downloaded between data downloading cost and the filial generation memory node between the parent memory node
Cost is identical；
Data downloading cost is different between the parent memory node and the filial generation memory node；
Data download time is greater than parent storage section between the parent memory node and the filial generation memory node
Data download time between point.
According to the one aspect of one or more other embodiments of the present disclosure, the perfect number based on distributed memory system is provided
According to restorative procedure.
A kind of precise information restorative procedure based on distributed memory system, this method are deposited based on a kind of distribution
Storage system, comprising:
When the parent memory node fails, the failure node number and memory node sum of parent memory node are judged
The size of half；
When the failure node of parent memory node is less than the half of memory node sum, stored from coupled parent
Node and/or filial generation memory node obtain data identical with fail data, carry out precise information reparation；
When the failure node of parent memory node is not less than the half of memory node sum, deposited from coupled filial generation
It stores up node and obtains data identical with fail data, carry out precise information reparation.
Further, in the method, when the failure node of parent memory node is less than the half of memory node sum
Failure node including a parent memory node failure, parent memory node is less than the half and failure section of memory node sum
Non-conterminous and parent memory node failure node is less than the half of memory node sum between point but there are at least two failures
Three kinds of adjacent situations between node.
Further, in the method, when parent memory node fails, distributed memory system is from failure node
Adjacent parent memory node obtains data identical with fail data, carries out precise information reparation.
Further, in the method, when the failure node of parent memory node be less than memory node sum half and
When non-conterminous between failure node, all failure nodes of distributed memory system from its adjacent parent memory node obtain with
The identical data of fail data carry out precise information reparation.
Further, in the method, when the failure node with parent memory node is less than the half of memory node sum
But there are it is adjacent between at least two failure nodes when, the precise information of failure cannot be obtained from adjacent parent memory node
Failure node obtains data identical with fail data, the adjacent father that remaining failure node never fails from filial generation memory node
Data identical with fail data are obtained for memory node, carry out precise information reparation.
Further, in the method, when the failure node of parent memory node is not less than the half of memory node sum
When, the failure node of all parent memory nodes obtains data identical with fail data from the filial generation memory node of reply, when
When whole parent memory nodes fail, directlys adopt filial generation memory node and be combined into corresponding parent memory node.
According to the one aspect of one or more other embodiments of the present disclosure, a kind of computer readable storage medium is provided.
A kind of computer readable storage medium, wherein being stored with a plurality of instruction, described instruction is suitable for by terminal device
Reason device loads and executes a kind of precise information restorative procedure based on distributed memory system.
According to the one aspect of one or more other embodiments of the present disclosure, a kind of terminal device is provided.
A kind of terminal device comprising processor and computer readable storage medium, processor is for realizing each instruction；Meter
Calculation machine readable storage medium storing program for executing is suitable for being loaded by processor and being executed described one kind and is based on for storing a plurality of instruction, described instruction
The precise information restorative procedure of distributed memory system.
The disclosure the utility model has the advantages that
A kind of distributed memory system and its precise information restorative procedure described in the present embodiment and device, part duplication are compiled
The tolerance of code is high, it can the failure node number of receiving is high, and can accurately repair initial data completely, reliability and peace
Quan Xinggao；
A kind of distributed memory system and its precise information restorative procedure described in the present embodiment and device, part duplication are compiled
The repair time of code is very fast, and without coding and decoding, convenient and efficient, computation complexity is low, is easy to implement.
Detailed description of the invention
The accompanying drawings constituting a part of this application is used to provide further understanding of the present application, and the application's shows
Meaning property embodiment and its explanation are not constituted an undue limitation on the present application for explaining the application.
Fig. 1 is the construction flow chart according to the part replica code of one or more embodiments；
Fig. 2 is that node selection flow chart is repaired according to the data of one or more embodiments；
Fig. 3 is according to the part replica code of one or more embodiments and the repair time of PM-MBR code and failure node
Several relational graphs；
Fig. 4 is between tolerance ρ and memory node n under the coding according to the three types of one or more embodiments
Relational graph；
Fig. 5 is according between the reparation bandwidth β and failure node m under the three types of one or more embodiments coding
Relational graph；
Fig. 6 is according between the total memory capacity S and memory node n under the three types of one or more embodiments coding
Relational graph.
Specific embodiment:
Below in conjunction with the attached drawing in one or more other embodiments of the present disclosure, to one or more other embodiments of the present disclosure
In technical solution be clearly and completely described, it is clear that described embodiment be only the present embodiment a part implement
Example, instead of all the embodiments.Based on one or more other embodiments of the present disclosure, those of ordinary skill in the art are not being done
Every other embodiment obtained under the premise of creative work out belongs to the range of the present embodiment protection.
It is noted that following detailed description is all illustrative, it is intended to provide further instruction to the application.Unless another
It indicates, all technical and scientific terms that the present embodiment uses have and the application person of an ordinary skill in the technical field
Normally understood identical meanings.
It should be noted that term used herein above is merely to describe specific embodiment, and be not intended to restricted root
According to the illustrative embodiments of the application.As used herein, unless the context clearly indicates otherwise, otherwise singular
Also it is intended to include plural form, additionally, it should be understood that, when in the present specification using term "comprising" and/or " packet
Include " when, indicate existing characteristics, step, operation, device, component and/or their combination.
It should be noted that flowcharts and block diagrams in the drawings show according to various embodiments of the present disclosure method and
The architecture, function and operation in the cards of system.It should be noted that each box in flowchart or block diagram can represent
A part of one module, program segment or code, a part of the module, program segment or code may include one or more
A executable instruction for realizing the logic function of defined in each embodiment.It should also be noted that some alternately
Realization in, function marked in the box can also occur according to the sequence that is marked in attached drawing is different from.For example, two connect
The box even indicated can actually be basically executed in parallel or they can also be executed in a reverse order sometimes,
This depends on related function.It should also be noted that each box and flow chart in flowchart and or block diagram
And/or the combination of the box in block diagram, the dedicated hardware based system that functions or operations as defined in executing can be used are come
It realizes, or the combination of specialized hardware and computer instruction can be used to realize.
In the absence of conflict, the feature in the embodiment and embodiment in the disclosure can be combined with each other, and tie below
It closes attached drawing and embodiment is described further the disclosure.
According to the one aspect of one or more other embodiments of the present disclosure, a kind of distributed memory system is provided.Using again
Raw code and copy mode combine, and not only reduce computation complexity, repair node convenient and efficient, and improve itself and tolerate
The number of node failure, that is, improve reliability and safety.
A kind of distributed memory system, the system include: parent memory node set and filial generation memory node set；
The parent memory node set includes several sequentially connected parent memory nodes, for storing initial data；
The filial generation memory node set includes several child nodes, and each parent memory node is connect with two child nodes, for certainly
The copy of body data content stores；
When the parent memory node fails, obtained from coupled parent memory node and/or filial generation memory node
Data identical with fail data carry out precise information reparation.
In the present embodiment, the memory node in distributed memory system is divided into parent memory node set and filial generation is deposited
Store up node set, wherein initial data is stored in parent memory node, filial generation memory node set is as parent memory node
Copy set, meet according to the principle of accurate repair data, when node failure obtained from other node it is identical as fail data
Data, make the complete Exact recovery of failure node, on this basis, select that computation complexity is low, repair time is fast, repairs bandwidth
Low node recovery scenario completes reconstruct.
It should be noted that the coding in distributed memory system is repaired node set by the present embodiment is divided into two class units
Set, every one kind has different data downloading costs and of a sort node has identical data downloading cost.
Further, the initial data in the distributed memory system constitutes a part according to graphtheoretic concept and replicates
Code, the part replica code are the set of data element set and several memory nodes several different, and the part is multiple
The set of several memory nodes is the parent memory node set in code processed.
In the present embodiment, by distributed memory system (n1, k, d, θ, α) size be M initial data according to graph theory it is general
It reads and constitutes a part replica codeWherein WhereinIt is θ
A different data element set, V n1The set of a memory node, referred to as parent memory node set, each storage section
The storing data size of point is α, and the duplication degree of each data element is ρ1, i.e.,Each of element all appear precisely at ρ1
In a different memory node.
This part replica codeThe essential structure condition of satisfaction are as follows:
Further, the storing data size of the parent memory node is the filial generation storage node storing data size
Twice；The parent memory node number is the half of the filial generation storage node number.
In the present embodiment, existConstruction on the basis of, n1Each memory node in a memory node
The copy that 2 child nodes are added as data content stores, and node set is referred to as child node set, each child node
α/2 storing data is stored, data can also be transmitted between child node, if the number of child node is n2, memory node storing data
Size be α2, then meet the essential structure condition of child node satisfaction are as follows:
n2=2n1, α2=α/2
Further, data downloading cost is identical between the parent memory node, number between the filial generation memory node
Identical according to downloading cost, data are downloaded between data downloading cost and the filial generation memory node between the parent memory node
Cost is identical；
Data downloading cost is different between the parent memory node and the filial generation memory node；
Data download time is greater than parent storage section between the parent memory node and the filial generation memory node
Data download time between point.
In the present embodiment, the downloading data size between parent memory node, between filial generation memory node is β1, downloading
Data time is t1(including the Data Detection timeData transmission period), the downloading number between parent and filial generation memory node
It is β according to size2, the downloading data time is t2(including the Data Detection timeData transmission period), t2>t1。
It is combined into when the parent failure node of distributed memory system integrates as L={ l1, l2..., lm, 1≤m≤n1, one is shared
M failure node, then node repairs repair time T used are as follows:
It can be obtained according to gained repair time T, when the failure node of parent memory node is less than the one of memory node sum
When half, the mode for selecting the reparation of parent memory node to combine with filial generation memory node reparation, and work as the mistake of parent memory node
It is then the data repair mode that the reparation of filial generation memory node is only selected in selection when imitating half of the node greater than memory node sum.
As described in the background art, the recoverability of distributed memory system for system itself data safety extremely
Close it is important, meanwhile, for the ease of practice, the memory capacity of system and repair that bandwidth is the smaller the better, and the shorter repair time the better, portion
Divide replica code to be not necessarily to coding and decoding as one kind, conveniently regenerate code, is concerned.But the part duplication constructed
Code tolerance is very low, i.e., recoverable failure node is less.
A kind of utilize, which is proposed, for the low problem the present embodiment of this part of replica code tolerance regenerates code and duplication side
The part replica code that formula combines obtains a kind of method based on the accurate repair data in distributed memory system, not only drops
Low computation complexity repairs node convenient and efficient, and improves the number for itself tolerating node failure, that is, improves reliable
Property and safety.
According to the one aspect of one or more other embodiments of the present disclosure, the perfect number based on distributed memory system is provided
According to restorative procedure.
A kind of precise information restorative procedure based on distributed memory system, this method are deposited based on a kind of distribution
Storage system, comprising:
When the parent memory node fails, the failure node number and memory node sum of parent memory node are judged
The size of half；
When the failure node of parent memory node is less than the half of memory node sum, stored from coupled parent
Node and/or filial generation memory node obtain data identical with fail data, carry out precise information reparation；
When the failure node of parent memory node is not less than the half of memory node sum, deposited from coupled filial generation
It stores up node and obtains data identical with fail data, carry out precise information reparation.
In the present embodiment, the technical solution of use, comprising:
Construction portion replica code first, as shown in Figure 1,
Step 1: the memory node in distributed memory system is divided into parent memory node setWith filial generation memory node setBetween parent, filial generation
Between repair time with repair bandwidth it is identical (reparations bandwidth be β1, repair time t1), so-called repair time, including be
Whether the data of system detection memory node storage fail the required timeAnd among the nodes transmit data needed for the timeBut the repair time carried out data transmission between parent and filial generation and reparation bandwidth are then different, and reparation bandwidth is β2, when reparation
Between be t2(including the Data Detection timeData transmission period), t2>t1。
Step 2: to n is put into after the initial data coding that size is M1A memory node, this n1A memory node is by annular
Topology distribution constitutes a foundation replica codeWhereinIt is θ different data element set, V n1
The set of a memory node, is expressed asReferred to as parent memory node, each memory node
Memory capacity be α, the duplication degree of each data element is ρ1, meet essential structure condition:That is both members
All indicate the total amount of data of parent memory node storage.
Step 3: introducing two filial generation memory nodes for each parent memory node, parent storage section is stored respectively
The storing data of point half, comparison introduce the method that a filial generation memory node stores whole copies, and The method avoids sons
For reparation problem caused by memory node failure, the number of filial generation memory node is n2, the size of memory node storing data
For α2, meetα2The structural environment of=α/2.In addition, can carry out data transmission between setting filial generation memory node,
So filial generation memory node can also be used to repair other filial generation memory nodes.
Step 4: the part replica code for defining the present embodiment is IFR, total repair time is T, and total reparation bandwidth is β,
We will design a set of node selection scheme for the data repair process of IFR code below:
Data as shown in Figure 2 repair node and select flow chart.In the method, when the failure node of parent memory node
Less than memory node sum half when include the failure of parent memory node, parent memory node failure node be less than and deposit
It stores up non-conterminous between the half and failure node of node total number and parent memory node failure node and is less than memory node sum
Half but there are three kinds of situations adjacent between at least two failure nodes.
(1) when a parent memory node fails, distributed memory system stores section from the adjacent parent of failure node
Point obtains data identical with fail data, carries out precise information reparation.
(2) when non-conterminous between the half and failure node that the failure node of parent memory node is less than memory node sum
When, all failure nodes of distributed memory system obtain number identical with fail data from its adjacent parent memory node
According to progress precise information reparation.
(3) when the failure node with parent memory node is less than the half of memory node sum but there are at least two failures
When adjacent between node, the failure node for the precise information that failure cannot be obtained from adjacent parent memory node is stored from filial generation
Node obtains data identical with fail data, and the adjacent parent memory node that remaining failure node never fails is obtained and lost
The identical data of data are imitated, precise information reparation is carried out.
In the present embodiment, when the failure node of parent memory node is less than the half of memory node sum, father is selected
For the mode that memory node reparation is combined with filial generation memory node reparation,
When parent memory node fails, father that distributed memory system can easily be connected from failure node
The precise information that failure is obtained for memory node, to accurately repair successfully, repair time t1, reparation bandwidth is β=α；
When non-conterminous between m parent memory node failure but node, the storing data of all failures all can be from failure
The connected parent memory node of node acquires fail data, to complete the accurate reparation of all failure memory nodes, repairs
The multiple time is mt1, total bandwidth of repairing is β=m α.
When the failure of m parent memory node but there are it is adjacent between w node when, i.e., if the adjacent father of failure node
The case where also failing for memory node, at this moment the partial data of failure node cannot be obtained from the adjacent parent memory node of failure
, so this partial data will be obtained from filial generation memory node, but remaining fail data can never fail it is adjacent
Parent memory node obtains, so the repair process of such case is:
(1) whether the adjacent parent memory node of system detection failure node is effective, first obtains the storage of efficient neighbor parent
The repair data of node-node transmission, repair time are (m+w) t1,
(2) rear send to corresponding filial generation memory node is requested, needed for two filial generation memory node difference simultaneous transmissions
Repair data, repair time wt2, failure node reparation completion, total repair time is (m+w) t1+wt2, total reparation band
Width is m α.
Step 6: in the method, when the failure node of parent memory node is not less than the half of memory node sum,
The failure node of all parent memory nodes obtains data identical with fail data from the filial generation memory node of reply, works as whole
When parent memory node fails, directlys adopt filial generation memory node and be combined into corresponding parent memory node.
In the present embodiment, when parent memory node failure number is more than the half of total parent memory node number,
We set all fail datas and are accurately repaired with corresponding filial generation memory node, and total repair time is mt2, always
Reparation bandwidth is m α,
But when whole parent memory nodes failures, it is higher at this moment to repair whole failure node costs, to directly adopt
Corresponding parent memory node is combined into filial generation memory node.
Step 7: preferentially accurately repaired by corresponding parent memory node when μ filial generation memory node fails,
Repair time is μ t2, reparation bandwidth is α/2 μ；If corresponding parent memory node equally fails, from adjacent effective filial generation
Memory node obtains, and repair time is μ t1, reparation bandwidth is α/2 μ.
In order to further verify the scheme of the present embodiment proposition, MATLAB software can be used and carry out emulation experiment.When
So, other software can be used to carry out emulation experiment in those skilled in the art.
In this experiment, it is assumed that our parameter model is (n1, k, d, θ, α)=(15,7,10,60,4), original number
It is 1GB, t according to size M1=1s, t2=2s,
Fig. 3 is shown the total repair time T's and failure memory node number m of the present embodiment part replica code IFR code
Relationship.When adjacent storage nodes fail number w value w=1, w=2, w=5, total repair time is higher and higher, so by Fig. 3
It can be seen that adjacent storage nodes failure number w is bigger, total repair time is higher, and a kind of product matrix is regenerated code PM-MBR in figure
Total repair time of code is as a comparison, it can be seen that total repair time is higher than the part replica code IFR code of the present embodiment, Cong Tuzhong
It can be seen that the I FR code of the present embodiment total repair time is low.
The relationship between the tolerance ρ under the coding of three types and memory node n is shown in Fig. 4.For being based on
The part replica code of full figure, no matter the value size of total memory node n, tolerance only has 1 node；For common regeneration code,
According to MDS characteristic, k node can be tolerated, increase with the increase of n；For the part replica code IFR code of the present embodiment,
Due to the introducing of filial generation memory node, n node failure can be endured, as can be seen from Figure 4 the part replica code of the present embodiment
The tolerance highest of IFR code.
The relationship between reparation bandwidth β and failure node m under Fig. 5 three types coding.For the portion based on complete graph
Divide replica code, since tolerance is 1, so working as m=1, this part replica code, which has, repairs bandwidth α；
For common MBR code, total bandwidth of repairing isWith failure node m's
Increase and increases；For the part replica code IFR code of the present embodiment, total bandwidth of repairing is m α GB.From fig. 5, it can be seen that this reality
The part replica code IFR code for applying example can repair that node is more and total to repair bandwidth small.
The relationship between the total memory capacity S and memory node n under three types coding is shown in Fig. 6.For being based on
The part replica code of complete graph, total memory capacity S are n α GB；For common MBR code, total memory capacityFor the part replica code IFR code of the present embodiment, total memory capacity S=
2mαGB.From fig. 6, it can be seen that although the part replica code IFR code of the present embodiment increases one times of memory capacity cost, but
Total memory capacity is lower than common MBR code.
According to the one aspect of one or more other embodiments of the present disclosure, a kind of computer readable storage medium is provided.
A kind of computer readable storage medium, wherein being stored with a plurality of instruction, described instruction is suitable for by terminal device
Reason device loads and executes a kind of precise information restorative procedure based on distributed memory system.
According to the one aspect of one or more other embodiments of the present disclosure, a kind of terminal device is provided.
A kind of terminal device comprising processor and computer readable storage medium, processor is for realizing each instruction；Meter
Calculation machine readable storage medium storing program for executing is suitable for being loaded by processor and being executed described one kind and is based on for storing a plurality of instruction, described instruction
The precise information restorative procedure of distributed memory system.
These computer executable instructions execute the equipment according to each reality in the disclosure
Apply method or process described in example.
In the present embodiment, computer program product may include computer readable storage medium, containing for holding
The computer-readable program instructions of row various aspects of the disclosure.Computer readable storage medium, which can be, can keep and store
By the tangible device for the instruction that instruction execution equipment uses.Computer readable storage medium for example can be-- but it is unlimited
In-- storage device electric, magnetic storage apparatus, light storage device, electric magnetic storage apparatus, semiconductor memory apparatus or above-mentioned
Any appropriate combination.The more specific example (non exhaustive list) of computer readable storage medium includes: portable computing
Machine disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM or
Flash memory), static random access memory (SRAM), Portable compressed disk read-only memory (CD-ROM), digital versatile disc
(DVD), memory stick, floppy disk, mechanical coding equipment, the punch card for being for example stored thereon with instruction or groove internal projection structure, with
And above-mentioned any appropriate combination.Computer readable storage medium used herein above is not interpreted instantaneous signal itself,
The electromagnetic wave of such as radio wave or other Free propagations, the electromagnetic wave propagated by waveguide or other transmission mediums (for example,
Pass through the light pulse of fiber optic cables) or pass through electric wire transmit electric signal.
Computer-readable program instructions described herein can be downloaded to from computer readable storage medium it is each calculate/
Processing equipment, or outer computer or outer is downloaded to by network, such as internet, local area network, wide area network and/or wireless network
Portion stores equipment.Network may include copper transmission cable, optical fiber transmission, wireless transmission, router, firewall, interchanger, gateway
Computer and/or Edge Server.Adapter or network interface in each calculating/processing equipment are received from network to be counted
Calculation machine readable program instructions, and the computer-readable program instructions are forwarded, for the meter being stored in each calculating/processing equipment
In calculation machine readable storage medium storing program for executing.
Computer program instructions for executing present disclosure operation can be assembly instruction, instruction set architecture (ISA)
Instruction, machine instruction, machine-dependent instructions, microcode, firmware instructions, condition setup data or with one or more programmings
The source code or object code that any combination of language is write, the programming language include the programming language-of object-oriented such as
C++ etc., and conventional procedural programming languages-such as " C " language or similar programming language.Computer-readable program refers to
Order can be executed fully on the user computer, partly be executed on the user computer, as an independent software package
Execute, part on the user computer part on the remote computer execute or completely on a remote computer or server
It executes.In situations involving remote computers, remote computer can include local area network by the network-of any kind
(LAN) or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as utilize internet
Service provider is connected by internet).In some embodiments, by being believed using the state of computer-readable program instructions
Breath comes personalized customization electronic circuit, such as programmable logic circuit, field programmable gate array (FPGA) or programmable logic
Array (PLA), the electronic circuit can execute computer-readable program instructions, to realize the various aspects of present disclosure.
It should be noted that although being referred to several modules or submodule of equipment in the detailed description above, it is this
Division is only exemplary rather than enforceable.In fact, in accordance with an embodiment of the present disclosure, two or more above-described moulds
The feature and function of block can embody in a module.Conversely, the feature and function of an above-described module can be with
Further division is to be embodied by multiple modules.
The disclosure the utility model has the advantages that
A kind of distributed memory system and its precise information restorative procedure described in the present embodiment and device, part duplication are compiled
The tolerance of code is high, it can the failure node number of receiving is high, and can accurately repair initial data completely, ensure that distribution
The safety of formula memory system data, total memory capacity is more common, and MBR code is lower, while the disclosure is compared to research side previous
Method has lower total repair time and total reparation bandwidth, and reliability and security is high；
A kind of distributed memory system and its precise information restorative procedure described in the present embodiment and device, part duplication are compiled
The repair time of code is very fast, and without coding and decoding, convenient and efficient, computation complexity is low, is easy to implement.
The foregoing is merely preferred embodiment of the present application, are not intended to limit this application, for the skill of this field
For art personnel, various changes and changes are possible in this application.Within the spirit and principles of this application, made any to repair
Change, equivalent replacement, improvement etc., should be included within the scope of protection of this application.Therefore, the present embodiment is not intended to be limited to
The embodiments shown herein, and it is to fit to widest model consistent with the principles and novel features disclosed in this article
It encloses.
Claims (10)
1. a kind of distributed memory system, which is characterized in that the system includes: parent memory node set and filial generation memory node
Set；
The parent memory node set includes several sequentially connected parent memory nodes, for storing initial data；It is described
Filial generation memory node set includes several child nodes, and each parent memory node is connect with two child nodes, is counted for itself
It is stored according to the copy of content；
When the parent memory node fails, obtains and lose from coupled parent memory node and/or filial generation memory node
The identical data of data are imitated, precise information reparation is carried out.
2. a kind of distributed memory system as described in claim 1, which is characterized in that the original in the distributed memory system
Beginning data constitute a part replica code according to graphtheoretic concept, and the part replica code is data element set several different
With the set of several memory nodes, the set of several memory nodes is that the parent is deposited in the part replica code
Store up node set.
3. a kind of distributed memory system as described in claim 1, which is characterized in that
The storing data size of the parent memory node is twice of the filial generation storage node storing data size；The father
It is the half of the filial generation storage node number for memory node number.
4. a kind of distributed memory system as described in claim 1, which is characterized in that
Data downloading cost is identical between the parent memory node, and data download cost phase between the filial generation memory node
Together, data downloading cost is identical between data downloading cost and the filial generation memory node between the parent memory node；
Data downloading cost is different between the parent memory node and the filial generation memory node；
Between the parent memory node and the filial generation memory node data download time be greater than the parent memory node it
Between data download time.
5. a kind of precise information restorative procedure based on distributed memory system, this method is based on such as any one of claim 1-4
A kind of distributed memory system, comprising:
When the parent memory node fails, the failure node number of parent memory node and the half of memory node sum are judged
Size；
When the failure node of parent memory node is less than the half of memory node sum, from coupled parent memory node
And/or filial generation memory node obtains data identical with fail data, carries out precise information reparation；
When the failure node of parent memory node is not less than the half of memory node sum, section is stored from coupled filial generation
Point obtains data identical with fail data, carries out precise information reparation.
6. a kind of precise information restorative procedure based on distributed memory system as claimed in claim 5, which is characterized in that
It include a parent memory node when the failure node of parent memory node is less than the half of memory node sum in this method
It fails, the failure node of parent memory node is less than non-conterminous and parent between the half and failure node of memory node sum and deposits
The failure node for storing up node is less than the half of memory node sum but there are three kinds of feelings adjacent between at least two failure nodes
Condition.
7. a kind of precise information restorative procedure based on distributed memory system as claimed in claim 6, which is characterized in that
In this method, when a parent memory node fails, the distributed memory system parent memory node adjacent from failure node
Data identical with fail data are obtained, precise information reparation is carried out.
Further, in the method, when the failure node of parent memory node is less than the half of memory node sum and failure
When non-conterminous between node, all failure nodes of distributed memory system are obtained from its adjacent parent memory node and failure
The identical data of data carry out precise information reparation.
Further, in the method, when the failure node with parent memory node is less than the half of memory node sum but deposits
When adjacent between at least two failure nodes, the failure of the precise information of failure cannot be obtained from adjacent parent memory node
Node obtains data identical with fail data from filial generation memory node, and the adjacent parent that remaining failure node never fails is deposited
It stores up node and obtains data identical with fail data, carry out precise information reparation.
8. a kind of precise information restorative procedure based on distributed memory system as claimed in claim 5, which is characterized in that
In this method, when the failure node of parent memory node is not less than the half of memory node sum, all parent memory nodes
Failure node obtain identical with fail data data from the filial generation memory node of reply, when the failure of whole parent memory nodes
When, it directlys adopt filial generation memory node and is combined into corresponding parent memory node.
9. a kind of computer readable storage medium, wherein being stored with a plurality of instruction, which is characterized in that described instruction is suitable for by handling
Device is loaded and is executed such as a kind of described in any item precise information reparation sides based on distributed memory system claim 1-8
Method.
10. a kind of terminal device comprising processor and computer readable storage medium, processor is for realizing each instruction；Meter
Calculation machine readable storage medium storing program for executing is for storing a plurality of instruction, which is characterized in that described instruction is suitable for being loaded by processor and being executed such as
A kind of described in any item precise information restorative procedures based on distributed memory system of claim 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910113215.7A CN109828723B (en) | 2019-02-13 | 2019-02-13 | Distributed storage system and accurate data restoration method and device thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910113215.7A CN109828723B (en) | 2019-02-13 | 2019-02-13 | Distributed storage system and accurate data restoration method and device thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109828723A true CN109828723A (en) | 2019-05-31 |
| CN109828723B CN109828723B (en) | 2020-05-05 |
Family
ID=66863534
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910113215.7A Expired - Fee Related CN109828723B (en) | 2019-02-13 | 2019-02-13 | Distributed storage system and accurate data restoration method and device thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109828723B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110532125A (en) * | 2019-07-15 | 2019-12-03 | 长安大学 | A kind of part repetition code constructing method decomposed based on factor of diagram |
| CN110781163A (en) * | 2019-09-29 | 2020-02-11 | 长安大学 | Heterogeneous part repeated code construction and fault node repairing method based on complete graph |
| US11513898B2 (en) | 2019-06-19 | 2022-11-29 | Regents Of The University Of Minnesota | Exact repair regenerating codes for distributed storage systems |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2570925A1 (en) * | 2011-09-19 | 2013-03-20 | Thomson Licensing | Method of exact repair of pairs of failed storage nodes in a distributed data storage system and corresponding device |
| CN105830041A (en) * | 2014-11-27 | 2016-08-03 | 华为技术有限公司 | Metadata recovery method and apparatus |
| CN105930232A (en) * | 2016-05-12 | 2016-09-07 | 南京大学 | Simple regenerating code reparation method by using network topology information |
| US20170255510A1 (en) * | 2016-03-02 | 2017-09-07 | Yunshang company limited | System and method for regenerating codes for a distributed storage system |
| CN108512918A (en) * | 2018-03-23 | 2018-09-07 | 山东大学 | The data processing method of heterogeneous distributed storage system |
| CN108540520A (en) * | 2018-02-06 | 2018-09-14 | 长安大学 | Locality reparation coding based on part duplication code and node failure restorative procedure |
| CN109194444A (en) * | 2018-09-01 | 2019-01-11 | 哈尔滨工程大学 | A kind of balanced binary tree restorative procedure based on network topology |
-
2019
- 2019-02-13 CN CN201910113215.7A patent/CN109828723B/en not_active Expired - Fee Related
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2570925A1 (en) * | 2011-09-19 | 2013-03-20 | Thomson Licensing | Method of exact repair of pairs of failed storage nodes in a distributed data storage system and corresponding device |
| US20130073896A1 (en) * | 2011-09-19 | 2013-03-21 | Thomson Licensing | Method of exact repair of pairs of failed storage nodes in a distributed data storage system and corresponding device |
| CN105830041A (en) * | 2014-11-27 | 2016-08-03 | 华为技术有限公司 | Metadata recovery method and apparatus |
| US20170255510A1 (en) * | 2016-03-02 | 2017-09-07 | Yunshang company limited | System and method for regenerating codes for a distributed storage system |
| CN105930232A (en) * | 2016-05-12 | 2016-09-07 | 南京大学 | Simple regenerating code reparation method by using network topology information |
| CN108540520A (en) * | 2018-02-06 | 2018-09-14 | 长安大学 | Locality reparation coding based on part duplication code and node failure restorative procedure |
| CN108512918A (en) * | 2018-03-23 | 2018-09-07 | 山东大学 | The data processing method of heterogeneous distributed storage system |
| CN109194444A (en) * | 2018-09-01 | 2019-01-11 | 哈尔滨工程大学 | A kind of balanced binary tree restorative procedure based on network topology |
Non-Patent Citations (3)
| Title |
|---|
| JIAN XU,YEWEN CAO,DEQIANG WANG: ""Security Degree of Distributed Storage Systems with Generalized Regenerating Codes"", 《PUBLISHED IN 2016 9TH INTERNATIONAL CONGRESS ON IMAGE AND SIGNAL PROCESSING, BIOMEDICAL ENGINEERING AND INFORMATICS (CISP-BMEI)》 * |
| K. V. RASHMI等: ""Explicit Construction of Optimal Exact Regenerating Codes for Distributed Storage"", 《PUBLISHED IN: 2009 47TH ANNUAL ALLERTON CONFERENCE ON COMMUNICATION, CONTROL, AND COMPUTING (ALLERTON)》 * |
| 丁尚等: ""基于简单再生码的带宽感知的分布式存储节点修复优化"", 《软件学报》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11513898B2 (en) | 2019-06-19 | 2022-11-29 | Regents Of The University Of Minnesota | Exact repair regenerating codes for distributed storage systems |
| CN110532125A (en) * | 2019-07-15 | 2019-12-03 | 长安大学 | A kind of part repetition code constructing method decomposed based on factor of diagram |
| CN110781163A (en) * | 2019-09-29 | 2020-02-11 | 长安大学 | Heterogeneous part repeated code construction and fault node repairing method based on complete graph |
| CN110781163B (en) * | 2019-09-29 | 2023-01-17 | 长安大学 | Heterogeneous part repeated code construction and fault node repairing method based on complete graph |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109828723B (en) | 2020-05-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108540520B (en) | Partial repeated code based locality repairing coding and node fault repairing method | |
| CN105095013B (en) | Date storage method, restoration methods, relevant apparatus and system | |
| CN109828723A (en) | A kind of distributed memory system and its precise information restorative procedure and device | |
| CN109359223A (en) | The block chain account book distributed storage technology realized based on correcting and eleting codes | |
| CN108681569B (en) | Automatic data analysis system and method thereof | |
| CN103209210B (en) | Method for improving erasure code based storage cluster recovery performance | |
| Wang et al. | Ta-update: An adaptive update scheme with tree-structured transmission in erasure-coded storage systems | |
| CN110022216B (en) | Efficient asynchronous Byzantine consensus method with low communication complexity and network communication platform | |
| CN113645044B (en) | Consensus method, block chain system and consensus node | |
| CN110597655B (en) | Migration and erasure code-based reconstruction coupling rapid prediction repair method and device | |
| CN113630258B (en) | Consensus method, block chain system and consensus node | |
| CN109799948A (en) | A kind of date storage method and device | |
| CN106776129A (en) | A kind of restorative procedure of the multinode data file based on minimum memory regeneration code | |
| Hollmann | Storage codes—Coding rate and repair locality | |
| CN107357689A (en) | The fault handling method and distributed memory system of a kind of memory node | |
| CN109194444A (en) | A kind of balanced binary tree restorative procedure based on network topology | |
| CN113541870A (en) | Recovery optimization method for erasure code storage single node failure | |
| CN113761071A (en) | Consensus method, block chain system and consensus node | |
| CN105610879B (en) | Data processing method and device | |
| CN103812719B (en) | The failure prediction method and device of group system | |
| Zorgui et al. | Centralized multi-node repair in distributed storage | |
| CN108304264A (en) | A kind of correcting and eleting codes archiving method based on SPARK streaming computings | |
| CN116955355A (en) | Block data processing method and device and electronic equipment | |
| CN104572987B (en) | A kind of method and system that simple regeneration code storage efficiency is improved by compressing | |
| CN112527800B (en) | Tree chain type account book structure, high-performance parallel consensus method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200505 Termination date: 20210213 |