CN110190926A - Correcting and eleting codes restorative procedure, correcting and eleting codes update method and system based on network query function - Google Patents
Correcting and eleting codes restorative procedure, correcting and eleting codes update method and system based on network query function Download PDFInfo
- Publication number
- CN110190926A CN110190926A CN201910341782.8A CN201910341782A CN110190926A CN 110190926 A CN110190926 A CN 110190926A CN 201910341782 A CN201910341782 A CN 201910341782A CN 110190926 A CN110190926 A CN 110190926A
- Authority
- CN
- China
- Prior art keywords
- node
- data
- correcting
- interchanger
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
- H04L1/0056—Systems characterized by the type of code used
- H04L1/0057—Block codes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/12—Discovery or management of network topologies
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/20—Support for services
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a kind of correcting and eleting codes restorative procedure, correcting and eleting codes update method and system based on network query function, belong to data reliable memory field, and restorative procedure includes: that starter node determines the enabled node for participating in repairing operation；Transmission path between node and the interchanger of participation calculating are determined by centralization controller；Enabled node sends data to starter node, participates in the interchanger calculated and carries out exclusive or to received data and forward operation result, until starter node receives data；Update method includes: that starter node determines node to be updated；Centralization controller determines the transmission path between node, and participates in the interchanger calculated；Starter node and back end are to interchanger transmission data, which carries out exclusive or to received data and result is sent to node to be updated, to complete the update to data block or check block to be updated.The present invention can reduce correcting and eleting codes reparation operation and correcting and eleting codes update the network flow that operation introduces, and improve operating efficiency.
Description
Technical field
The invention belongs to data reliable memory fields, more particularly, to a kind of correcting and eleting codes reparation based on network query function
Method, correcting and eleting codes update method and system.
Background technique
Correcting and eleting codes are widely used in distributed memory system reliable to provide data as a kind of data redundancy strategy
Property.In general, correcting and eleting codes generate a certain number of check blocks using a certain number of data blocks as input after encoding to it,
Such one group of data block and check block together form a kind of logical construction for being referred to as band.
Correcting and eleting codes reparation operation refers in degrade reading or data reconstruction task, is read from other nodes by network
Multiple available blocks are taken, the operation for losing data block are then repaired out by relevant matrix operation, as shown in Fig. 1 (a)；Due to coming
Client will be transmitted to from the data of multiple nodes or rebuilds node, therefore the downlink where client or reconstruction node
It will form a network bottleneck.Correcting and eleting codes update operation, refer to the consistency to guarantee data block and check block on same band,
The operation that check blocks all in band are also updated while updating a data block, as shown in Fig. 1 (b)；Due to by more
New back end can undertake all data and send task, and the uplink where back end will form a network bottle
Neck.According to operating principle shown in Fig. 1 (a) and Fig. 1 (b) it is found that traditional correcting and eleting codes reparation operation and update operation, can draw
Enter a large amount of network flow, and there are single-link bottleneck problem, this aspect will lead to reparation and update the low efficiency of operation
Under, take a long time, on the other hand can seriously affect the performance of other application in network.
Restorative procedure based on chain type transmission path shown in Fig. 1 (c) can be relieved the single-link in correcting and eleting codes reparation operation
Bottleneck problem, in the restorative procedure, each node receives it is subjected to XOR operation with the data being locally stored after data after
Operation result continuation is forwarded backward, this process is repeated up to always data and reaches the starter node for starting reparation operation.
Correcting and eleting codes restorative procedure shown in Fig. 1 (c) can alleviate the transmission pressure of bottleneck link, achieve the purpose that balancing link load,
But introduced network flow is not reduced really, these flows will be held in rare bandwidth resources and then influence system
The performance of other application.
Correcting and eleting codes update method shown in Fig. 1 (d) based on tree-shaped transmission path can be relieved correcting and eleting codes and update in operation
Single-link bottleneck problem, the update method by the node in need being updated carry out tissue according to tree, wherein by more
New back end is as root node, and for the check-node being updated as intermediate node and leaf node, root node is responsible for calculating
Increment is simultaneously transmitted to its child node by more new increment, more new data block, and child node updates local verification after receiving more new increment
The continuation of more new increment is simultaneously transmitted to downwards its child node by block, this process is repeated until that more new increment reaches all leaves
Child node.Correcting and eleting codes update method shown in Fig. 1 (d) will be updated the transmission pressure dissipation of back end to multiple schools originally
Node is tested, but does not completely eliminate network bottleneck, moreover, this method does not reduce introduced network flow really, this
A little flows will hold over the performance of other application in rare bandwidth resources and then influence system.
In general, existing correcting and eleting codes restorative procedure and correcting and eleting codes update method can introduce a large amount of network flow, meeting
The performance of other application in network is seriously affected, and operating efficiency is lower.
Summary of the invention
In view of the drawbacks of the prior art and Improvement requirement, the correcting and eleting codes reparation based on network query function that the present invention provides a kind of
Method, correcting and eleting codes update method and system, it is intended that reduction correcting and eleting codes reparation operation and correcting and eleting codes update what operation introduced
Network flow, and improve operating efficiency.
To achieve the above object, according to the invention to be repaired in a first aspect, providing a kind of correcting and eleting codes based on network query function
Compound method, comprising:
(S1) starter node determines the enabled node for participating in repairing operation according to encoding relation, and by each enabled node and its
The location information of itself is sent to the centralization controller in network；
(S2) centralization controller according to network topology determine starter node to the transmission path between each enabled node, and
The interchanger calculated is participated in transmission path, and identified information is sent to starter node；
(S3) after starter node establishes the connection between each enabled node according to transmission path, number is sent to each enabled node
According to request；
(S4) each enabled node sends data to starter node along transmission path, so that participating in calculating in transmission path
After interchanger carries out XOR operation to received data, operation result is forwarded along transmission path to starter node；
(S5) starter node receives after the data of transmission path to get to data block to be repaired, repairs operation
Terminate；
Wherein, starter node is the network node that operation is repaired in starting, is stored with and complex data category to be repaired on enabled node
In the data block or check block of same band.
The above method utilizes the computing capability of the network switch, executes XOR operation to the data from different nodes, so
Afterwards operation result is continued to forward, thus, it is possible to effectively avoid correcting and eleting codes reparation operate in part network transmission end to end, from
And alleviate the network bottleneck problem in correcting and eleting codes reparation operation, and reduce correcting and eleting codes reparation and operate introduced network flow.
Further, step (S1) includes:
Starter node band according to belonging to data block to be repaired obtains x available block in the band, and will store each
The network node of available block is determined to participate in the enabled node for repairing operation, and x enabled node is obtained；
The location information of identified x enabled node and the location information of its own are sent to center by starter node
Change controller；
Wherein, available block is to belong to the data block or check block of same band with data block to be repaired, and x obtained can
With block for decoding to obtain data block to be repaired.
Further, the interchanger for participating in calculating in step (S2), determined by centralization controller is all transmission roads
The interchanger of diameter point of intersection.
Further, step (S4) includes:
(S41) each enabled node obtains the available block being locally stored, and according between data block to be repaired and available block
Decoding relationship sends operation result to starter node along transmission path by each available block with after corresponding multiplication；
(S42) after the interchanger in each transmission path receives data, if the interchanger is the interchanger for participating in calculating,
XOR operation is carried out to received data, and forwards operation result to starter node along transmission path；Otherwise, the interchanger
Data directly received to starter node forwarding along transmission path；
Wherein, available block is to belong to the data block or check block of same band with data block to be repaired.
In data reparation operation, XOR operation is executed by the interchanger of all transmission path point of intersection, passes through exclusive or layer by layer
Mode by multiple data merge transmit, to reduce the data volume for needing to transmit in network, and eliminate starter node
The network bottleneck of downlink, improves the effect for repairing operation where (the reconstruction node of the degrade client or reconstruction tasks read)
Rate；Correcting and eleting codes restorative procedure based on network query function provided by through the invention, repair time of a data block close to
Normally read the time of a data block.
Second aspect according to the invention provides a kind of system, comprising: processor and computer readable storage medium；
Computer-readable recording medium storage has executable program code；
Processor executes the present invention first for calling the executable program code stored in computer readable storage medium
Correcting and eleting codes restorative procedure based on network query function provided by aspect.
The third aspect according to the invention provides a kind of correcting and eleting codes update method based on network query function, comprising:
(T1) starter node is determined according to encoding relation is stored with the back end of data block to be updated and is stored with to more
The check-node of new check block, and back end, check-node and the location information of its own be sent in network
Heart controller；
(T2) centralization controller determines starter node between back end and starter node is to each school according to network topology
The interchanger calculated is participated in the transmission path and transmission path tested between node, and identified information is sent to starting
Node；
(T3) after starter node establishes the connection between back end according to transmission path, data are sent more to back end
New request starts to send data with notification data node；
(T4) the interchanger transmission data that starter node and back end are calculated along transmission path to participation, so that the friendship
It changes planes after obtaining more new increment to received data progress XOR operation, data increment is sent to back end and each verification saves
Point；
(T5) after back end receives data increment, data block to be updated is updated, each check-node receives
After data increment, the check block to be updated being locally stored is updated respectively, updating operation terminates；
Wherein, starter node is the network node that starting updates operation.
The above method utilizes the computing capability of the network switch, executes to the data from starter node and back end different
Or operation, operation result is then transmitted to back end and all check-nodes, thus, it is possible to effectively avoid correcting and eleting codes more
Part network transmission end to end in new operation so that alleviating correcting and eleting codes updates network bottleneck problem in operation, and is reduced and is entangled
It deletes code and updates the introduced network flow of operation.
Further, step (T1) includes:
Starter node band according to belonging to data block to be updated obtains all check blocks in the band as to be updated
Check block, and the network node for being stored with data block to be updated is determined as back end, the network of each check block will be stored with
Node is determined as check-node；
Starter node is equal by the location information and the location information of its own of identified back end and each check-node
It is sent to centralization controller.
Further, the interchanger for participating in calculating in step (T2), determined by centralization controller is all transmission roads
Apart from the least interchanger of starter node hop count in the interchanger of diameter point of intersection.
Further, step (T4) includes:
(T41) data that starter node is written into are sent to the interchanger for participating in calculating, and back end will be to more
New data block is sent to the interchanger；
It (T42), will more new increment after the interchanger carries out XOR operation to received data to obtain more new increment
Back end and each check-node are sent in a manner of multicast.
Correcting and eleting codes update method provided by the present invention based on network query function, the execution of You Yitai interchanger are calculated more
New increment, and more new increment is sent to back end and each check-node via different transmission paths, it effectively reduces
The data volume of required transmission between back end and interchanger both reduces and updates the network flow that operation introduces, also eliminates
Network bottleneck improves update efficiency.
Fourth aspect according to the invention provides a kind of system, comprising: processor and computer readable storage medium；
Computer-readable recording medium storage has executable program code；
Processor executes third of the present invention for calling the executable program code stored in computer readable storage medium
Correcting and eleting codes update method based on network query function provided by aspect.
In general, contemplated above technical scheme through the invention, can obtain it is following the utility model has the advantages that
(1) the correcting and eleting codes restorative procedure and correcting and eleting codes update method provided by the present invention based on network query function, utilizes net
The computing capability of network interchanger executes XOR operation to the data from different nodes, and is forwarded to operation result, avoids
Correcting and eleting codes reparation operation and correcting and eleting codes update in operation part network transmission end to end, effectively reduce introduced network
Flow avoids the influence to other application in network.
(2) the correcting and eleting codes restorative procedure provided by the present invention based on network query function resolves into data reparation repeatedly different
Or operation, and XOR operation is moved in the network switch and is executed, so that the data of multiple nodes are after interchanger
It is merged into an intermediate result, the data for needing to transmit in network are reduced with this, and eliminates client or rebuilds section
The network bottleneck of downlink where point, improves the efficiency of correcting and eleting codes reparation operation；Net is based on provided by through the invention
The correcting and eleting codes restorative procedure that network calculates repairs time close to the normal time for reading a data block of a data block.
(3) script occurs at back end end the correcting and eleting codes update method provided by the present invention based on network query function
Incremental computations process be moved in the network switch and execute so that increment directly generates in a network and by interchanger via not
Same transmission path more will be sent to back end and check-node by new increment, avoid back end to link between interchanger
On network transmission, both reduce update operation introduce network flow, also eliminate network bottleneck, improve update operation
Efficiency.
Detailed description of the invention
Fig. 1 (a) is that existing RS (5,3) encode lower correcting and eleting codes restorative procedure schematic diagram；
Fig. 1 (b) is that existing RS (6,3) encode lower correcting and eleting codes update method schematic diagram；
Fig. 1 (c) is the correcting and eleting codes restorative procedure schematic diagram based on chain type transmission path under existing RS (5,3) coding；
Fig. 1 (d) is the correcting and eleting codes update method schematic diagram based on tree-shaped transmission path under existing RS (6,3) coding；
Fig. 2 is that RS provided in an embodiment of the present invention (5,3) encode lower correcting and eleting codes restorative procedure schematic diagram；
Fig. 3 is that RS provided in an embodiment of the present invention (6,3) encode lower correcting and eleting codes update method schematic diagram.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.As long as in addition, technical characteristic involved in the various embodiments of the present invention described below
Not constituting a conflict with each other can be combined with each other.
Correcting and eleting codes are many kinds of, wherein RS code has best storage efficiency and supports high level fault-tolerant, the coding of RS code with
And decoding algorithm is based on Galois Field (galois field), usually there are two configurable parameter n and k, n expressions for RS coding
One item takes the quantity of block (including data block and check block), and k then indicates the quantity of data block.In following inventive embodiments
In, technical solution of the present invention is explained in detail by taking RS code as an example.It should be understood that the present disclosure applies equally to use
The correcting and eleting codes of other coding modes.
Correcting and eleting codes restorative procedure provided by the present invention based on network query function, as shown in Figure 2, comprising:
(S1) starter node determines the enabled node for participating in repairing operation according to encoding relation, and by each enabled node and its
The location information of itself is sent to the centralization controller in network；
Wherein, starter node is the network node that operation is repaired in starting, specifically, in degradation reading task, starter node
For client, in data reconstruction task, starter node is to rebuild node；It is stored on enabled node and belongs to complex data to be repaired
The data block or check block of same band；
In an optional embodiment, step (1) is specifically included:
Starter node band according to belonging to data block to be repaired obtains x available block in the band, and will store each
The network node of available block is determined to participate in the enabled node for repairing operation, and x enabled node is obtained；
The location information of identified x enabled node and the location information of its own are sent to center by starter node
Change controller；
Wherein, available block is to belong to the data block or check block of same band with data block to be repaired, and x obtained can
With block for decoding to obtain data block to be repaired；The specific value of x, and the available block specifically chosen can be deleted according to actual entangle
Code type determines, in RS code, the value of x is the quantity k of data block in a band；
(S2) centralization controller according to network topology determine starter node to the transmission path between each enabled node, and
The interchanger calculated is participated in transmission path, and identified information is sent to starter node；
Centralization controller determines that the strategy of transmission path can be determined according to actual application scenarios；
In the present embodiment, the interchanger for participating in calculating determined by centralization controller is all transmission path point of intersection
Interchanger；
In data reparation operation, XOR operation is executed by the interchanger of all transmission path point of intersection, exclusive or can be passed through
Data are merged, to effectively reduce the data volume for needing to transmit in network；
(S3) after starter node establishes the connection between each enabled node according to transmission path, number is sent to each enabled node
According to request；
(S4) each enabled node sends data to starter node along transmission path, so that participating in calculating in transmission path
After interchanger carries out XOR operation to received data, operation result is forwarded along transmission path to starter node；
In an optional embodiment, step (S4) is specifically included:
(S41) each enabled node obtains the available block being locally stored, and according between data block to be repaired and available block
Decoding relationship sends operation result to starter node along transmission path by each available block with after corresponding multiplication；
(S42) after the interchanger in each transmission path receives data, if the interchanger is the interchanger for participating in calculating,
XOR operation is carried out to received data, and forwards operation result to starter node along transmission path；Otherwise, the interchanger
Data directly received to starter node forwarding along transmission path；
Wherein, available block is to belong to the data block or check block of same band with data block to be repaired.
(S5) starter node receives after the data of transmission path to get to data block to be repaired, repairs operation
Terminate.
By Fig. 2 respectively with Fig. 1 (a) and Fig. 1 (c) comparison it is found that the above-mentioned correcting and eleting codes restorative procedure based on network query function, is incited somebody to action
Multiple XOR operation is resolved into data reparation, and XOR operation is moved in the network switch and is executed, so that multiple sections
The data of point are merged into an intermediate result after interchanger, and the data for needing to transmit in network are reduced with this, and
The network bottleneck of downlink, improves the efficiency of correcting and eleting codes reparation operation where eliminating client or rebuilding node；Experiment
Show the provided correcting and eleting codes restorative procedure based on network query function through the invention, the time for repairing a data block is close
In the time for normally reading a data block.
With the above-mentioned correcting and eleting codes restorative procedure based on network query function correspondingly, the present invention also provides a kind of system, packet
It includes: processor and computer readable storage medium；Computer-readable recording medium storage has executable program code；
Processor executes above-mentioned based on net for calling the executable program code stored in computer readable storage medium
The correcting and eleting codes restorative procedure that network calculates.
The correcting and eleting codes update method based on network query function that the present invention also provides a kind of, as shown in Figure 3, comprising:
(T1) starter node is determined according to encoding relation is stored with the back end of data block to be updated and is stored with to more
The check-node of new check block, and back end, check-node and the location information of its own be sent in network
Heart controller；
Wherein, starter node is the network node that starting updates operation, and specifically, in writing task, starter node is visitor
Family end；
In an optional embodiment, step (T1) is specifically included:
Starter node band according to belonging to data block to be updated obtains all check blocks in the band as to be updated
Check block, and the network node for being stored with data block to be updated is determined as back end, the network of each check block will be stored with
Node is determined as check-node；
Starter node is equal by the location information and the location information of its own of identified back end and each check-node
It is sent to centralization controller；
(T2) centralization controller determines starter node between back end and starter node is to each school according to network topology
The interchanger calculated is participated in the transmission path and transmission path tested between node, and identified information is sent to starting
Node；
Centralization controller determines that the strategy of transmission path can be determined according to actual application scenarios；
In the present embodiment, the interchanger for participating in calculating determined by centralization controller is all transmission path point of intersection
Interchanger in apart from the least interchanger of starter node hop count；
(T3) after starter node establishes the connection between back end according to transmission path, data are sent more to back end
New request starts to send data with notification data node；
(T4) the interchanger transmission data that starter node and back end are calculated along transmission path to participation, so that the friendship
It changes planes after obtaining more new increment to received data progress XOR operation, data increment is sent to back end and each verification saves
Point；
In an optional embodiment, step (T4) is specifically included:
(T41) data that starter node is written into are sent to the interchanger for participating in calculating, and back end will be to more
New data block is sent to the interchanger；
It (T42), will more new increment after the interchanger carries out XOR operation to received data to obtain more new increment
Back end and each check-node are sent in a manner of multicast；
(T5) after back end receives data increment, data block to be updated is updated, each check-node receives
After data increment, the check block to be updated being locally stored is updated respectively, updating operation terminates.
By Fig. 3 respectively with Fig. 1 (b) and Fig. 1 (d) comparison it is found that the above-mentioned correcting and eleting codes update method based on network query function, is incited somebody to action
Originally incremental computations process of the generation at back end end, which is moved in the network switch, executes, so that increment is directly in a network
It generates and more will be sent to back end and check-node by new increment via different transmission paths by interchanger, avoid data
The network transmission of chain road between Node-to-switch both reduces and updates the network flow that operation introduces, also eliminates network
Bottleneck improves the efficiency for updating operation.
With the above-mentioned correcting and eleting codes update method based on network query function correspondingly, the present invention also provides a kind of system, packet
It includes: processor and computer readable storage medium；Computer-readable recording medium storage has executable program code；
Processor executes above-mentioned based on net for calling the executable program code stored in computer readable storage medium
The correcting and eleting codes update method that network calculates.
In general, the correcting and eleting codes restorative procedure and correcting and eleting codes update method provided by the present invention based on network query function,
XOR operation is executed to the data from different nodes using the computing capability of the network switch, then continues to turn by calculated result
Hair eliminates correcting and eleting codes reparation and updates the network bottleneck in operation so as to avoid partially network transmission end to end, and subtracts
Network flow is lacked, can be improved operating efficiency, and has avoided having an impact other application in network.
Correcting and eleting codes restorative procedure and update method provided by the present invention based on network query function is applicable to SDN
(Software Defined Networking, software defined network) etc. have centralization controller network in, it is therein in
Heart controller can obtain complete network topology, and be gone out between any two network node according to actual application scenarios decision
Transmission path.
It should be noted that according to actual application load characteristic, in the same distributed storage system based on correcting and eleting codes
In system, it can only realize one of above-mentioned correcting and eleting codes restorative procedure and correcting and eleting codes update method, also can be realized simultaneously this hair
Correcting and eleting codes restorative procedure provided by bright and correcting and eleting codes update method.For example, if systematic difference load, which has, is write intensive spy
Property without considering data reconstruction and degradation reading performance, can only realize the correcting and eleting codes provided by the present invention based on network query function
Update method；If systematic difference load can frequently trigger degradation reading or data reconstruction task and data are hardly updated, can
Only to realize the correcting and eleting codes restorative procedure provided by the present invention based on network query function；It is write intensively if systematic difference load has
Characteristic, and can frequently trigger degradation read or data reconstruction task, then can in systems simultaneously realize it is provided by the present invention
The correcting and eleting codes restorative procedure based on network query function and the correcting and eleting codes update method based on network query function.
Application example 1:
As shown in Fig. 2, the coding mode of correcting and eleting codes is specially RS (5,3) coding, i.e. n=5, k=3；
According to original encoding relation, in same band, data block and check block meet following relationship:
Wherein, d1、d2And d3For data block, p1And p2For check block, MCFor encoder matrix, element therein is corresponding
Coefficient, encoder matrix MCIt specifically can be generalized circular matrix, be also possible to other encoder matrixs；
It is operated by the client C starting correcting and eleting codes reparation of degradation reading task, data block to be repaired is data block d1；It is clear
Chu compactly explains technical solution provided by the present invention, subnetwork node and interchanger without drawn in the figure；According to this
Correcting and eleting codes restorative procedure based on network query function provided by inventing, treats repair data block d1Repair process it is as follows:
By client C according to data block d1The band at place obtains 3 available blocks in the band, is data block respectively
d2, data block d3With check block p1, by storing data block d2Node D2, storing data block d3Node D3And storage check block
p1Node P1It is determined to participate in the enabled node of reparation operation；
After determining above- mentioned information, client C is by node D2, node D3With node P1And the IP address of client C itself is equal
It is sent to centralization controller；
Centralization controller determines client C and node D according to network topology2, node D3With node P1Between transmission
Path is respectively path 1:D2→S1→S2→ C, path 2:D3→S1→S2→ C and path 3:P1→S2→C；Three transmission roads
There are two intersection points altogether for diameter, by the interchanger S of two point of intersection1And S2It is determined to participate in the interchanger of calculating；Centralization controller
Identified transmission path information and the exchanger information for participating in calculating are sent to client C；
Client C according to 3 transmission paths respectively with node D2, node D3With node P1Between establish connection after, to this 3
A node sends request of data；
According to above-mentioned encoding relation, available data block d1With data block d2, data block d3With check block p1Between solution
Code relationship are as follows:
Wherein, MDFor decoding matrix, the element in gust is corresponding coefficient；
Node D2By data block d2Multiplied by coefficient a1, node D3By data block d3Multiplied by coefficient a2, node P1By check block p1Multiply
With coefficient a3, each node sends operation result along corresponding transmission path to client C later；
Interchanger S1To received data a1d2And a2d3XOR operation is carried out, intermediate result t is obtained1=a1d2+
a2d3, then by intermediate result t1Continue to forward to client C along transmission path；
Interchanger S2To received data t1And a3p1XOR operation is carried out, obtaining data block to be repaired is d1=t1
+a3p1, then operation result is continued to forward to client C along transmission path；
Client C is received after the data that transmission path sends over to get to data block d to be repaired1, reparation behaviour
Work terminates.
Application example 2:
As shown in figure 3, the coding mode of correcting and eleting codes is specially RS (6,3) coding, i.e. n=6, k=3；
According to original encoding relation, data block and check block meet following relationship:
Wherein, d1、d2And d3For data block, p1、p2And p3Check block, ai,j(i, j ∈ { 1,2,3 }) are corresponding coefficient；
Operation is updated by client C starting correcting and eleting codes, data block to be updated is data block d1；For it is clear, compactly explain
Technical solution provided by the present invention, subnetwork node and interchanger are without drawn in the figure；Provided by according to the present invention
Correcting and eleting codes update method based on network query function, to data block d to be updated1Renewal process it is as follows:
By client C according to data block d1The band at place obtains all check block p in the band1、p2And p3, high-ranking officers
Test block p1、p2And p3It is used as check block to be updated, and determination is stored with data block d1Node D1, storage check block p1Node
P1, storage check block p2Node P2And storage check block p3Node P3；
After determining above- mentioned information, client C is by node D1, node P1, node P2With node P3And client C itself
IP address is sent to centralization controller；
Centralization controller determines client C and node D according to network topology1, node P1, node P2With node P3Between
Transmission path be respectively path 1:D1→ S → C, path 2:P1→ S → C and path 3:P2→ S → C and path 4:P3→S
→C；The interchanger of calculating will be determined to participate on four transmission paths intersection points apart from the least interchanger S of hop count with client；
Identified transmission path information and the exchanger information for participating in calculating are sent to client C by centralization controller；
Client C according to 4 transmission paths respectively with node D1, node P1, node P2With node P3Between establish connection
Afterwards, Xiang Jiedian D1It sends data and updates request, to notify node D1Start to send data；
The data d that client C is written intonewInterchanger S, node D are sent to along transmission path1Along transmission path to friendship
The S that changes planes sends data block d1Value dold；
Interchanger carries out XOR operation to obtain more new increment being δ=d to received datanew+dold, will more new increment
δ is sent to node D in a manner of multicast1, node P1, node P2With node P3；
Node D1After receiving more new increment δ, it is subjected to XOR operation with the data block that is locally stored, can be completed pair
Data block d1Update；
After each check-node receives more new increment δ, by its with after corresponding multiplication with the check block that is locally stored
XOR operation is carried out, the update to check block can be completed；Specifically, the update mode of each check block is,
Wherein, p1′、p2' and p3' it is respectively check block p1、p2And p3Updated value；
Data block and update after the completion of, the correcting and eleting codes update operation terminate.
As it will be easily appreciated by one skilled in the art that the foregoing is merely illustrative of the preferred embodiments of the present invention, not to
The limitation present invention, any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should all include
Within protection scope of the present invention.
Claims (10)
1. a kind of correcting and eleting codes restorative procedure based on network query function characterized by comprising
(S1) starter node according to encoding relation determine participate in repair operation enabled node, and by each enabled node and its own
Location information be sent to the centralization controller in network；
(S2) the centralization controller according to network topology determine the starter node to the transmission path between each enabled node,
And the interchanger calculated is participated in transmission path, and identified information is sent to the starter node；
(S3) it after the starter node establishes the connection between each enabled node according to the transmission path, is sent out to each enabled node
Send request of data；
(S4) each enabled node sends data to the starter node along the transmission path, so that joining in the transmission path
After carrying out XOR operation to received data with the interchanger of calculating, operation result is opened along the transmission path to described
Dynamic node forwarding；
(S5) starter node receives after the data of the transmission path to get to data block to be repaired, described
Repairing operation terminates；
Wherein, the starter node is the starting network node for repairing operation, be stored on the enabled node with it is described
Complex data to be repaired belongs to the data block or check block of same band.
2. the correcting and eleting codes restorative procedure based on network query function as described in claim 1, which is characterized in that step (S1) packet
It includes:
Starter node band according to belonging to the data block to be repaired, obtains x available block in the band, and will deposit
The network node for storing up each available block is determined to participate in the enabled node for repairing operation, and x enabled node is obtained；
The location information of identified x enabled node and the location information of its own are sent to described by the starter node
Centralization controller；
Wherein, the available block is to belong to the data block or check block of same band with the data block to be repaired, and the x can
With block for decoding to obtain the data block to be repaired.
3. the correcting and eleting codes restorative procedure based on network query function as described in claim 1, which is characterized in that the step (S2)
In, the interchanger for participating in calculating determined by the centralization controller is the interchanger of all transmission path point of intersection.
4. the correcting and eleting codes restorative procedure based on network query function as claimed in claim 3, which is characterized in that step (S4) packet
It includes:
(S41) each enabled node obtains the available block being locally stored, and according between the data block to be repaired and available block
Decoding relationship sends operation knot to the starter node along the transmission path by each available block with after corresponding multiplication
Fruit；
(S42) after the interchanger in each transmission path receives data, if the interchanger is the interchanger for participating in calculating, to institute
The data received carry out XOR operation, and forward operation result to the starter node along transmission path；Otherwise, the interchanger
Data directly received to starter node forwarding along transmission path；
Wherein, the available block is to belong to the data block or check block of same band with the data block to be repaired.
5. a kind of system, comprising: processor and computer readable storage medium；It is characterized in that, the computer-readable storage
Media storage has executable program code；
The processor is for calling the executable program code stored in the computer readable storage medium, right of execution
The benefit correcting and eleting codes restorative procedure based on network query function that requires 1-4 described in any item.
6. a kind of correcting and eleting codes update method based on network query function characterized by comprising
(T1) starter node is determined according to encoding relation is stored with the back end of data block to be updated and is stored with school to be updated
The check-node of block is tested, and the back end, the check-node and the location information of its own are sent in network
Centralization controller；
(T2) the centralization controller determines the starter node between the back end and described opening according to network topology
Participate in the interchanger of calculating on dynamic node to the transmission path and transmission path between each check-node, and by identified letter
Breath is sent to the starter node；
(T3) after the starter node establishes the connection between the back end according to the transmission path, Xiang Suoshu data section
Point sends data and updates request, to notify the back end to start to send data；
(T4) the interchanger transmission data that the starter node and the back end are calculated along the transmission path to participation, with
So that the data increment is sent to institute after the interchanger obtains more new increment to the received data progress XOR operation
State back end and each check-node；
(T5) after the back end receives the data increment, the data block to be updated is updated, each verification section
After point receives the data increment, the check block to be updated being locally stored is updated respectively, the update operation terminates；
Wherein, the starter node is the starting network node for updating operation.
7. the correcting and eleting codes update method based on network query function as claimed in claim 6, which is characterized in that step (T1) packet
It includes:
Starter node band according to belonging to the data block to be updated, obtain all check blocks in the band be used as to
Check block is updated, and the network node for being stored with the data block to be updated is determined as the back end, will be stored with each
The network node of check block is determined as the check-node；
The starter node is equal by the location information and the location information of its own of identified back end and each check-node
It is sent to the centralization controller.
8. the correcting and eleting codes update method based on network query function as claimed in claim 6, which is characterized in that the step (T2)
In, the interchanger for participating in calculating determined by the centralization controller is distance in the interchanger of all transmission path point of intersection
The least interchanger of starter node hop count.
9. the correcting and eleting codes update method based on network query function as claimed in claim 8, which is characterized in that step (T4) packet
It includes:
(T41) data that the starter node is written into are sent to the interchanger for participating in calculating, and the back end will
The data block to be updated is sent to the interchanger；
It (T42), will the more new increment after the interchanger carries out XOR operation to received data to obtain more new increment
The back end and each check-node are sent in a manner of multicast.
10. a kind of system, comprising: processor and computer readable storage medium；It is characterized in that, the computer-readable storage
Media storage has executable program code；
The processor is for calling the executable program code stored in the computer readable storage medium, right of execution
The benefit correcting and eleting codes update method based on network query function that requires 6-9 described in any item.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910341782.8A CN110190926B (en) | 2019-04-26 | 2019-04-26 | Erasure code updating method and system based on network computing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910341782.8A CN110190926B (en) | 2019-04-26 | 2019-04-26 | Erasure code updating method and system based on network computing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110190926A true CN110190926A (en) | 2019-08-30 |
| CN110190926B CN110190926B (en) | 2020-09-18 |
Family
ID=67715147
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910341782.8A Active CN110190926B (en) | 2019-04-26 | 2019-04-26 | Erasure code updating method and system based on network computing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110190926B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110618895A (en) * | 2019-09-29 | 2019-12-27 | 北京天融信网络安全技术有限公司 | Data updating method and device based on erasure codes and storage medium |
| CN111385200A (en) * | 2020-03-04 | 2020-07-07 | 中国人民解放军国防科技大学 | Control method and device for data block repair |
| CN111506428A (en) * | 2020-04-20 | 2020-08-07 | 中国科学技术大学 | Load balancing repair scheduling method based on erasure code storage system |
| CN112256471A (en) * | 2020-10-19 | 2021-01-22 | 北京京航计算通讯研究所 | Erasure code repairing method based on separation of network data forwarding and control layer |
| CN114143174A (en) * | 2021-11-30 | 2022-03-04 | 深信服科技股份有限公司 | Node repairing method, device, equipment and readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7681104B1 (en) * | 2004-08-09 | 2010-03-16 | Bakbone Software, Inc. | Method for erasure coding data across a plurality of data stores in a network |
| CN104461781A (en) * | 2014-12-01 | 2015-03-25 | 华中科技大学 | Data block reconstruction method based on erasure codes |
| CN105610921A (en) * | 2015-12-23 | 2016-05-25 | 华中科技大学 | Erasure code filing method based on data cache in cluster |
| CN105930103A (en) * | 2016-05-10 | 2016-09-07 | 南京大学 | Distributed storage CEPH based erasure correction code overwriting method |
| CN106708651A (en) * | 2016-11-16 | 2017-05-24 | 北京三快在线科技有限公司 | Erasure code-based partial write-in method and device, storage medium and equipment |
| CN108170555A (en) * | 2017-12-21 | 2018-06-15 | 浙江大华技术股份有限公司 | A kind of data reconstruction method and equipment |
-
2019
- 2019-04-26 CN CN201910341782.8A patent/CN110190926B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7681104B1 (en) * | 2004-08-09 | 2010-03-16 | Bakbone Software, Inc. | Method for erasure coding data across a plurality of data stores in a network |
| CN104461781A (en) * | 2014-12-01 | 2015-03-25 | 华中科技大学 | Data block reconstruction method based on erasure codes |
| CN105610921A (en) * | 2015-12-23 | 2016-05-25 | 华中科技大学 | Erasure code filing method based on data cache in cluster |
| CN105930103A (en) * | 2016-05-10 | 2016-09-07 | 南京大学 | Distributed storage CEPH based erasure correction code overwriting method |
| CN106708651A (en) * | 2016-11-16 | 2017-05-24 | 北京三快在线科技有限公司 | Erasure code-based partial write-in method and device, storage medium and equipment |
| CN108170555A (en) * | 2017-12-21 | 2018-06-15 | 浙江大华技术股份有限公司 | A kind of data reconstruction method and equipment |
Non-Patent Citations (1)
| Title |
|---|
| 唐英杰等: "纠删码存储系统中基于网络计算的高效故障重建方法", 《计算机研究与发展》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110618895A (en) * | 2019-09-29 | 2019-12-27 | 北京天融信网络安全技术有限公司 | Data updating method and device based on erasure codes and storage medium |
| CN110618895B (en) * | 2019-09-29 | 2023-06-09 | 北京天融信网络安全技术有限公司 | Data updating method and device based on erasure codes and storage medium |
| CN111385200A (en) * | 2020-03-04 | 2020-07-07 | 中国人民解放军国防科技大学 | Control method and device for data block repair |
| CN111506428A (en) * | 2020-04-20 | 2020-08-07 | 中国科学技术大学 | Load balancing repair scheduling method based on erasure code storage system |
| CN111506428B (en) * | 2020-04-20 | 2022-09-02 | 中国科学技术大学 | Load balancing repair scheduling method based on erasure code storage system |
| CN112256471A (en) * | 2020-10-19 | 2021-01-22 | 北京京航计算通讯研究所 | Erasure code repairing method based on separation of network data forwarding and control layer |
| CN114143174A (en) * | 2021-11-30 | 2022-03-04 | 深信服科技股份有限公司 | Node repairing method, device, equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110190926B (en) | 2020-09-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110190926A (en) | Correcting and eleting codes restorative procedure, correcting and eleting codes update method and system based on network query function | |
| CN111241114B (en) | Intelligent service transaction block chain credit management method and system combining transaction and consensus | |
| Bauer et al. | ARIES: A rearrangeable inexpensive edge-based on-line Steiner algorithm | |
| Loo et al. | Declarative networking: language, execution and optimization | |
| Naor et al. | A simple fault tolerant distributed hash table | |
| US7813302B2 (en) | Data distributing method, system transmitting method, and program | |
| Lee et al. | On scheduling redundant requests with cancellation overheads | |
| Patra et al. | Efficient asynchronous verifiable secret sharing and multiparty computation | |
| Altman et al. | Dynamic control of coding in delay tolerant networks | |
| CN105450741A (en) | Supply server selection and restoration data transmission method in distributed storage system | |
| CN109951391B (en) | Network path rapid calculation method based on multi-QoS constraint | |
| CN110113131A (en) | A kind of network communication method and system based on batch coding | |
| CN112232619A (en) | Block output and sequencing method, node and block chain network system of alliance chain | |
| CN104601473A (en) | Route generating method and system for multi-target path with restriction | |
| CN106991008A (en) | A kind of resource lock management method, relevant device and system | |
| CN107547346A (en) | A kind of message transmitting method and device | |
| WO2022110980A1 (en) | Intelligent network coding method and device based on deep reinforcement learning | |
| Sipos et al. | Erasure coded storage on a changing network: The untold story | |
| CN108304264A (en) | A kind of correcting and eleting codes archiving method based on SPARK streaming computings | |
| US7716250B1 (en) | Erasure coding and group computations using rooted binary and ternary trees | |
| Zorgui et al. | Centralized multi-node repair in distributed storage | |
| CN114237985B (en) | Method for repairing failed memory block in erasure code memory system and related device | |
| Avresky | Embedding and reconfiguration of spanning trees in faulty hypercubes | |
| Su et al. | A multi-stage metaheuristic algorithm for shortest simple path problem with must-pass nodes | |
| CN103929188B (en) | A kind of interpretation method, apparatus and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |