CN103684718A - Error-tolerant network redundancy elimination implementation method - Google Patents

Error-tolerant network redundancy elimination implementation method Download PDF

Info

Publication number
CN103684718A
CN103684718A CN201310645795.7A CN201310645795A CN103684718A CN 103684718 A CN103684718 A CN 103684718A CN 201310645795 A CN201310645795 A CN 201310645795A CN 103684718 A CN103684718 A CN 103684718A
Authority
CN
China
Prior art keywords
finger
print codes
packet
current
hash value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201310645795.7A
Other languages
Chinese (zh)
Other versions
CN103684718B (en
Inventor
张楠
阳小龙
张敏
隆克平
孙岩
姬桂君
赵慧杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Science and Technology Beijing USTB
Original Assignee
University of Science and Technology Beijing USTB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Science and Technology Beijing USTB filed Critical University of Science and Technology Beijing USTB
Priority to CN201310645795.7A priority Critical patent/CN103684718B/en
Publication of CN103684718A publication Critical patent/CN103684718A/en
Application granted granted Critical
Publication of CN103684718B publication Critical patent/CN103684718B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Data Exchanges In Wide-Area Networks (AREA)
  • Detection And Prevention Of Errors In Transmission (AREA)

Abstract

The invention provides an error-tolerant network redundancy elimination implementation method to solve the problem that error codes are easily generated in the redundancy elimination process, and belongs to the technical field of networks. The method includes the steps of firstly, conducting fingerprint coding on redundant data segments in a data packet to be sent; secondly, enabling a sending end to transmit the data packet where the fingerprint coding is conducted to a receiving end or a downstream node; finally, conducting decoding on the data packet through the receiving end or the downstream node. According to the method, under the condition that network transmission errors exist, the downstream node can timely and accurately extract fingerprint information and correctly decode and restore original grouped data.

Description

A kind of network redundancy of mistake tolerance is eliminated implementation method
Technical field
The present invention relates to networking technology area, the network redundancy that relates in particular to a kind of mistake tolerance is eliminated implementation method.
Background technology
Because a large amount of same or analogous contents on the Internet are repeatedly asked by the user of network edge, caused data repeating transmission on network, caused a large amount of redundancy flows, consume network bandwidth not only, and reduced the efficiency of the Internet.Along with capacity of memory device constantly expands, arithmetic facility performance improves constantly, price constantly reduces, storage facility in net is introduced in the Internet, make network there is memory function and become possibility, network redundancy flow technology for eliminating can be realized in the Internet with memory function, and it records the data of transmission over networks, and identification is repeated the data of transmission, thereby elimination redundant transmission, plays the effect that reduces network traffics, improves network performance.But, at data redundancy, eliminate (Data Redundancy Elimination, DRE) in process, the finger print information of coding can produce error code, the phenomenon such as overtime in transmitting procedure, network environment is complicated, this phenomenon is obvious, causes receiving terminal or downstream node to receive wrong finger print information, normally decoding.
For solving existing data redundancy technology for eliminating, easily produce error code, cause the normally problem of decoding of downstream node, in prior art, mainly contain following several settlement mechanism:
1, the settlement mechanism based on retransmitting: downstream node detects and receives data cases, finds that mistake requires transmitting terminal to resend packet.The problem that the method exists: serious waste Internet resources, especially for wireless transmission environment, the number of times that repeats transmission can be very many, had a strong impact on network transmission efficiency.
2, ACK intercepts: downstream node feeds back to transmitting terminal by the whether normal situation of packet after receiving packet.If data message is normal, transmitting terminal can normally send to downstream node really according to this packet of this validation of information.If packet information is undesired, transmitting terminal (is somebody's turn to do the information that the packet of error of transmission once appearred in " blacklist " record) by this information cache in " blacklist ", therefore, if same packet makes a mistake again, transmitting terminal was by should " blacklist " arriving the wrong packet of appearance by fast finding.There are 2 problems in the method: one, receiving terminal or downstream node receive normal packet and have sent confirmation, but because the reason transmitting terminal of network delay is not received confirmation and is thought that receiving terminal or downstream node successfully do not receive data by mistake, transmitting terminal can again send the data message that already success receives and cause information misinformation, thus waste Internet resources.Its two, the method cannot be applicable to not have in the agreement of feedback acknowledgment mechanism, such as udp protocol (User Datagram Protocol).
In sum, existing data redundancy technology for eliminating exists and in transmitting procedure, easily produces error code and cause receiving terminal or the downstream node cannot correct decoding, and network transmission efficiency is slow, the problem that network resources waste is serious.
Summary of the invention
The object of the present invention is to provide a kind of network redundancy of mistake tolerance to eliminate implementation method, to solve prior art when the finger print information that solves coding produces the problem of error code in transmitting procedure, the settlement mechanism of utilization based on retransmitting detects mistake and requires transmitting terminal to resend packet, serious waste Internet resources, be also difficult to guarantee the problem of the real-time transmission of message; And utilize the method that ACK intercepts to need feedback acknowledgment information, feedback acknowledgment information cannot be applicable to not exist the host-host protocol of feedback acknowledgment mechanism, and if can cause serious wasting of resources problem because the reasons such as network delay can not be transferred to the phenomenon that transmitting terminal causes transmitting terminal to repeat to send packet in time.
For solving the problems of the technologies described above, the network redundancy of a kind of mistake tolerance provided by the invention is eliminated implementation method, comprising: the redundant data segment in packet is carried out finger-print codes; Transmitting terminal by the data packet transmission after finger-print codes to receiving terminal or downstream node; Receiving terminal or downstream node carry out decoding to packet.
Preferably, described redundant data segment in packet is carried out finger-print codes, and its concrete steps are:
From current packet to be sent, select redundant data fragment;
By hash function, calculate the HASH value H of M redundant data fragment of first redundant data fragment to the 1to H m; Wherein, M is the sum of the redundant data fragment selected in current packet to be sent, and M>1;
By current first redundant data segment finger-print codes of selecting, be H mh 1h 2, by current M the redundant data segment finger-print codes of selecting, be H m-1h mh 1, by current n the redundant data segment finger-print codes of selecting, be H n-1h nh n+1, wherein, n is greater than 1 positive integer that is less than M;
Redundant data segment in current packet to be sent is replaced with its corresponding finger-print codes.
Preferably, before described receiving terminal or downstream node carry out decoding to packet, at described receiving terminal or downstream node HASH value table corresponding with data fragments of buffer memory in advance.
Preferably, described receiving terminal or downstream node carry out decoding to packet, and concrete steps are: extract in order all finger-print codes in packet; Determine the correct HASH value of the redundant data fragment that each finger-print codes is corresponding; According to the HASH value of each redundant data fragment, from buffer memory, in the HASH value table corresponding with data fragments, extract corresponding data fragments; Each finger-print codes in packet is replaced to the work decoding of complete paired data bag with its corresponding data fragments.
The concrete grammar of the correct HASH value of the redundant data fragment that preferably, described definite each finger-print codes is corresponding is:
S1: extract second HASH value in current finger-print codes, be designated as H;
S2: judge whether can be in described buffer memory to find the data fragments that H is corresponding in the HASH value table corresponding with data fragments, be the correct HASH value using H as redundant data segment corresponding to current finger-print codes, otherwise continue to carry out S3;
S3: extract respectively the HASH of the redundant data segment that current finger-print codes is corresponding in the previous finger-print codes of current finger-print codes and a rear finger-print codes, be designated as respectively HF, HL, carry out subsequently S4; Wherein, if current finger-print codes is first finger-print codes that in current data packet, order extracts, last finger-print codes order in current data packet being extracted is as the previous finger-print codes of described first finger-print codes; If current finger-print codes is last finger-print codes that in current data packet, order extracts, first finger-print codes order in current data packet being extracted is as a rear finger-print codes of described last finger-print codes;
S4: whether judgement can find HF or data fragments corresponding to HL in the described HASH value table corresponding with data fragments, it is the correct HASH value using the HASH value corresponding to this data slot as redundant data fragment corresponding to current finger-print codes, otherwise end data packet decoding flow process, requires transmitting terminal to resend this packet.
The beneficial effect of technique scheme of the present invention is as follows:
1, method provided by the invention is by adopting the HASH value of current redundant data fragment and front and back redundant data fragment thereof in conjunction with each redundant data fragment is carried out to finger-print codes, make because error of transmission causes receiving terminal or downstream node or downstream node normally during decoding, only need to can obtain the current correct HASH value that needs the redundant data fragment of decoding by the current last redundant data fragment of redundant data fragment of decoding or the finger-print codes of a rear redundant data fragment of needing, thereby get the data slot that this HASH is corresponding, therefore generally needn't require transmitting terminal again to send data, improved network transmission efficiency, saved the network bandwidth.
2, for the more severe network environment of transmission conditions, the likelihood ratio of transmission error code is larger, in method provided by the invention, receiving terminal or downstream node are normally during decoding, at receiving terminal or downstream node, just can to received information, confirm in time according to front and back finger print information, solve the method for existing feedback acknowledgment because the confirmation that confirmation postpones to make transmitting terminal also likely cannot normally receive receiving terminal or downstream node causes packet to repeat the problem sending.
Accompanying drawing explanation
The network redundancy elimination implementation method schematic flow sheet that Fig. 1 tolerates for a kind of mistake that the embodiment of the present invention provides;
Fig. 2 carries out the schematic flow sheet of finger-print codes to sent the redundant data segment in packet in Fig. 1;
Fig. 3 is the redundant data segment finger-print codes schematic diagram in current packet to be sent;
Fig. 4 is the HASH value expression intention corresponding with data fragments;
Fig. 5 is the schematic flow sheet of step S13 in Fig. 1;
Fig. 6 is the schematic flow sheet that obtains the correct HASH value of the redundant data fragment that each finger-print codes is corresponding.
Embodiment
For making the technical problem to be solved in the present invention, technical scheme and advantage clearer, be described in detail below in conjunction with the accompanying drawings and the specific embodiments.
The network redundancy that Figure 1 shows that a kind of mistake tolerance that the embodiment of the present invention provides is eliminated implementation method, comprising:
S11: the redundant data segment in packet is carried out finger-print codes;
S12: transmitting terminal by the data packet transmission after finger-print codes to receiving terminal or downstream node;
S13: receiving terminal or downstream node carry out decoding to packet.
Fig. 2 carries out the schematic flow sheet of finger-print codes to sent the redundant data segment in packet in Fig. 1, S11 specifically comprises the steps:
S21: select redundant data fragment from current packet to be sent.
S22: the HASH value H that calculates current M the redundant data fragment of first redundant data fragment to the of selecting by hash function 1to H m; Wherein, M is the sum of the redundant data fragment selected in current packet to be sent, while M>1.
S23: redundant data fragment is carried out to finger-print codes: be H by current first redundant data segment finger-print codes of selecting mh 1h 2, by current M the redundant data segment finger-print codes of selecting, be H m-1h mh 1, by current n the redundant data segment finger-print codes of selecting, be H n-1h nh n+1, wherein, n is greater than 1 positive integer that is less than M.
S24: the redundant data segment in current packet to be sent is replaced with its corresponding finger-print codes, complete the finger-print codes work to current packet to be sent.
Fig. 3 is the redundant data segment finger-print codes schematic diagram in current packet to be sent, and wherein M is the sum of data redundancy segment, and n is greater than 1 positive integer that is less than M; Row corresponding to " HASH value " shown the HASH value that each redundant data fragment is corresponding; The row of " finger-print codes " has shown the finger-print codes generating according to the HASH value of the redundancy segment of the HASH value of each data redundancy segment and front and back thereof.
Preferably, before the step S13 in Fig. 1, also comprise step: at receiving terminal or downstream node HASH value table corresponding with data fragments of buffer memory in advance, be illustrated in figure 4 the HASH value table corresponding with data fragments.
Preferably, as shown in Figure 5, the step S13 in Fig. 1 specifically comprises the following steps:
S51: extract all finger-print codes in packet;
S52: according to extracted all finger-print codes, determine the correct HASH value of the redundant data fragment that each finger-print codes is corresponding;
S53: according to the HASH value of each redundant data fragment, extract corresponding data fragments in the HASH value table corresponding with data fragments from buffer memory;
S54: each finger-print codes in packet is replaced to the work decoding of complete paired data bag with its corresponding data fragments.
Preferably, determine the redundant data fragment that each finger-print codes is corresponding correct HASH value concrete grammar as shown in Figure 6:
S61: extract second HASH value in current finger-print codes, be designated as H;
S62: judging whether to find the data fragments that H is corresponding in the HASH value table corresponding with data fragments, is to carry out S67; Otherwise continue to carry out S63;
S63: extract respectively the HASH value of the redundant data segment that current finger-print codes is corresponding in the previous finger-print codes of current finger-print codes and a rear finger-print codes, be designated as respectively HF, HL, carry out subsequently S64; Wherein, if current finger-print codes is first finger-print codes that in current data packet, order extracts, last finger-print codes order in current data packet being extracted is as the previous finger-print codes of described first finger-print codes; If current finger-print codes is last finger-print codes that in current data packet, order extracts, first finger-print codes order in current data packet being extracted is compiled as a rear fingerprint of described last finger-print codes;
S64: whether judgement can find HF or data fragments corresponding to HL in the HASH value table corresponding with data fragments, is to carry out S66; Otherwise carry out S65;
S65: end data packet decoding flow process, and send information to transmitting terminal, and inform the normally decoding of its receiving terminal or downstream node, require transmitting terminal to resend this packet;
S66: the HASH value corresponding to this data slot is returned as the correct HASH value of redundant data fragment corresponding to current finger-print codes;
S67: H is returned as the correct HASH value of redundant data segment corresponding to current finger-print codes.
For example, suppose that current receiving terminal will be to second finger-print codes H in the packet of current reception 1h 2h 3carry out decoding, its last finger-print codes is H 10h 1h 2, a finger-print codes is for compiling H thereafter 2h 3h 4, first from finger-print codes H 1h 2h 3in extract HASH value H 2, then in the HSAH of Fig. 4 value table corresponding with data slot, find H 2corresponding data slot, completes the correct decoding process of this finger-print codes.If second HASH value code word in second finger-print codes in the packet of current reception makes a mistake, in the HSAH of Fig. 4 value table corresponding with data slot, can not find second the HASH value extracting in second finger-print codes from the packet of current reception, extract the 3rd HASH value in its last finger-print codes, and extract first HASH value in a finger-print codes thereafter, if can find the 3rd HASH value in its last finger-print codes and any HASH value in first HASH value in a finger-print codes thereafter in the HSAH value table corresponding with data slot, current receiving terminal can be directly according to the H of first finger-print codes and the 3rd finger-print codes 2duan Jinhang decoding, and needn't require transmit leg again to transmit.
To sum up, the present invention the in the situation that of Internet Transmission mistake, the downstream node information that also can promptly and accurately take the fingerprint, and correct decoding reduction original packet data.Particularly, method provided by the invention is by adopting the HASH value of current redundant data fragment and front and back redundant data fragment thereof in conjunction with each redundant data fragment is carried out to finger-print codes, make to cause receiving terminal or downstream node normally during decoding because transmission misleads, only need to can obtain the current correct HASH value that needs the redundant data fragment of decoding by the current last redundant data fragment of redundant data fragment of decoding or the finger-print codes of a rear redundant data fragment of needing, thereby get the data slot that this HASH is corresponding, therefore generally needn't require transmitting terminal again to send data, improved network transmission efficiency, saved the network bandwidth.In addition, for the more severe network environment of transmission conditions, the likelihood ratio of transmission error code is larger, in method provided by the invention, receiving terminal or downstream node are normally during decoding, at receiving terminal or downstream node, just can to received information, confirm in time according to front and back finger print information, solve the method for existing feedback acknowledgment because the confirmation that confirmation postpones to make transmitting terminal also likely cannot normally receive receiving terminal or downstream node causes packet to repeat the problem sending.
It is more than the preferred embodiment of the present invention; it should be pointed out that for those skilled in the art, do not departing under the prerequisite of principle of the present invention; can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.

Claims (5)

1. the network redundancy of mistake tolerance is eliminated an implementation method, it is characterized in that, comprising:
Redundant data segment in packet is carried out finger-print codes;
Transmitting terminal by the data packet transmission after finger-print codes to receiving terminal or downstream node;
Receiving terminal or downstream node carry out decoding to packet.
2. the network redundancy of a kind of mistake tolerance according to claim 1 is eliminated implementation method, it is characterized in that, described redundant data segment in packet is carried out finger-print codes, and its concrete steps are:
From current packet to be sent, select redundant data fragment;
By hash function, calculate the HASH value H of M redundant data fragment of first redundant data fragment to the 1to H m; Wherein, M is the sum of the redundant data segment selected in current packet to be sent, and M>1;
By current first redundant data segment finger-print codes of selecting, be H mh 1h 2, by current M the redundant data segment finger-print codes of selecting, be H m-1h mh 1, by current n the redundant data segment finger-print codes of selecting, be H n-1h nh n+1, wherein, n is greater than 1 positive integer that is less than M;
Redundant data segment in current packet to be sent is replaced with its corresponding finger-print codes.
3. the network redundancy of a kind of mistake tolerance according to claim 2 is eliminated implementation method, it is characterized in that, before described receiving terminal or downstream node carry out decoding to packet, at described receiving terminal or downstream node HASH value table corresponding with data fragments of buffer memory in advance.
4. the network redundancy of a kind of mistake tolerance according to claim 3 is eliminated implementation method, it is characterized in that, described receiving terminal or downstream node carry out decoding to packet, and concrete steps are:
Extract in order all finger-print codes in packet;
Determine the correct HASH value of the redundant data fragment that each finger-print codes is corresponding;
According to the correct HASH value of each redundant data fragment, from buffer memory, in the HASH value table corresponding with data fragments, extract corresponding data fragments;
Each finger-print codes in packet is replaced to the work decoding of complete paired data bag with its corresponding data fragments.
5. the network redundancy of a kind of mistake tolerance according to claim 4 is eliminated implementation method, it is characterized in that, the concrete grammar of the correct HASH value of the redundant data fragment that described definite each finger-print codes is corresponding is:
S1: extract second HASH value in current finger-print codes, be designated as H;
S2: judge whether can be in described buffer memory to find the data fragments that H is corresponding in the HASH value table corresponding with data fragments, be the correct HASH value using H as redundant data segment corresponding to current finger-print codes, otherwise continue to carry out S3;
S3: extract respectively the HASH of the redundant data segment that current finger-print codes is corresponding in the previous finger-print codes of current finger-print codes and a rear finger-print codes, be designated as respectively HF, HL, carry out subsequently S4; Wherein, if current finger-print codes is first finger-print codes that in current data packet, order extracts, last finger-print codes order in current data packet being extracted is as the previous finger-print codes of described first finger-print codes; If current finger-print codes is last finger-print codes that in current data packet, order extracts, first finger-print codes order in current data packet being extracted is as a rear finger-print codes of described last finger-print codes;
S4: whether judgement can find HF or data fragments corresponding to HL in the described HASH value table corresponding with data fragments, it is the correct HASH value using the HASH value corresponding to this data slot as redundant data fragment corresponding to current finger-print codes, otherwise end data packet decoding flow process, requires transmitting terminal to resend this packet.
CN201310645795.7A 2013-12-04 2013-12-04 Error-tolerant network redundancy elimination implementation method Expired - Fee Related CN103684718B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310645795.7A CN103684718B (en) 2013-12-04 2013-12-04 Error-tolerant network redundancy elimination implementation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310645795.7A CN103684718B (en) 2013-12-04 2013-12-04 Error-tolerant network redundancy elimination implementation method

Publications (2)

Publication Number Publication Date
CN103684718A true CN103684718A (en) 2014-03-26
CN103684718B CN103684718B (en) 2017-01-18

Family

ID=50321121

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310645795.7A Expired - Fee Related CN103684718B (en) 2013-12-04 2013-12-04 Error-tolerant network redundancy elimination implementation method

Country Status (1)

Country Link
CN (1) CN103684718B (en)

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5608396A (en) * 1995-02-28 1997-03-04 International Business Machines Corporation Efficient Ziv-Lempel LZI data compression system using variable code fields
CN102195790A (en) * 2010-03-05 2011-09-21 华为技术有限公司 Resource control method, device and system for peer-to-peer network
CN103036642B (en) * 2011-10-09 2015-11-25 华为技术有限公司 A kind of transmission method of data and transmitting terminal and receiving terminal
CN103401795B (en) * 2013-08-07 2016-03-23 电子科技大学 The removing method of transferring content redundancy in a kind of content center network

Also Published As

Publication number Publication date
CN103684718B (en) 2017-01-18

Similar Documents

Publication Publication Date Title
US9369255B2 (en) Method and apparatus for reducing feedback and enhancing message dissemination efficiency in a multicast network
CN102160315B (en) Iterative decoding of blocks with cyclic redundancy checks
CN107567107B (en) Data transmission method and device
CN106031072A (en) Harq frame data structure and method of transmitting and receiving with harq in systems using blind detection
CN113132063B (en) Physical layer retransmission control method
EP2782281A1 (en) Data transmission using rateless coding
CN103716130A (en) Physical layer self-adaption processing method for improving network transmission reliability
CN103078707A (en) File transmission method in deep space communication
US20160323068A1 (en) Hybrid Automatic Repeat Request Method and Related Apparatus
CN107209713B (en) Method and system for on-demand file repair
CN104683073A (en) A decoding method and receiving set
US9294227B2 (en) LT staircase FEC code
EP3499768A1 (en) Data processing method, device and node
US20160218825A1 (en) Rateless decoding
CN114389758A (en) Data transmission method and device
WO2015036086A1 (en) Rateless encoding
CN111246428A (en) Network reliable communication method, system, device and storage medium
CN103684718A (en) Error-tolerant network redundancy elimination implementation method
CN109039536A (en) Transmission method, equipment and the readable medium of data
CN114553375A (en) Data transmission method and device, electronic equipment and storage medium
CN103346862A (en) On-chip network data transmission device and method with cascade protection function
Willig Memory-efficient segment-based packet-combining schemes in face of deadlines
EP2930852B1 (en) Constructing a reliable data stream
Lin et al. Efficient frame aggregation transmission using random linear network coding
CN102694635A (en) Method and device for generating and using selective acknowledgement (SACK) option

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20170118

Termination date: 20211204

CF01 Termination of patent right due to non-payment of annual fee