CN101997930A - Retransmission-erasure code transmission protocol-based remote mirroring method and system - Google Patents
Retransmission-erasure code transmission protocol-based remote mirroring method and system Download PDFInfo
- Publication number
- CN101997930A CN101997930A CN2010106037393A CN201010603739A CN101997930A CN 101997930 A CN101997930 A CN 101997930A CN 2010106037393 A CN2010106037393 A CN 2010106037393A CN 201010603739 A CN201010603739 A CN 201010603739A CN 101997930 A CN101997930 A CN 101997930A
- Authority
- CN
- China
- Prior art keywords
- backup
- data
- center
- batch
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a retransmission-erasure code transmission protocol-based remote mirroring method and a retransmission-erasure code transmission protocol-based remote mirroring system. The system comprises a write request interception module, a network block equipment driving module, a compression and encryption module, a batch code transmitting module and a backup module. The method comprises that: a production center intercepts a write request; a calculation center compresses and encrypts backup data; the calculation center transmits the backup data by using a Hybrid automatic repeat request/forward error correction (ARQ/FEC) transmission protocol; and a backup center backs up the data and the like. Compression, encryption and calculation can be efficiently performed according to the hardware configuration of the calculation center by using thread level parallel technology. The invention provides the Hybrid ARQ/FEC transmission protocol, and performs instruction level parallel optimization on odd and even calibration array codes in an FEC mechanism. The invention also discloses a Hybrid ARQ/FEC transmission protocol-based remote mirroring system. The problem of low efficiency under a wide area network environment in the prior art is solved, particularly the load of the production center can be effectively relieved in the application of a multi-backup center, and the throughput of the remote mirroring system is improved.
Description
Technical field
The invention belongs to the Remote Switched Port Analyzer technical field, particularly a kind of Remote Switched Port Analyzer method and Remote Switched Port Analyzer system.
Background technology
Now, the data disaster tolerance technology ever more important that become.Place remote mirror image technology particularly, it has caused that in the natural advantage that shows in the face of as natural calamities such as fire, earthquake, floods the time academia and industrial quarters more and more pay close attention to.
The Remote Switched Port Analyzer technology is applied under the wan environment, so many characteristics of wide area network are determining the characteristics in its design to a great extent: 1.RTT(Round Trip Time, two ends two-way time) can not ignore, it usually can be in the scope of 100 ~ 500ms.2. exist the possibility of packet loss, studies show that most of packet loss can float in 0% ~ 1%.3. transmission limited bandwidth is usually in 0.01 ~ 600Mbps scope.
In order to save the network bandwidth, can adopt compression algorithm to handle the data to wide area network to be sent, as lzw algorithm.In order to ensure the network security of data in the wide area network transmission course, can adopt cryptographic algorithm to treat to be sent to the data in the wide area network to handle, as aes algorithm.Yet the introducing of compression, cryptographic algorithm must bring the computing cost of can not ignore, if can not well be optimized, can cause the serious reduction of overall system performance.
Traditional Remote Switched Port Analyzer technology is in order to obtain reliable data transmission, use TCP in transport layer mostly, being that two ends rely on the simple automatic request retransmission of ARQ() mechanism guarantees data reliability, in a single day receiving terminal finds that packet loss can only rely on transmitting terminal to retransmit, this design has some fatal shortcomings: the more complicated of 1.TCP realization own, its realization can take a large amount of precious resources in two ends.2. big and when having packet loss as RTT, once retransmit and can seriously reduce the entire system throughput.3.TCP can't support the characteristic of the multicast system that makes to be difficult to be applied in the applied environment that has a plurality of backup centers.
The FEC(forward error correction) the existing very long history of Study on Technology, its core concept can be summarized as: at transmitting terminal the k blocks of data is encoded to n(n〉k) blocks of data, at receiving terminal, as long as receive k blocks of data different in this n blocks of data, thereby can finish decoding and obtain initial data, it is promptly a kind of that (n, coding k) can hold (n-k) individual data lost block.FEC is applied to a lot of fields: particularly at communication system, can use special-purpose hardware device to handle the complicated Reed Solomon coding based on finite field operations.Yet in computer system, owing to lacked special-purpose hardware device, high coding calculation cost makes the application of FEC be difficult to popularize.
Summary of the invention
The present invention seeks to solve the problem of prior art inefficiency under wan environment, a kind of Remote Switched Port Analyzer method and Remote Switched Port Analyzer system based on Hybrid ARQ/FEC host-host protocol is provided, this method can be good at handling load balancing, improve the throughput of whole system, can support to exist simultaneously the application of a plurality of backup centers.
Remote Switched Port Analyzer method based on Hybrid ARQ/FEC host-host protocol provided by the invention comprises:
1st, intercept and capture the write request of local source book the production center, and duplicate the Backup Data in the write request;
2nd, the production center is passed to the Backup Data in the write request of intercepting and capturing with being in a computer center in the local area network (LAN) by TCP;
3rd, computer center compresses and encryption the Backup Data of receiving, adopt Hybrid ARQ/FEC agreement as the host-host protocol of Backup Data in wide area network, the Backup Data that will compress with encryption sends to backup center, not only can avoid retransmitting but also data transmission credibility can being provided as far as possible;
4th, backup center to the Backup Data that computer center sends be decrypted, decompression, finally write disk as backup.
The described production center of the 1st step is intercepted and captured the write request of local source book and is meant at the logical volume layer write request of this locality is intercepted and captured, and has promptly adopted the so-called host-based(of industrial quarters based on main website) method.The method is than the method (as intercepting and capturing, intercept and capture, intercept and capture in disk array at SAN fabric at I/O card) of other intercepting and capturing write request, and advantage has: be easy to dispose, with low cost, use suitable medium-sized and small enterprises, business enterprice sector inside.
Described computer center is a notion in logic, can for one or more physically independently computing node constitute jointly, be used for the task of independent process computation-intensive.
Because the production center and computer center are in a local area network (LAN) inside together, and in local area network (LAN) phenomenon such as rare packet loss, RTT is generally less simultaneously, so preferential employing TCP finishes this transmission.Certainly, also can adopt udp protocol to finish this transformation task, but in order to prevent that receiving terminal (computer center) from receiving buffering area and overflowing the packet loss that causes, must take corresponding mechanism to guarantee data transmission credibility in application layer, the flow control of its basic logic and TCP be similar.
Computer center adopts the Thread-Level Parallelism technology that compression, encryption are optimized, various strategies are arranged, as: 1. will compress, the encryption stream waterline is subdivided into the thinner sub of granularity, promptly segment pipelining, this technology is more common in the hardware handles instruction level parallelism, but because the operation of computer center is difficult to be divided into more sub, also is difficult to each phased mission of reasonable distribution even can divide, make each sub easily because mutual exclusion forms a large amount of idle waiting, the cpu resource that waste is valuable.2. use multithreading that data are divided processing, a collection of relatively large data are divided into a plurality of small lot data parallels handle, the benefit of this technology is to divide task amount easily equally, makes each threads load equilibrium, and maximum possible reduces the cross-thread idle waiting time.
Preferably, utilize the data reliability of Hybrid ARQ/FEC mechanism in guaranteeing to transmit to be specially: data to be carried out sending these data to backup center by udp protocol behind the calculating operations such as compress-encrypt in computer center and back up.1. in order to guarantee transmission reliability, adopt explicit ARQ mechanism: promptly backup center is enabled timeout mechanism and is predicted whether packet loss has taken place, take place in prediction under the situation of packet loss, send repeat requests, the strategy of selecting re-transmission is selected in re-transmission for use, and computer center makes an immediate response after receiving repeat requests: the data that the transmission backup center need retransmit discharge the memory headroom that it confirms that paid-in data are shared simultaneously.2. to retransmit in order avoiding as possible, to adopt FEC mechanism: under wan environment, have the bigger possibility of packet loss and RTT,, can seriously reduce the throughput of total system if rely on ARQ mechanism to retransmit merely; And utilizing FEC mechanism, the packet that can rely on redundant data to lose at local instant recovery has overcome this problem.
FEC mechanism optionally encoding scheme has: can recover the Reed Solomon coding of a packet loss arbitrarily 1., it is a kind of coding based on finite field operations, in the application of communication system, can utilize special-purpose hardware device to handle Reed Solomon coding, and in computer system just its higher calculation cost make to use and face big difficulty.2. simple parity check array coding, as RDP, STAR coding etc., this type of coding is used for the fault-tolerant of storage system because lower time complexity is applied in the disk array widely, and its shortcoming is that fault-tolerant ability is more limited.
Preferably, utilize the SSE technology that the fault-tolerant encoding that adopts in the FEC mechanism is carried out parallel optimization and be specially: computer center carries out parallel optimization to the fault-tolerant encoding in the FEC mechanism that is adopted, and two kinds of selections of instruction level parallelism or Thread-Level Parallelism still can be arranged.Because encoding operation is lower with respect to the time complexity that compression, cryptographic operation calculate, grabbing system resource for the compression of getting along well, encryption thread causes frequent thread to switch, can use instruction level parallelism that coding is optimized at this, optimize version as the SSE that can develop RDP, STAR coding.Certainly, also can replace instruction level parallelism to carry out code optimization with the selection of configuration Thread-Level Parallelism according to the actual requirements.
Preferably, utilize the support of Hybrid ARQ/FEC host-host protocol to exist the application of a plurality of backup centers to be specially: to utilize the multicast feature of UDP, a kind of Hybrid ARQ/FEC agreement of design on application layer.Exist under the applied environment of a plurality of backup centers if adopt pure ARQ mechanism, it is serious that system effectiveness is descended: shown in the following formula, if the probability of a data packet loss is p, r recipient arranged, the probability that this packet is lost on each channel is identical, can be under pure ARQ mechanism this packet probability that must retransmit be:
Very fast change is big along with increasing of receiving terminal number by the visible retransmission probability of above-mentioned formula, promptly causes retransmitting in the transmission frequency and increases.And increasing of number of retransmissions not only can reduce the backup efficient of backup center, more can increase the weight of the processing burden (transmitting terminal must be handled too much repeat requests, promptly so-called " response storm ") of transmitting terminal, and then influences the performance of total system.
The present invention also discloses a kind of Remote Switched Port Analyzer system based on Hybrid ARQ/FEC host-host protocol, comprising: write request is intercepted and captured module, network block device driver module, compress-encrypt module, batch coding sending module and backup module.
Described write request is intercepted and captured module, is used for the logical volume layer of the heart aborning and intercepts and captures write request to source book, duplicates Backup Data wherein and passes to local network block device driver module;
Described network block device driver module is used for the generic block device drive layer reception Backup Data of the heart aborning, and Backup Data is passed to computer center;
Described compress-encrypt module is used in computer center Backup Data being carried out parallelly compressed, cryptographic operation;
Described batch of coding sending module, be used for the Backup Data that carries out overcompression, encryption being formed batch in computer center, and carry out numbering scheme to one batch and produce the redundant data that is used to recover packet loss, at last by send thread according to batch granularity Backup Data is sent to backup module by wide area network;
Described backup module, be used for backup center receive data and to its be decrypted, decompression operation, finally write disk.
Advantage of the present invention and good effect:
The present invention can utilize compression, the cryptographic operation of Thread-Level Parallelism technical finesse Backup Data according to computer center's hardware configuration, has not only effectively saved the network bandwidth, guaranteed network security but also has alleviated the load of the production center.Utilize Hybrid ARQ/FEC host-host protocol, avoid to greatest extent retransmitting, guarantee the throughput of system.The parity check array coding that utilizes instruction level concurrent technique to optimize simultaneously to use in the FEC technology has further improved the treatment effeciency of computer center.The Remote Switched Port Analyzer method based on the ARQ/FEC host-host protocol of the present invention's design can be good at handling load balancing, improves the throughput of whole system, may operate in simultaneously in the applied environment of a plurality of backup centers.
Description of drawings
Fig. 1: the flow chart of handling compression, encrypted backup data for computer center;
Fig. 2: for computer center to data organize batch, coding and the flow chart that sends;
Fig. 3: the flow chart of accepting data for backup center;
Fig. 4: be the flow chart of backup center Backup Data;
Fig. 5: be the flow chart of computer center's processing response under the situation that single backup center exists;
Fig. 6: be the flow chart of computer center's processing response under the situation that a plurality of backup centers exist;
Fig. 7: be Remote Switched Port Analyzer system module schematic diagram based on Hybrid ARQ/FEC host-host protocol;
Fig. 8: be the data message form schematic diagram of Hybrid ARQ/FEC host-host protocol;
Fig. 9: be the response message form schematic diagram of Hybrid ARQ/FEC host-host protocol;
Figure 10: be the inventive method The general frame.
Embodiment
Embodiment 1:
For above-mentioned purpose of the present invention, feature and advantage can be become apparent more, the present invention will be further described in detail below in conjunction with the drawings and specific embodiments.
The first step of the present invention is that intercept and capture the write request of local source book the production center, duplicates the Backup Data in the write request; After finishing above-mentioned steps, the production center is passed to the Backup Data in the write request of intercepting and capturing with being in a computer center in the local area network (LAN) by TCP.
Consult Fig. 1, the flow chart that computer center of the present invention handles compression, encrypted backup data is shown, concrete steps are as described below:
Take out data to be calculated from the raw data field: the raw data field is meant deposits the initial data that the production center sends to computer center, initial data is made up of data cell, each data cell is made up of disk start sector number, Backup Data length and Backup Data, and promptly a data unit pack has contained the required whole important informations of backup operation.
Open n thread and allocating task:, carry out the division and the distribution of task according to the data volume size to be calculated of taking out and the hardware configuration of computer center from the raw data field.Can when system initialization, open some computational threads and block them, only be responsible for waking up them by the thread of allocating task.If data volume to be calculated is too little, promptly overall tasks itself is very little, then is not suitable for waking up more thread computes.Can establish T is the general assignment deadline,
For waking up the time of all computational threads,
Be the used time of the thread of finishing calculating at last,
For blocking the time of all computational threads, be easy to get: T=
+
+
, promptly effectively calculate proportion and be
/ (
+
+
), obviously
Too small meeting makes that effective shared time of calculating is too small, and promptly expense is excessive.So can whether use multithreading to calculate according to data volume size to be calculated and hardware configuration decision, and use several threads to calculate.
The compress-encrypt data: this step refer to single thread use LZW algorithm and aes algorithm to data compress, computations.Wherein the AES cryptographic algorithm can not change data length, and the AES computing cost strengthens with data length, so locate to use compression back method of encrypting earlier.
Finish to calculate, will calculate the back data and put into the ripe data field: the ripe data field is the global data district of depositing through overcompression, encryption, simultaneously can be for the thread accesses of group batch, coding and transmission.After each thread was all finished calculating, the Task Distribution thread blocked them respectively, carry out synchronously, and the data after will calculating was put into the ripe data field.
Consult Fig. 2, illustrate computer center of the present invention to data organize batch, coding and the flow chart that sends, concrete steps are as described below:
Calculate the message number that batch_num:batch_num refers to batch.When fault-tolerant ability certain, promptly for the MDS sign indicating number, redundant data bar several regularly: few more this batch fault-tolerant ability that means of batch_num number is strong more, and redundancy rate is simultaneously also high more, and the wide area network bandwidth of waste is also many more; Otherwise more this batch fault-tolerant abilitys that mean of batch_num number weak more more, and redundancy rate is simultaneously also low more, and the wide area network bandwidth of waste is also few more.According to the principle that strategy separates with mechanism, agreement should provide a kind of mechanism can select the fault-tolerant ability of FEC for the user according to current network conditions.Specifically, the user provides the upper limit of a batch of number before each assembling batch, this upper limit must be less than 257, and this is the coding decision used by agreement: if this moment the ripe data field data can fill the upper limit of this batch, promptly select the value of this upper limit as batch_num; Otherwise the data volume of ripe data field draws the value of batch_num according to this moment.
The assembling lot data: one batch be by batch_num message be used for fault-tolerant some messages and constitute.A complete host-host protocol must provide standardized message format, in Hybrid ARQ/FEC agreement of the present invention, message format as shown in Figure 8, comprise batch number batch_id, batch number batch_num, this message batch in sequence number seq and Backup Data data.Batch number batch_id is by one 64 unsigned long type data representation, can allow batch number that per second consumes less than a upper limit by strategy, batch number as per second consumption in the 1024(experiment is far smaller than this value), 64 the data type of being easy to get enough satisfies actual the use, the problem that batch number overflows can not take place.The length of Backup Data data also is to be selected by the user, and selectable length must satisfied divide exactly 256 requirement, and this is by the employed coding decision of agreement.By analysis as can be known, Backup Data length is big more, and the heading amount of information that send is few more, and network bandwidth waste is few more.
Batch number batch_id, batch number batch_num and the sequence number seq of all messages take out the proper data unit of ripe data field afterwards in the initialization batch, in the message that is placed on batch that their first places are joined.Certainly, can have the situation of the discontented data cell of last message dress, mend 0 with remaining space and get final product this moment.
Be a batch coding: for fault-tolerant ability and performance demands, select suitable correcting and eleting codes to batch encoding according to the user.Optionally correcting and eleting codes is the MDS sign indicating number, comprising: RDP, STAR, Reed Solomon coding etc.Wherein, RDP is encoded to two fault-tolerant encodings, and STAR is encoded to three fault-tolerant encodings, and this two be simple parity check array coding.Reed Solomon coding can hold how wrong arbitrarily, is the coding based on finite field operations.Can use concurrent technique that coding is optimized in the realization, as SSE instruction level concurrent technique etc.
Send lot data: computer center uses UDP after backup center sends batch Backup Data, sends the redundant data that coding draws again.The purpose done of behaviour is to guarantee the backup center computing of only decoding in the case of necessary like this.
Consult Fig. 3, the flow chart that backup center of the present invention is accepted data is shown, concrete steps are as described below:
The message that step S301, reception computer center send.
Step S302, judge according to the batch number batch_id that receives message whether the contained data of this message back up, i.e. batch_id and backed up batch number update_batch relatively specifically is as if batch_id 〉=update_batch shows that this batch backs up.If back up, then jump to step S301.
Whether step S303, any one message of judging under this data strip batch according to the batch number batch_id that receives message were received, if then jump to step S305.
Step S304, set up and the data structure of one batch of initialization, this message that receives is put into this batch.And the renewal corresponding information, the message bitmap as affiliated this batch received, jumps to step S301 at this batch latest update time etc.
Whether step S305, this message were received, if then jump to step S301.
Step S306, this message that will receive are put into this batch and are upgraded corresponding information, as the message bitmap that affiliated this batch received, this batch latest update time etc.Jump to step S301.
Consult Fig. 4, the flow chart of backup center Backup Data of the present invention is shown, concrete steps are as described below:
Step S401, detect (update_batch+1) batch.
Step S402, detect any one message whether received (update_batch+1) batch, if then jump to step S405.
Step S403, find not receive any one message of (update_batch+1) batch, whether overtimely detect, then jump to step S401 if not.
Step S404, find overtimely, send and retransmit command request computer center and transmit (update_batch+1) lot data.Jump to step S401.
Step S405, detect (update_batch+1) and batch whether reach decode condition.If then jump to step S408.
Step S406, whether overtimely detect, then jump to step S401 if not.
Step S407, find overtimely, send and retransmit the obliterated data that command request computer center transmits (update_batch+1) batch.Jump to step S401.
Whether step S408, detection need decoding, then jump to step S410 if not.
Step S409, according to decoding algorithm, decode
Step S410, deciphering, decompress(ion) contract Backup Data to disk, upgrade and have backed up batch number update_batch.Jump to step S401.
Consult Fig. 5, flow chart (wherein response message form such as Fig. 9 of computer center of the present invention processing response under the situation that single backup center exists are shown, batch_id points out batch number, type points out it is that ack confirms response or retr retransmits response, the bitmap bitmap illustrates paid-in batch message of backup center), concrete steps are as described below:
Step S501, receive response from backup center.
Step S502, if what receive is not ack response, promptly backup center is confirmed complete certain lot data of receiving, then jumps to step S504.
What step S503, discovery were received is the ack response, and the batch number batch_id that carries in meeting with a response discharges the memory headroom that batch number occupies less than all lot data of batch_id.Jump to step S501.
Step S504, if what receive is not retr response, i.e. the order of backup center request retransmission lot data then jumps to step S501.
What step S505, discovery were received is the retr response, the batch number batch_id that carries at first obtaining to respond, according to the bitmap mark that carries in the response, backup center is confirmed the memory headroom that paid-in data are shared in discharging batch, the data of backup center request retransmission in retransmitting batch.Jump to step S501.
Consult Fig. 6, the flow chart of computer center of the present invention processing response under the situation that a plurality of backup centers exist is shown, concrete steps are as described below:
Step S601, receive response from all backup centers.
Step S602, if what receive is not ack response, promptly backup center is confirmed complete certain lot data of receiving, then jumps to step S605.
What step S603, discovery were received is the ack response, and the batch number batch_id that carries in meeting with a response and the id of this backup center upgrade this backup center and confirm the complete batch number of receiving.Whether detect has certain batch to be received by all backup centers are complete.If no such batch, then jump to step S601.
Step S604, find to have certain batch to be received by all backup centers are complete, obtain this batch number batch_id.Discharge the memory headroom that batch number occupies less than all lot data of batch_id.Jump to step S601.
Step S605, if what receive is not retr response, i.e. the order of backup center request retransmission lot data then jumps to step S601.
Step S606, what find to receive is the retr response, at first obtains the batch number batch_id that carries in the response, according to the bitmap mark that carries in the response, and the data of backup center request retransmission in retransmitting batch.Jump to step S601.
Consult Fig. 7, the software module schematic diagram of system of the present invention is shown, concrete steps are as described below:
Write request is intercepted and captured module: the intercepting and capturing request module of production center end is intercepted and captured this write request when write request is carried out map operation, and the write request of being intercepted and captured duplicated and at local cache, and this write request is submitted to the network block device driver module of the production center end that coexists, after receiving replying of successfully receiving with the write request deletion of local cache.
The network block device driver module: the network block device driver module that is positioned at the production center receives the write request that the logical volume layer transmits, and these write requests is sent to the computer center that is in the same local area network (LAN) according to certain scheduling strategy.
Compress-encrypt module: after receiving the write request that production center network block device driver sends, computer center takes out the data in the write request, and m is done a calculating that the data that put forward are compressed and encrypted from write request according to the computational threads (being designated as k) that the hardware configuration of computer center is opened proper number, the order of calculating is that elder generation compresses afterwards and encrypts.The data number that preceding (k-1) individual computational threads is responsible for is
Individual, k thread is responsible for
Individual data.
Batch coding sending module: computer center's compressing data, encrypt finish after, these data are placed in the data structure that an overall situation shares, handle for the thread of being responsible for group batch and coding.The thread of being responsible for group batch and coding is at first got a collection of data according to user's setting and is formed the plurality of data message, there is plurality of data unit (data, write the initial address of disk sector, the length of data) in each data message, these data cells are emitted in the data strip successively end to endly, and a plurality of data messages may be crossed in a data unit.Select certain coding (as RDP, STAR, Reed Solomon coding) that the data message of this batch is carried out encoding operation at last.The present invention also provides the SSE of every kind of coding to optimize version.
Backup module: computer center's group batch, coding can send to the data strip of this batch the backup center that is positioned at far-end and back up after finishing.After backup center receives data, at first be decrypted, decompression operation, data write disk and finish backup the most at last.
More than to a kind of Remote Switched Port Analyzer method and Remote Switched Port Analyzer system provided by the present invention based on Hybrid ARQ/FEC host-host protocol, be described in detail, used specific case herein principle of the present invention and execution mode are set forth, the explanation of above embodiment just is used for helping to understand method of the present invention and core concept thereof; So this description should not be construed as limitation of the present invention.
Claims (7)
1. one kind based on re-transmission-the Remote Switched Port Analyzer method of correcting and eleting codes host-host protocol, it is characterized in that, and comprising:
1st, intercept and capture the write request of local source book the production center, and duplicate the Backup Data in the write request;
2nd, the production center is passed to the Backup Data in the write request of intercepting and capturing with being in a computer center in the local area network (LAN) by TCP;
3rd, computer center compresses and encryption the Backup Data of receiving, adopt Hybrid ARQ/FEC agreement as the host-host protocol of Backup Data in wide area network, the Backup Data that will carry out overcompression and encryption sends to backup center, not only can avoid re-transmission as far as possible but also data transmission credibility can be provided;
4th, backup center to the Backup Data that computer center sends be decrypted, decompression, finally write disk as backup.
2. the method for claim 1 is characterized in that, the described production center of the 1st step is intercepted and captured the write request of local source book and is meant at the logical volume layer write request of this locality is intercepted and captured.
3. the method for claim 1 is characterized in that, the 3rd step described to Backup Data compress, in the encryption process, the user can adopt the Thread-Level Parallelism technology that compression, cryptographic operation are optimized according to hardware configuration.
4. the method for claim 1 is characterized in that, uses the scheme of Hybrid ARQ/FEC host-host protocol to be:
The fault-tolerant encoding that FEC adopted comprises: Reed Solomon and parity check array coding RDP, STAR.
5. method as claimed in claim 4 is characterized in that, the scheme of the fault-tolerant encoding that adopts among the FEC is:
Utilize the SSE technology that parity check array coding RDP, the STAR that adopts in the FEC mechanism carried out instruction level parallelism optimization.
6. as each described method in the claim 1 to 5, it is characterized in that, have the applied environment of many backup centers based on the Remote Switched Port Analyzer method support of Hybrid ARQ/FEC host-host protocol:
Hybrid ARQ/FEC mechanism can be utilized the function of ip multicast, and original Remote Switched Port Analyzer method is expanded under the environment that has many backup centers, effectively alleviates the burden of transmitting terminal.
7. one kind based on re-transmission-and the Remote Switched Port Analyzer system of correcting and eleting codes host-host protocol is characterized in that comprise: write request is intercepted and captured module, network block device driver module, compress-encrypt module, batch coding sending module and backup module;
Described write request is intercepted and captured module, is used for the logical volume layer of the heart aborning and intercepts and captures write request to source book, duplicates Backup Data wherein and passes to local network block device driver module;
Described network block device driver module is used for the generic block device drive layer reception Backup Data of the heart aborning, and Backup Data is passed to computer center;
Described compress-encrypt module is used in computer center Backup Data being carried out parallelly compressed, cryptographic operation;
Described batch of coding sending module is used in computer center the Backup Data that carries out overcompression, encryption being formed batch, encodes and sends;
Described backup module, be used for backup center receive data and to its be decrypted, decompression operation, finally write disk.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010106037393A CN101997930A (en) | 2010-12-24 | 2010-12-24 | Retransmission-erasure code transmission protocol-based remote mirroring method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010106037393A CN101997930A (en) | 2010-12-24 | 2010-12-24 | Retransmission-erasure code transmission protocol-based remote mirroring method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101997930A true CN101997930A (en) | 2011-03-30 |
Family
ID=43787500
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010106037393A Pending CN101997930A (en) | 2010-12-24 | 2010-12-24 | Retransmission-erasure code transmission protocol-based remote mirroring method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101997930A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102253871A (en) * | 2011-07-13 | 2011-11-23 | 重庆大学 | Data backup and reduction method, device and system |
| CN103179204A (en) * | 2013-03-13 | 2013-06-26 | 广东新支点技术服务有限公司 | Double-proxy-based WAN (wide area network) disk image optimization method and device |
| CN103259797A (en) * | 2013-05-17 | 2013-08-21 | 中国联合网络通信集团有限公司 | Data file transmission method and platform |
| CN106250784A (en) * | 2016-07-20 | 2016-12-21 | 乐视控股（北京）有限公司 | Full disk encryption method and device |
| CN109472680A (en) * | 2018-10-26 | 2019-03-15 | 中国银行股份有限公司 | A kind of accounting processing method and system |
| CN110943803A (en) * | 2019-12-09 | 2020-03-31 | 西南交通大学 | Data transmission control method based on erasure coding |
| CN111694692A (en) * | 2020-06-24 | 2020-09-22 | 山东云海国创云计算装备产业创新中心有限公司 | Data storage erasure method, device and equipment and readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1378395A (en) * | 2001-03-26 | 2002-11-06 | Lg电子株式会社 | Method for tramsmitting and receiving data grouping by using mixed automatic repeate demand |
| CN101316274A (en) * | 2008-05-12 | 2008-12-03 | 华中科技大学 | Data disaster tolerance system suitable for WAN |
| CN101414946A (en) * | 2008-11-21 | 2009-04-22 | 上海爱数软件有限公司 | Method and medium server for remote data backup |
| CN101635638A (en) * | 2008-07-25 | 2010-01-27 | 中兴通讯股份有限公司 | Disaster tolerance system and disaster tolerance method thereof |
-
2010
- 2010-12-24 CN CN2010106037393A patent/CN101997930A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1378395A (en) * | 2001-03-26 | 2002-11-06 | Lg电子株式会社 | Method for tramsmitting and receiving data grouping by using mixed automatic repeate demand |
| CN101316274A (en) * | 2008-05-12 | 2008-12-03 | 华中科技大学 | Data disaster tolerance system suitable for WAN |
| CN101635638A (en) * | 2008-07-25 | 2010-01-27 | 中兴通讯股份有限公司 | Disaster tolerance system and disaster tolerance method thereof |
| CN101414946A (en) * | 2008-11-21 | 2009-04-22 | 上海爱数软件有限公司 | Method and medium server for remote data backup |
Non-Patent Citations (1)
| Title |
|---|
| 王锋，刘晓光，王刚，刘璟: "基于LVM的异步实时远程复制系统", 《计算机工程》, vol. 33, no. 16, 31 August 2007 (2007-08-31), pages 261 - 263 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102253871A (en) * | 2011-07-13 | 2011-11-23 | 重庆大学 | Data backup and reduction method, device and system |
| CN103179204A (en) * | 2013-03-13 | 2013-06-26 | 广东新支点技术服务有限公司 | Double-proxy-based WAN (wide area network) disk image optimization method and device |
| CN103259797A (en) * | 2013-05-17 | 2013-08-21 | 中国联合网络通信集团有限公司 | Data file transmission method and platform |
| CN103259797B (en) * | 2013-05-17 | 2016-03-23 | 中国联合网络通信集团有限公司 | data file transmission method and platform |
| CN106250784A (en) * | 2016-07-20 | 2016-12-21 | 乐视控股（北京）有限公司 | Full disk encryption method and device |
| CN109472680A (en) * | 2018-10-26 | 2019-03-15 | 中国银行股份有限公司 | A kind of accounting processing method and system |
| CN110943803A (en) * | 2019-12-09 | 2020-03-31 | 西南交通大学 | Data transmission control method based on erasure coding |
| CN110943803B (en) * | 2019-12-09 | 2021-10-08 | 西南交通大学 | Data transmission control method based on erasure coding |
| CN111694692A (en) * | 2020-06-24 | 2020-09-22 | 山东云海国创云计算装备产业创新中心有限公司 | Data storage erasure method, device and equipment and readable storage medium |
| CN111694692B (en) * | 2020-06-24 | 2022-04-22 | 山东云海国创云计算装备产业创新中心有限公司 | Data storage erasure method, device and equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101997930A (en) | Retransmission-erasure code transmission protocol-based remote mirroring method and system | |
| Guermouche et al. | Hydee: Failure containment without event logging for large scale send-deterministic mpi applications | |
| CN102273118B (en) | The method of data re-transmission, Apparatus and system | |
| CN108631947B (en) | RDMA (remote direct memory Access) network data transmission method based on erasure codes | |
| CN103209210B (en) | Method for improving erasure code based storage cluster recovery performance | |
| CN110313138B (en) | Related method and apparatus for achieving high availability using multiple network elements | |
| CN101132347A (en) | System and method for implementing TCP communication backup | |
| CN104393951A (en) | Queuing based remote sensing load universal data processing system | |
| WO2005099412A2 (en) | Information processing and transportation architecture for data storage | |
| US9118478B2 (en) | Fault-tolerant data transmission system for networks with non-full-duplex or asymmetric transport | |
| US20210288830A1 (en) | Early acknowledgment for write operations | |
| CN105743614A (en) | Data transmission method and device | |
| US20150333874A1 (en) | A transmitting and a receiving device and respective methods performed thereby for improving retransmissions in wireless networks | |
| WO2014166604A1 (en) | Dynamic scaling of redundancy elimination middleboxes | |
| CN101789846A (en) | Dissociation rate matching method and device | |
| WO2018209658A1 (en) | Data transmission method and device, machine readable storage medium, and system | |
| US7191356B2 (en) | Method for asynchronous support of fault-tolerant and adaptive communication | |
| US7178051B2 (en) | Method for synchronous support of fault-tolerant and adaptive communication | |
| US9973215B1 (en) | Controlled multipath data packet delivery with forward error correction | |
| Bouteiller et al. | Reasons for a pessimistic or optimistic message logging protocol in MPI uncoordinated failure, recovery | |
| CN101908978A (en) | Method and system for implementing mass data transmission | |
| JP2002330118A (en) | Method for controlling data delivery, data delivery system, data delivery control program, and medium where data delivery program is stored | |
| CN106375064A (en) | Password keyboard USB communication anomaly recovery method and system | |
| US9348776B2 (en) | Receiver and method for data processing | |
| JP5998923B2 (en) | Program, information processing apparatus, and communication method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20110330 |