CN118199800A - Data transmission optimization method and system - Google Patents
Data transmission optimization method and system Download PDFInfo
- Publication number
- CN118199800A CN118199800A CN202410294968.3A CN202410294968A CN118199800A CN 118199800 A CN118199800 A CN 118199800A CN 202410294968 A CN202410294968 A CN 202410294968A CN 118199800 A CN118199800 A CN 118199800A
- Authority
- CN
- China
- Prior art keywords
- data
- code
- hash code
- repair
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000005457 optimization Methods 0.000 title claims abstract description 21
- 230000008439 repair process Effects 0.000 claims abstract description 108
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
Landscapes
- Detection And Prevention Of Errors In Transmission (AREA)
Abstract
The invention relates to the technical field of computer communication, in particular to a data transmission optimization method and a system thereof, wherein the method comprises the following steps: s1, equipment A generates a hash code A for data A, and stores the hash code A serving as a key in a local disk; step S2, the equipment A transmits the data A to the equipment B; step S3, after the equipment A finishes transmitting the data A, the equipment A sends a hash code A to the equipment B; s4, after receiving the data A, the equipment B generates a hash code B for the data A; s5, the equipment B compares the received hash code A with the hash code B, and if the received hash code A is the same as the hash code B, the data transmission is correct; if it is different, a data transmission error is specified, data a is saved as data B, and then step S6 is performed. The invention can save bandwidth and avoid the occupation of bandwidth caused by the transmission of redundant codes when each error occurs; and when errors occur occasionally, the repair codes are transmitted in increments which are much smaller than the original data packets.
Description
Technical Field
The invention relates to the technical field of computer communication, in particular to a data transmission optimization method and a system thereof.
Background
At present, when data needs to be transmitted, signal interference or other reasons may possibly cause errors in the transmission process, so that the data is generally redundant or the retransmission of the whole data is failed, and thus, the bandwidth is actually wasted.
Disclosure of Invention
In order to solve the technical problems, the invention provides a data transmission optimization method and a system thereof, which can save bandwidth and avoid the occupation of bandwidth caused by transmission of redundant codes when each error occurs.
The invention is realized by adopting the following technical scheme:
A data transmission optimization method, the method comprising the steps of:
S1, equipment A generates a hash code A for data A, and stores the hash code A serving as a key in a local disk;
Step S2, the equipment A transmits the data A to the equipment B;
step S3, after the equipment A finishes transmitting the data A, the equipment A sends a hash code A to the equipment B;
S4, after receiving the data A, the equipment B generates a hash code B for the data A;
S5, the equipment B compares the received hash code A with the hash code B, and if the received hash code A is the same as the hash code B, the data transmission is correct; if the data is different, indicating that the data is transmitted in error, storing the data A as the data B, and then executing the step S6;
And S6, the equipment A generates a repair code and sends the repair code to the equipment B, and the equipment B repairs the data B according to the repair code after receiving the repair code.
Preferably, the step S6 is further specifically: if the data transmission errors are different, the data A is stored as data B, and the device B sends a request repair code instruction and the hash code A to the device A.
Preferably, the step S6 is further specifically: the device A searches the data A according to the hash code A, generates a repair code with the redundancy of 1% from the data A, and then sends the repair code to the device B.
Preferably, the repair code is generated by:
the method comprises the steps of dividing data A into blocks by taking N bytes as a unit, and generating a repair code with redundancy of 1% for each block A; the N is greater than 200 bytes.
Preferably, in the step S1, the data a is greater than 2000 bytes.
Preferably, the data B is segmented by taking N bytes as a unit, a corresponding repair code is applied to each segmented B, then data C is generated, then a hash code C is generated for the data C, the hash code C is compared with the hash code A, and if the hash code C and the hash code A are the same, the repair is successful; if the codes are different, the restoration is failed, the step S6 is continuously executed, and the multiplication and redundancy are added when the restoration codes are generated.
The system comprises a generation module A, a transmission module, a sending module, a generation module B, a comparison module and a restoration module;
Generating a module A: the device A generates a hash code A for the data A, and stores the hash code A as a key in a local disk;
and a transmission module: the device A transmits the data A to the device B;
And a sending module: after the equipment A finishes transmitting the data A, sending the hash code A to the equipment B;
generating a module B: after receiving the data A, the equipment B generates a hash code B for the data A;
And a comparison module: the equipment B compares the received hash code A with the hash code B, and if the received hash code A and the hash code B are the same, the data transmission is correct; if the data is different, indicating that the data is transmitted in error, storing the data A as the data B, and then executing the step S6;
and (3) a repair module: and the equipment A generates a repair code and sends the repair code to the equipment B, and the equipment B repairs the data B according to the repair code after receiving the repair code.
Preferably, the repair module further specifically includes: if the data transmission errors are different, the data A is stored as data B, and the device B sends a request repair code instruction and the hash code A to the device A.
Preferably, the repair module further specifically includes: the device A searches the data A according to the hash code A, generates a repair code with the redundancy of 1% from the data A, and then sends the repair code to the device B.
Preferably, the repair code is generated by:
the method comprises the steps of dividing data A into blocks by taking N bytes as a unit, and generating a repair code with redundancy of 1% for each block A; the N is greater than 200 bytes.
Preferably, in the generating module a, the data a is greater than 2000 bytes.
Preferably, the repair module further specifically includes: the data B is segmented by taking N bytes as a unit, a corresponding repair code is applied to each segmented B, then data C is generated, then a hash code C is generated for the data C, the hash code C is compared with the hash code A, and if the hash code C is the same as the hash code A, the repair is successful; if the codes are different, the restoration is failed, the step S6 is continuously executed, and the multiplication and redundancy are added when the restoration codes are generated.
The invention has the beneficial effects that:
the invention provides a data transmission optimization method and a system thereof, which can save bandwidth and avoid the occupation of bandwidth caused by transmission of redundant codes when each error occurs; and when errors occur occasionally, the repair codes are transmitted in increments which are much smaller than the original data packets.
Drawings
FIG. 1 is a schematic flow chart of the method of the present invention.
Fig. 2 is a system schematic block diagram of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
Referring to fig. 1, the present invention provides a data transmission optimization method, which includes the following steps:
S1, equipment A generates a hash code A for data A, and stores the hash code A serving as a key in a local disk;
Step S2, the equipment A transmits the data A to the equipment B;
step S3, after the equipment A finishes transmitting the data A, the equipment A sends a hash code A to the equipment B;
S4, after receiving the data A, the equipment B generates a hash code B for the data A;
S5, the equipment B compares the received hash code A with the hash code B, and if the received hash code A is the same as the hash code B, the data transmission is correct; if the data is different, indicating that the data is transmitted in error, storing the data A as the data B, and then executing the step S6;
And S6, the equipment A generates a repair code and sends the repair code to the equipment B, and the equipment B repairs the data B according to the repair code after receiving the repair code.
The invention is further described with reference to the following specific examples:
A method for optimizing the transmission of data,
Step 1, equipment A generates a hash code A for data A, and stores the hash code A serving as a key in a local disk; the data A is larger than 2000 bytes;
1. the device A generates a hash code A for the data A, and the hash code A is used as a key to be stored in a local disk;
Device a generates its md5 value 97ba9c6563136b6116029d6b50f6ef63 for data beginning at abcdabcd followed by all \x00 of the specified 2MB size, and saves it to the local disk with 97ba9c6563136b6116029d6b50f6ef63 as a key.
Step 2, the equipment A transmits the data A to the equipment B;
2. The device A transmits the data to the device B;
device a sends data beginning at abcdabcd specifying a 1MB size to device B;
step 3, after the equipment A finishes transmitting the data A, sending the hash code A to the equipment B;
3. After the data transmission is completed, the device A sends a hash code A to the device B;
after the data transmission is completed, the device 1 sends 97ba9c6563136b6116029d6b50f6ef63 to the device 2;
step 4, after the equipment B receives the data A, generating a hash code B for the data A;
4. After the equipment 2 receives the data, the same algorithm is used for generating a hash code B;
the device 2 receives the data abcdabcd.
Step 5, the equipment B compares the received hash code A with the hash code B, and if the received hash code A is the same as the hash code B, the data transmission is correct; if the data is different, indicating that the data is transmitted in error, storing the data A as the data B, and then executing the step S6;
5. the device 2 compares the hash code A with the hash code B, and if the hash codes A and B are the same, the data transmission is correct;
both md5 are 97ba9c6563136b6116029d6b50f6ef63 indicating correct transmission;
otherwise, a data transmission error (here saved as error data 1) is specified;
If device B receives 1bcdabcd, the same algorithm generates the md5 value da5887aa5f0ea7fcbca33293c90f064a, not equal to 97ba9c6563136B6116029d6B50f6ef63 from device 1.
And 6, the equipment A generates a repair code and sends the repair code to the equipment B, and the equipment B repairs the data B according to the repair code after receiving the repair code.
If the data transmission errors are different, the data A is stored as data B, and the device B sends a request repair code instruction and the hash code A to the device A.
The device A searches the data A according to the hash code A, generates a repair code with the redundancy of 1% from the data A, and then sends the repair code to the device B.
The method comprises the steps of dividing data A into blocks by taking N bytes as a unit, and generating a repair code with redundancy of 1% for each block A; the N is greater than 200 bytes.
The data B is segmented by taking N bytes as a unit, a corresponding repair code is applied to each segmented B, then data C is generated, then a hash code C is generated for the data C, the hash code C is compared with the hash code A, and if the hash code C is the same as the hash code A, the repair is successful; if the codes are different, the restoration is failed, the step S6 is continuously executed, and the multiplication and redundancy are added when the restoration codes are generated.
6. The device 2 sends a repair code request instruction and a data hash code to the device 1;
Device 2 sends repair codes repair and 97ba9c6563136b6116029d6b50f6ef63 to device 1;
The device 1 searches corresponding data according to the hash code, generates a repair code with redundancy of 1% by reed-solomon encoding of the original data, and sends the repair code to the device 2;
the device 1 finds abcdbcd data from 97ba9c6563136b6116029d6b50f6ef63, generates a repair with redundancy of 1%;
Partitioning data according to N bytes, and generating 1 repair code for each partition;
according to 200 byte blocks, the first block
abcdabcd\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00;
Generating a repair code for a first block
\x27\xa7\x0f\x8f；
Second block
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00;
Repair code
\x00\x00\x00\x00；
Transmitting the repair code to the device 2;
The device 2 receives the repair code, applies the repair code to the error data 1 to generate original data, generates a hash code C, compares the hash code C with the hash code, and if the hash code C is correct, the repair is successfully finished. Otherwise repeating steps 7 to 9, doubling and adding redundancy (2%, 4%,8%. 256%).
The device 2 receives the data to repair each line, and the repair code is spliced on the first line
1bcdabcd\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x27\xa7\x0f\x8f;
Repair by reed-solomon algorithm
abcdabcd\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00;
After the later lines are all without errors and are repaired, the whole data is regenerated to have md5 of 97ba9c6563136b6116029d6b50f6ef63, the comparison is correct, the repair is completed, and otherwise, the redundancy is increased by repeating the step 6.
With continued reference to fig. 2, the present invention further provides a data transmission optimization system, where the system includes a generating module a, a transmitting module, a sending module, a generating module B, a comparing module, and a repairing module;
Generating a module A: the device A generates a hash code A for the data A, and stores the hash code A as a key in a local disk;
and a transmission module: the device A transmits the data A to the device B;
And a sending module: after the equipment A finishes transmitting the data A, sending the hash code A to the equipment B;
generating a module B: after receiving the data A, the equipment B generates a hash code B for the data A;
And a comparison module: the equipment B compares the received hash code A with the hash code B, and if the received hash code A and the hash code B are the same, the data transmission is correct; if the data is different, indicating that the data is transmitted in error, storing the data A as the data B, and then executing the step S6;
and (3) a repair module: and the equipment A generates a repair code and sends the repair code to the equipment B, and the equipment B repairs the data B according to the repair code after receiving the repair code.
Preferably, the repair module further specifically includes: if the data transmission errors are different, the data A is stored as data B, and the device B sends a request repair code instruction and the hash code A to the device A.
Preferably, the repair module further specifically includes: the device A searches the data A according to the hash code A, generates a repair code with the redundancy of 1% from the data A, and then sends the repair code to the device B.
Preferably, the repair code is generated by:
the method comprises the steps of dividing data A into blocks by taking N bytes as a unit, and generating a repair code with redundancy of 1% for each block A; the N is greater than 200 bytes.
Preferably, in the generating module a, the data a is greater than 2000 bytes.
Preferably, the repair module further specifically includes: the data B is segmented by taking N bytes as a unit, a corresponding repair code is applied to each segmented B, then data C is generated, then a hash code C is generated for the data C, the hash code C is compared with the hash code A, and if the hash code C is the same as the hash code A, the repair is successful; if the codes are different, the restoration is failed, the step S6 is continuously executed, and the multiplication and redundancy are added when the restoration codes are generated.
In a word, the invention can save the bandwidth and avoid the occupation of the bandwidth caused by the transmission of redundant codes when each error occurs; and when errors occur occasionally, the repair codes are transmitted in increments which are much smaller than the original data packets.
The above description is only of the preferred embodiments of the present application, and should not be construed as limiting the application, but rather as covering all equivalent variations and modifications according to the appended claims.
Claims (12)
1. A data transmission optimization method is characterized in that: the method comprises the following steps:
S1, equipment A generates a hash code A for data A, and stores the hash code A serving as a key in a local disk;
Step S2, the equipment A transmits the data A to the equipment B;
step S3, after the equipment A finishes transmitting the data A, the equipment A sends a hash code A to the equipment B;
S4, after receiving the data A, the equipment B generates a hash code B for the data A;
S5, the equipment B compares the received hash code A with the hash code B, and if the received hash code A is the same as the hash code B, the data transmission is correct; if the data is different, indicating that the data is transmitted in error, storing the data A as the data B, and then executing the step S6;
And S6, the equipment A generates a repair code and sends the repair code to the equipment B, and the equipment B repairs the data B according to the repair code after receiving the repair code.
2. A data transmission optimization method according to claim 1, characterized in that: the step S6 is further specifically: if the data transmission errors are different, the data A is stored as data B, and the device B sends a request repair code instruction and the hash code A to the device A.
3. A data transmission optimization method according to claim 2, characterized in that: the step S6 is further specifically: the device A searches the data A according to the hash code A, generates a repair code with the redundancy of 1% from the data A, and then sends the repair code to the device B.
4. A data transmission optimization method according to claim 3, characterized in that: the repair code is generated by:
the method comprises the steps of dividing data A into blocks by taking N bytes as a unit, and generating a repair code with redundancy of 1% for each block A; the N is greater than 200 bytes.
5. A data transmission optimization method according to claim 1, characterized in that: in the step S1, the data a is greater than 2000 bytes.
6. The data transmission optimization method according to claim 4, wherein: the step S6 is further specifically: the data B is segmented by taking N bytes as a unit, a corresponding repair code is applied to each segmented B, then data C is generated, then a hash code C is generated for the data C, the hash code C is compared with the hash code A, and if the hash code C is the same as the hash code A, the repair is successful; if the codes are different, the restoration is failed, the step S6 is continuously executed, and the multiplication and redundancy are added when the restoration codes are generated.
7. A data transmission optimization system, characterized by: the system comprises a generating module A, a transmission module, a sending module, a generating module B, a comparison module and a repairing module;
Generating a module A: the device A generates a hash code A for the data A, and stores the hash code A as a key in a local disk;
and a transmission module: the device A transmits the data A to the device B;
And a sending module: after the equipment A finishes transmitting the data A, sending the hash code A to the equipment B;
generating a module B: after receiving the data A, the equipment B generates a hash code B for the data A;
And a comparison module: the equipment B compares the received hash code A with the hash code B, and if the received hash code A and the hash code B are the same, the data transmission is correct; if the data is different, indicating that the data is transmitted in error, storing the data A as the data B, and then executing the step S6;
and (3) a repair module: and the equipment A generates a repair code and sends the repair code to the equipment B, and the equipment B repairs the data B according to the repair code after receiving the repair code.
8. A data transmission optimization system in accordance with claim 7, wherein: the repair module is further specifically: if the data transmission errors are different, the data A is stored as data B, and the device B sends a request repair code instruction and the hash code A to the device A.
9. A data transmission optimization system in accordance with claim 8, wherein: the repair module is further specifically: the device A searches the data A according to the hash code A, generates a repair code with the redundancy of 1% from the data A, and then sends the repair code to the device B.
10. A data transmission optimization system in accordance with claim 9, wherein: the repair code is generated by:
the method comprises the steps of dividing data A into blocks by taking N bytes as a unit, and generating a repair code with redundancy of 1% for each block A; the N is greater than 200 bytes.
11. A data transmission optimization system in accordance with claim 7, wherein: in the generating module a, the data a is greater than 2000 bytes.
12. A data transmission optimization system in accordance with claim 7, wherein: the repair module is further specifically: the data B is segmented by taking N bytes as a unit, a corresponding repair code is applied to each segmented B, then data C is generated, then a hash code C is generated for the data C, the hash code C is compared with the hash code A, and if the hash code C is the same as the hash code A, the repair is successful; if the codes are different, the restoration is failed, the step S6 is continuously executed, and the multiplication and redundancy are added when the restoration codes are generated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410294968.3A CN118199800A (en) | 2024-03-15 | 2024-03-15 | Data transmission optimization method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410294968.3A CN118199800A (en) | 2024-03-15 | 2024-03-15 | Data transmission optimization method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118199800A true CN118199800A (en) | 2024-06-14 |
Family
ID=91406155
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410294968.3A Pending CN118199800A (en) | 2024-03-15 | 2024-03-15 | Data transmission optimization method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118199800A (en) |
-
2024
- 2024-03-15 CN CN202410294968.3A patent/CN118199800A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US4833679A (en) | Method and apparatus with improved error correction and error information availability | |
| CN111697976B (en) | RS erasure correcting quick decoding method and system based on distributed storage | |
| CN113489570B (en) | Data transmission method, device and equipment of PCIe link | |
| US6327688B1 (en) | Data bus with automatic data integrity verification and verification method | |
| CN115454712B (en) | Check code recovery method, system, electronic equipment and storage medium | |
| CN111858169A (en) | Data recovery method, system and related components | |
| CN111444042A (en) | Block chain data storage method based on erasure codes | |
| CN114816837A (en) | Erasure code fusion method and system, electronic device and storage medium | |
| CN114741231B (en) | Data reading and writing method, device, equipment and storage medium based on memory | |
| US20080133967A1 (en) | Distributed object sharing system and method thereof | |
| US6098194A (en) | Detecting memory problems in computers | |
| US20040117549A1 (en) | Control method for distributed storage system | |
| US20170077952A1 (en) | Sensor interface that provides a long package crc to improve functional safety | |
| US5752251A (en) | Method and apparatus for recovering aborted file (or data) transmission | |
| CN101938280A (en) | Coding and decoding method and codec of error correction code | |
| CN118199800A (en) | Data transmission optimization method and system | |
| US20180122494A1 (en) | Raid decoding architecture with reduced bandwidth | |
| CN111414277B (en) | Data recovery method and device, electronic equipment and medium | |
| CN115129509B (en) | Data transmission method, device and medium | |
| CN110231999B (en) | Method and device for improving reliability of storage system based on local repair coding | |
| US11277154B2 (en) | Polar code-based interleaving method and communication apparatus | |
| CN108762975B (en) | ECC data storage method, system and storage medium | |
| US6941503B1 (en) | Method and apparatus for retaining error-control code protection across block-size discontinuities | |
| JP2000349792A (en) | Data transmission device | |
| CN111858126B (en) | Data processing method and device based on K + M erasure cluster |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |