CN112491989A - Data transmission method, device, equipment and storage medium - Google Patents
Data transmission method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112491989A CN112491989A CN202011282966.0A CN202011282966A CN112491989A CN 112491989 A CN112491989 A CN 112491989A CN 202011282966 A CN202011282966 A CN 202011282966A CN 112491989 A CN112491989 A CN 112491989A
- Authority
- CN
- China
- Prior art keywords
- data
- file
- data transmission
- original
- original data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/12—Applying verification of the received information
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/12—Applying verification of the received information
- H04L63/123—Applying verification of the received information received data contents, e.g. message integrity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Detection And Prevention Of Errors In Transmission (AREA)
Abstract
The invention relates to the technical field of network security, and discloses a data transmission method, a data transmission device, data transmission equipment and a storage medium. The method utilizes an MD5 algorithm to carry out encryption verification on service data to be transmitted, in the process of carrying out encryption by utilizing an MD5 algorithm, the service data are converted into an MD5 file according to a set data filling rule corresponding to the MD5 algorithm, then the MD5 file and the service data are transmitted, after a receiving end receives the service data, data in an MD5 file are extracted for encryption calculation to obtain a verification value, and the verification value is compared with an original encryption value, so that the comparison of the integrity and consistency of the data is realized, the safety of the data is improved, the data loss is avoided, and the generation problem is reduced. In addition, the invention also relates to a block chain technology, and the service data and the MD5 file can be stored in the block chain.
Description
Technical Field
The present application relates to the field of network security technologies, and in particular, to a data transmission method, apparatus, device, and storage medium.
Background
With the development of internet technology, especially in the application of file data transmission, network transmission has been the main means, and after the file data is transmitted, the received data is generally checked to avoid data inconsistency caused by transmission or thought modification.
At present, data is mainly prevented from being modified in a transmission process by using a data encryption mode, for example, when both parties are business systems, business data between the two systems can be transmitted or synchronized at regular time, such as deposit information, repayment plan, repayment, overdue repayment and other data, the data usually interacts with banks in a file mode due to large data volume, but a batch file mode is often adopted, and each file is possibly very large, so that not only the transmission time is too long, but also the transmission failure condition exists, if the condition exists, none of the final states represents whether the file is uploaded successfully or transmitted successfully, the data is incomplete, and in subsequent verification, only whether the data is consistent can be verified, and whether the data is complete cannot be verified, thereby missing data occurs and even causes production problems.
Disclosure of Invention
The invention mainly aims to solve the technical problem that the simultaneous identification of the integrity and consistency of data cannot be realized in the traditional data verification scheme.
A first aspect of the present invention provides a data transmission method, where the data transmission method includes:
reading a data transmission file to be verified from a server in a service system, wherein the data transmission file comprises an original data file and a first verification value;
extracting an original data file in the data transmission file, calling a preset data packaging rule, and extracting an original data group from the original data file;
calculating a second check value of the original data set according to a message digest algorithm;
comparing the first check value with the second check value to obtain a comparison result;
and if the comparison result is inconsistent, determining that the data transmission file is incomplete and the original data file is inconsistent, and informing an output end of resending the data.
Optionally, in a first implementation manner of the first aspect of the present invention, before reading the data transmission file to be verified from the server in the service system, the method further includes:
acquiring data to be transmitted in the service system;
grouping and packaging the data to be transmitted according to the data packaging rule to obtain an original data file, and calculating a first check value of the data in the original data file through the message digest algorithm;
and synthesizing the first check value and the data transmission file into a data transmission file, and sending the data transmission file to the server for storage.
Optionally, in a second implementation manner of the first aspect of the present invention, the grouping and encapsulating the data to be transmitted according to the data encapsulation rule to obtain an original data file includes:
dividing the data to be transmitted into a plurality of data sections according to the bit length of a data block specified in the data encapsulation rule, wherein the data sections comprise N original data sections with the length equal to the bit length and 1 original data residual block, and N is a natural number;
according to a data filling algorithm in the data encapsulation rule, expanding the original data residual block to obtain an expanded data segment with the length equal to the bit length;
averagely dividing the plurality of data segments into M groups, and adding a data length record field behind each data segment in the M groups of original data groups to obtain a complete data block;
and grouping and cascading all the data blocks to obtain the original data file.
Optionally, in a third implementation manner of the first aspect of the present invention, the calculating, by the message digest algorithm, a first check value of data in the original data file includes:
extracting an endian type adopted by a set message digest algorithm in the data encapsulation rule;
judging whether the endian type is the same as the endian type in the currently used message digest algorithm;
if the M original data groups are the same, directly calculating the sum of the check values of the M original data groups to obtain a first check value;
if not, reversing the byte order of the M original arrays according to 32-bit bytes, and calculating the checksum of the check value after the byte order is to be reversed to obtain a first check value.
Optionally, in a fourth implementation manner of the first aspect of the present invention, if the bit length of the data block is 512 bits, the dividing the data to be transmitted into a plurality of data segments according to the bit length of the data block specified in the data encapsulation rule includes:
the data to be transmitted is complemented with 512 to obtain a calculation result;
and judging the quotient value in the calculation result, and dividing the data to be transmitted into original data segments with the number equal to the quotient value and 1 original data residual block.
Optionally, in a fifth implementation manner of the first aspect of the present invention, the calculating a second check value of the original data set according to a message digest algorithm includes:
calculating a second syndrome value of each data segment in the raw data set according to the message digest algorithm;
and calculating a second check value sum of the original data group based on the check value of each data segment.
Optionally, in a sixth implementation manner of the first aspect of the present invention, the comparing the first check value with the second check value to obtain a comparison result includes:
comparing a first checksum in the first check value with the second checksum for comparison;
and if the comparison is inconsistent, comparing the second check sub value with a first check sub value in the first check sub value, and outputting a comparison result.
A second aspect of the present invention provides a data transmission apparatus, including:
the system comprises a reading module, a verification module and a verification module, wherein the reading module is used for reading a data transmission file to be verified from a server in a service system, and the data transmission file comprises an original data file and a first verification value;
the extraction module is used for extracting an original data file in the data transmission file, calling a preset data packaging rule and extracting an original data group from the original data file;
the calculation module is used for calculating a second check value of the original data set according to a message digest algorithm;
the comparison module is used for comparing the first check value with the second check value to obtain a comparison result;
and the notification module is used for determining that the data transmission file is incomplete and the original data file is inconsistent when the comparison result is inconsistent, and notifying an output end of resending the data.
Optionally, in a first implementation manner of the second aspect of the present invention, the data transmission apparatus further includes a storage module, which is specifically configured to
Acquiring data to be transmitted in the service system;
grouping and packaging the data to be transmitted according to the data packaging rule to obtain an original data file, and calculating a first check value of the data in the original data file through the message digest algorithm;
and synthesizing the first check value and the data transmission file into a data transmission file, and sending the data transmission file to the server for storage.
Optionally, in a second implementation manner of the second aspect of the present invention, the storage module is specifically configured to:
dividing the data to be transmitted into a plurality of data sections according to the bit length of a data block specified in the data encapsulation rule, wherein the data sections comprise N original data sections with the length equal to the bit length and 1 original data residual block, and N is a natural number;
according to a data filling algorithm in the data encapsulation rule, expanding the original data residual block to obtain an expanded data segment with the length equal to the bit length;
averagely dividing the plurality of data segments into M groups, and adding a data length record field behind each data segment in the M groups of original data groups to obtain a complete data block;
and grouping and cascading all the data blocks to obtain the original data file.
Optionally, in a third implementation manner of the second aspect of the present invention, the storage module is specifically configured to:
extracting an endian type adopted by a set message digest algorithm in the data encapsulation rule;
judging whether the endian type is the same as the endian type in the currently used message digest algorithm;
if the M original data groups are the same, directly calculating the sum of the check values of the M original data groups to obtain a first check value;
if not, reversing the byte order of the M original arrays according to 32-bit bytes, and calculating the checksum of the check value after the byte order is to be reversed to obtain a first check value.
Optionally, in a fourth implementation manner of the second aspect of the present invention, the storage module is specifically configured to:
if the bit length of the data block is 512 bits, the data to be transmitted is subjected to remainder calculation on 512 bits to obtain a calculation result;
and judging the quotient value in the calculation result, and dividing the data to be transmitted into original data segments with the number equal to the quotient value and 1 original data residual block.
Optionally, in a fifth implementation manner of the second aspect of the present invention, the calculation module includes:
a first calculating unit, configured to calculate a second syndrome value of each data segment in the raw data set according to the message digest algorithm;
and the second calculation unit is used for calculating a second check value sum of the original data group based on the check value of each data segment.
Optionally, in a sixth implementation manner of the second aspect of the present invention, the comparing module includes:
the comparison unit is used for comparing a first check sum in the first check value with the second check sum to carry out comparison;
and the output unit is used for comparing the second check sub value with the first check sub value in the first check value and outputting a comparison result when the first check sum is compared with the second check sum to be inconsistent.
A third aspect of the present invention provides a data transmission device, including: a memory having instructions stored therein and at least one processor, the memory and the at least one processor interconnected by a line;
the at least one processor invokes the instructions in the memory to cause the data transfer device to perform the data transfer method described above.
A fourth aspect of the present invention provides a computer-readable storage medium having stored therein a computer program which, when run on a computer, causes the computer to execute the above-described data transmission method.
In the technical scheme provided by the invention, the MD5 encryption and verification processing of the file can ensure that the file is more reliable and the content of the file is safer. The situations of data omission and data loss can be avoided, the production problem and the cost of the positioning problem of developers are reduced, and the similar problems can be solved by using the solution aiming at the data completeness and the safety.
Drawings
Fig. 1 is a first flowchart illustrating a data transmission method according to an embodiment of the present invention;
fig. 2 is a second flowchart illustrating a data transmission method according to an embodiment of the invention;
fig. 3 is a third flowchart illustrating a data transmission method according to an embodiment of the invention;
FIG. 4 is a schematic diagram of an embodiment of a data transmission apparatus according to the present invention;
fig. 5 is a schematic diagram of another embodiment of a data transmission apparatus according to an embodiment of the present invention;
fig. 6 is a schematic diagram of an embodiment of a data transmission device in an embodiment of the present invention.
Detailed Description
In order to solve the above problems, the present application provides a method for performing encryption verification on service data to be transmitted by using an MD5 algorithm, and in the process of performing encryption by using an MD5 algorithm, the service data is converted into an MD5 file according to a set data filling rule corresponding to the MD5 algorithm, then the MD5 file and the service data are transmitted, and after receiving the service data, a receiving end extracts data in the MD5 file for encryption calculation to obtain a verification value, and compares the verification value with an original encryption value, thereby implementing comparison between integrity and consistency of the data, improving security of the data, avoiding data loss, and reducing generation problems.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
For convenience of understanding, a specific flow of the embodiment of the present invention is described below, and referring to fig. 1, a first embodiment of a data transmission method in the embodiment of the present invention includes:
101. reading an original data file and a first check value in a data transmission file to be checked from a server in a service system;
in this step, the data transmission file to be verified may be service data to be transmitted by the service system at the current time, or may be service data synchronized to other service systems or terminals in the service system.
In particular, the files stored in the server are stored in a specific data file or data format, and in practical applications, for the data stored in the server, the service system modifies and converts the data according to the storage requirement of the server to form a data transmission file, and then sends the data transmission file to the server for storage, wherein the modification conversion of the data to be transmitted by the service system is realized by determining the types of the server and the server corresponding to the service system, determining the data storage format of the server based on the types, the storage format can be a storage file template and a data format, data preprocessing is carried out on data to be transmitted in the service system based on the data storage format, a data transmission file is obtained, and the data transmission file is synchronized to the server.
When the data storage format is a storage file template, the preprocessing specifically comprises the steps of extracting data contents in the data to be transmitted according to the storage file template, filling the data contents in the data to be transmitted into corresponding items in the storage file template, and generating corresponding data transmission files; further, the check value of the original data is calculated according to the storage format of the data in the data transmission file, specifically, the check value is calculated by using a message digest algorithm, while the check value is calculated, the original data can be divided into a plurality of data segments to be calculated to obtain a plurality of syndrome values, and then the total check value is calculated based on the plurality of syndrome values to obtain the first check value.
In this embodiment, after the data transmission file is obtained from the server, the data analysis of the data transmission file is further included, specifically, the original data file carrying the original data and the first check value are extracted, and of course, in practical application, if the service system performs the segmentation processing on the data, the method further includes extracting the first check value and the syndrome value in the data.
102. Extracting an original data file in the data transmission file, calling a preset data packaging rule, and extracting an original data group from the original data file;
in this step, the data encapsulation rule refers to a data conversion method or algorithm that is used by the service system when uploading data to the server, specifically, when the service system uploads data, the data encapsulation rule is added to the data transmission file to be uploaded together, but encryption processing needs to be performed, the encryption and decryption algorithm is obtained by performing protocol handshake between the service system and other systems, when extracting the original data file, the cryptograph of the data encapsulation rule in the original data file is extracted, then decryption is performed based on the decryption algorithm of the protocol handshake to obtain the data encapsulation rule, and then data carried in the original data file is extracted based on the data encapsulation rule to obtain the original data set.
In this embodiment, when the data encapsulation rule is segmented data processing, when data is extracted, according to an encapsulation manner of the segmented data, first, each segment of data in the original data file is identified, then, an encapsulation structure in each segment of data is extracted and deleted, so as to obtain an original data segment, and finally, all the original data segments are connected in series, so as to obtain an original data group.
103. Calculating a second check value of the original data set according to a message digest algorithm;
in this step, when calculating the second check value, the calculation method is the same as the calculation method of the first check value in the service system, and specifically, the calculation may be performed in a packet manner:
firstly, the original data group is grouped in a 512-bit grouping format, and a value consisting of four 32-bit groups is output;
then, the second check value is obtained after the values composed of the four 32-bit packets are concatenated.
In practical applications, the MD5 processes the input information in 512-bit packets, each packet is divided into 16 32-bit sub-packets, and after a series of processing, the output of the algorithm is composed of four 32-bit packets, and after the four 32-bit packets are concatenated, a 128-bit hash value is generated, and the data of the 128-bit hash value is the second check data and is stored in the original data file.
104. Comparing the first check value with the second check value to obtain a comparison result;
in this embodiment, the comparison result includes two types, which are respectively inconsistent and consistent, and when the comparison result is inconsistent, if the calculation method of the first check value is a data segment type, the syndrome values of the data segments need to be compared.
In practical application, the original data stored in the original data file can be a plurality of complete data, each complete data is correspondingly calculated to obtain a syndrome value, then a total value is calculated from all the syndrome values to serve as a first check value, and a second check value is calculated in the checking process, so that the integrity of single data is verified, the transmission of a plurality of complete data in another data transmission file is realized, and the data processing efficiency is greatly improved.
105. And if the comparison result is inconsistent, determining that the data transmission file is incomplete and the original data file is inconsistent, and informing the output end of resending the data.
In the step, if the comparison result is inconsistent, before notifying to resend the data, judging whether the data is composite data or not, wherein the composite data refers to the fact that an original data file contains a plurality of complete data, if the comparison result is yes, identifying each section of data recorded in the original data file, calculating a syndrome value based on each section of data, comparing the syndrome value with a syndrome value stored in a data transmission file, determining the data section with incomplete data according to the comparison result, generating a data updating request based on specific identification information and the verification result of the data section, returning the data updating request to an output end, and intercepting the corresponding data section according to the data updating request by the output end to be synchronized into the server.
By executing the method provided by the invention, the MD5 encryption and verification processing of the file can ensure that the file is more reliable and the content of the file is more secure. The situations of data omission and data loss can not occur any more, the production problem and the cost of the positioning problem of developers are reduced, and the similar problems can be solved by using the solution aiming at the data completeness and the safety.
Referring to fig. 2, a second embodiment of a data transmission method according to the embodiment of the present invention includes:
201. acquiring data to be transmitted in a service system;
202. grouping and packaging data to be transmitted according to a data packaging rule to obtain an original data file, and calculating a first check value of the data in the original data file through a message digest algorithm;
in this embodiment, grouping the data to be transmitted according to the data packaging rule for packet packaging, and obtaining the original data file includes:
dividing the data to be transmitted into a plurality of data sections according to the bit length of a data block specified in the data encapsulation rule, wherein the data sections comprise N original data sections with the length equal to the bit length and 1 original data residual block, and N is a natural number;
according to a data filling algorithm in the data encapsulation rule, expanding the original data residual block to obtain an expanded data segment with the length equal to the bit length;
averagely dividing the plurality of data segments into M groups, and adding a data length record field behind each data segment in the M groups of original data groups to obtain a complete data block;
and grouping and cascading all the data blocks to obtain the original data file.
203. Synthesizing the first check value and the data transmission file into a data transmission file, and sending the data transmission file to a server for storage;
204. reading an original data file and a first check value in a data transmission file to be checked from a server in a service system;
205. extracting an original data file in the data transmission file, calling a preset data packaging rule, and extracting an original data group from the original data file;
206. calculating a second check value of the original data set according to a message digest algorithm;
in this step, the original data stored in the server is essentially the data backed up from the service system, in practical application, the service system and the server are a pair of systems for realizing mutual authentication, the service system and the server are connected with each other through a data interface, and the server is controlled by a timer to read and store the data from the service system at regular time; or the server can perform cleaning and replacing on the data in the server every time the server reads the data from the service system.
207. Comparing the first check value with the second check value to obtain a comparison result;
208. and if the comparison result is inconsistent, determining that the data transmission file is incomplete and the original data file is inconsistent, and informing the output end of resending the data.
When the service system sends the data to be transmitted to the server, the data to be transmitted can be fragmented one by one, and the fragmented data blocks are transmitted according to the sequence of the stored timestamps.
The integrity comparison of the data in the server is realized by combining the MD5 algorithm to calculate the check value in the above mode, and the time length of data comparison is greatly saved, so that the comparison of the integrity and consistency of the data is realized, the safety of the data is improved, the data loss is avoided, and the generation problem is reduced.
Referring to fig. 3, another embodiment of a data transmission method according to the embodiment of the present invention includes:
301. acquiring data to be transmitted in a service system;
302. dividing data to be transmitted into a plurality of data segments according to the bit length of a data block specified in a data encapsulation rule;
the data segments comprise N original data segments with the length equal to the bit length and 1 original data residual block, wherein N is a natural number;
303. according to a data filling algorithm in a data encapsulation rule, expanding the residual blocks of the original data to obtain an expanded data section with the length equal to the bit length;
304. averagely dividing a plurality of data segments into M groups, and adding a data length record field behind each data segment in the M groups of original data groups to obtain a complete data block;
305. grouping and cascading all the data blocks to obtain an original data file;
in practical application, the specific implementation process for calculating the first check value by using the MD5 algorithm is as follows:
first step, filling: if the length of the input information (bit) is not equal to 448 as a result of the remainder for 512, padding is required so that the result of the remainder for 512 is equal to 448. The method of padding is to pad one 1 and n 0. After filling, the length of the information is N × 512+448 (bit);
in practical application, the data extracted from the original data file is subjected to packet filling processing according to a data packaging rule, specifically, the bit length of the data is subjected to complementation for 512 and 448, and filling is performed according to the complementation result, namely, the result of complementation of 512 by the bit length is equal to 448. The bit Length (Bits Length) of the last data is extended to N512 +448, N being a non-negative integer, N may be zero. The filling method comprises the following steps: one 1 and an infinite number of 0's are filled in behind the data, and the filling of the data with 0's is not stopped until the above condition is satisfied.
Step two, recording information length: the pre-padding information length is stored with 64 bits. These 64 bits are added after the result of the first step, so that the information length becomes N512 +448+64 ═ N +1 ═ 512 bits.
Specifically, a pre-padding data length (unit is Bit) expressed in 64-Bit binary is added to the end of the result after the grouping, and if the pre-padding data length expressed in binary exceeds 64 bits, the lower 64 bits are taken to realize the recording of the length of the data.
After the two steps of processing, the bit length of the data is N × 512+448+64 (N +1) × 512, i.e., the length is exactly an integer multiple of 512.
Thirdly, loading standard magic numbers;
specifically, an initial 128-bit hash sequence is used as an initial chaining variable, and these parameters are used in the first round of operation, and are expressed in big-endian, which are:
A＝0x01234567，
B＝0x89ABCDEF，
C＝0xFEDCBA98，
D＝0x76543210。
(Each variable gives a value of 0x67452301, 0xEFCDAB89, 0x98BADCFE, 0x10325476 for the value of variable A, B, C, D in the program).
Step four, four-wheel cycle operation: the number of cycles is the number of packets (N + 1).
In practical applications, the algorithm flow for each packet is as follows:
(1) the first packet requires the copying of the top four chaining variables into four other variables: a to a, B to B, C to C, D to D.
(2) The variables from the second packet are the result of the operation of the previous packet, i.e., a, B, C, and D.
Further, the main cycle has four cycles (MD4 has only three cycles), and each cycle is similar. The first round performed 16 operations. Each operation performs a non-linear function operation on three of a, b, c, and d, and then adds a fourth variable, a sub-group of text and a constant, to the result. The result is then shifted to the left by an indefinite number and added to one of a, b, c or d. Finally, one of a, b, c or d is replaced by the result.
An MD5 operation consists of similar 64 cycles, divided into 4 groups of 16.
306. Calculating a first check value of data in the original data file through a message digest algorithm;
extracting an endian type adopted by a set message digest algorithm in the data encapsulation rule;
judging whether the endian type is the same as the endian type in the currently used message digest algorithm;
if the M original data groups are the same, directly calculating the sum of the check values of the M original data groups to obtain a first check value;
if not, reversing the byte order of the M original arrays according to 32-bit bytes, and calculating the checksum of the check value after the byte order is to be reversed to obtain a first check value.
In this embodiment, when the first check value or the second check value of the data is calculated by using the message digest algorithm, the method specifically includes selecting a corresponding calculation mode according to an actual calculation requirement, in an actual application, the message digest algorithm includes two applications, where the two applications are divided according to an endian, and the endian includes a small endian and a large endian, and the specific implementation steps include:
identifying a data security level on the server, and determining an endian type based on the data security level;
judging whether the endian type is consistent with the current message digest algorithm, if so, directly calculating the MD5 checksum of the N64-Byte original data packets, if not, firstly reversing the Byte order of the N64-Byte original data packets according to 32-bit bytes, and then calculating the MD5 checksum;
further, after determining the endian type of the server, judging whether the server is a small endian calculation type server, if so, combining 1 original data residual block and 1 extended data block, grouping according to 64Byte, and continuing to calculate to obtain a final MD5 checksum, if not, combining 1 original data residual block and 1 extended data block, grouping according to 64Byte, performing Byte order reversal on each obtained grouping according to 32 bit bytes, and then calculating an MD5 checksum to obtain a final MD5 value.
Further, if the bit length of the data block is 512 bits, the dividing the data to be transmitted into a plurality of data segments according to the bit length of the data block specified in the data encapsulation rule includes:
the data to be transmitted is complemented with 512 to obtain a calculation result;
and judging the quotient value in the calculation result, and dividing the data to be transmitted into original data segments with the number equal to the quotient value and 1 original data residual block.
307. Synthesizing the first check value and the data transmission file into a data transmission file, and sending the data transmission file to a server for storage;
308. reading an original data file and a first check value in a data transmission file to be checked from a server in a service system;
309. extracting an original data file in the data transmission file, calling a preset data packaging rule, and extracting an original data group from the original data file;
310. calculating a second check value of the original data set according to a message digest algorithm;
calculating a second syndrome value of each data segment in the original data set according to the message digest algorithm;
and calculating a second check value sum of the original data group based on the check value of each data segment.
311. Comparing the first check value with the second check value to obtain a comparison result;
in this step, comparing a first checksum in the first check value with the second checksum for comparison;
and if the comparison is inconsistent, comparing the second check sub value with a first check sub value in the first check sub value, and outputting a comparison result.
312. And if the comparison result is inconsistent, determining that the data transmission file is incomplete and the original data file is inconsistent, and informing the output end of resending the data.
Through the implementation of the scheme, the MD5 algorithm is used for carrying out encryption verification on the service data to be transmitted, in the process of carrying out encryption by using the MD5 algorithm, the service data are converted into the MD5 file according to the set data filling rule corresponding to the MD5 algorithm, then the MD5 file and the service data are transmitted, after the receiving end receives the service data, the data in the MD5 file are extracted for carrying out encryption calculation to obtain a verification value, and the verification value is compared with the original encryption value, so that the comparison of the integrity and the consistency of the data is realized, the safety of the data is improved, the data loss is avoided, and the generation problem is reduced.
With reference to fig. 4, the data transmission method in the embodiment of the present invention is described above, and the following describes the data transmission device in the embodiment of the present invention, where the first embodiment of the data transmission device in the embodiment of the present invention includes:
a reading module 401, configured to read a data transmission file to be verified from a server in a service system, where the data transmission file includes an original data file and a first verification value;
an extracting module 402, configured to extract an original data file in the data transmission file, and invoke a preset data encapsulation rule to extract an original data group from the original data file;
a calculating module 403, configured to calculate a second check value of the original data set according to a message digest algorithm;
a comparing module 404, configured to compare the first check value with the second check value to obtain a comparison result;
a notification module 405, configured to determine that the data transmission file is incomplete and the original data file is inconsistent when the comparison result is inconsistent, and notify an output end to resend the data.
In this embodiment, the data transmission apparatus runs the data transmission method, and the method can make the file more reliable and the content of the file more secure by performing MD5 encryption and verification on the file. The situations of data omission and data loss can be avoided, the production problem and the cost of the problem of positioning of developers are reduced, and the similar problems can be solved by using the solution aiming at the completeness and the safety of data.
Referring to fig. 5, a second embodiment of a data transmission device according to an embodiment of the present invention specifically includes:
a reading module 401, configured to read a data transmission file to be verified from a server in a service system, where the data transmission file includes an original data file and a first verification value;
an extracting module 402, configured to extract an original data file in the data transmission file, and invoke a preset data encapsulation rule to extract an original data group from the original data file;
a calculating module 403, configured to calculate a second check value of the original data set according to a message digest algorithm;
a comparing module 404, configured to compare the first check value with the second check value to obtain a comparison result;
a notification module 405, configured to determine that the data transmission file is incomplete and the original data file is inconsistent when the comparison result is inconsistent, and notify an output end to resend the data.
In this embodiment, the data transmission apparatus further includes a storage module 406, which is specifically configured to
Acquiring data to be transmitted in the service system;
grouping and packaging the data to be transmitted according to the data packaging rule to obtain an original data file, and calculating a first check value of the data in the original data file through the message digest algorithm;
and synthesizing the first check value and the data transmission file into a data transmission file, and sending the data transmission file to the server for storage.
Optionally, the storage module 406 is specifically configured to:
dividing the data to be transmitted into a plurality of data sections according to the bit length of a data block specified in the data encapsulation rule, wherein the data sections comprise N original data sections with the length equal to the bit length and 1 original data residual block, and N is a natural number;
according to a data filling algorithm in the data encapsulation rule, expanding the original data residual block to obtain an expanded data segment with the length equal to the bit length;
averagely dividing the plurality of data segments into M groups, and adding a data length record field behind each data segment in the M groups of original data groups to obtain a complete data block;
and grouping and cascading all the data blocks to obtain the original data file.
Optionally, the storage module 406 is specifically configured to:
extracting an endian type adopted by a set message digest algorithm in the data encapsulation rule;
judging whether the endian type is the same as the endian type in the currently used message digest algorithm;
if the M original data groups are the same, directly calculating the sum of the check values of the M original data groups to obtain a first check value;
if not, reversing the byte order of the M original arrays according to 32-bit bytes, and calculating the checksum of the check value after the byte order is to be reversed to obtain a first check value.
Optionally, the storage module 406 is specifically configured to:
if the bit length of the data block is 512 bits, the data to be transmitted is subjected to remainder calculation on 512 bits to obtain a calculation result;
and judging the quotient value in the calculation result, and dividing the data to be transmitted into original data segments with the number equal to the quotient value and 1 original data residual block.
In this embodiment, the calculating module 403 includes:
a first calculating unit 4031, configured to calculate a second syndrome value of each data segment in the raw data set according to the message digest algorithm;
a second calculating unit 4032, configured to calculate a second checksum of the original data set based on the checksum of each data segment.
In this embodiment, the comparing module 404 includes:
a comparing unit 4041, configured to compare a first checksum in the first check value with the second checksum for comparison;
an output unit 4042, configured to, when the first checksum and the second checksum are compared to be inconsistent, compare the second checksum value with the first checksum value in the first checksum value, and output a comparison result.
Fig. 4 and 5 describe the data transmission apparatus in the embodiment of the present invention in detail from the perspective of the modular functional entity, and the data transmission device in the embodiment of the present invention is described in detail from the perspective of hardware processing.
Fig. 6 is a schematic structural diagram of a data transmission apparatus 700 according to an embodiment of the present invention, where the data transmission apparatus 700 may have a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 710 (e.g., one or more processors) and a memory 720, and one or more storage media 730 (e.g., one or more mass storage devices) for storing applications 733 or data 732. Memory 720 and storage medium 730 may be, among other things, transient storage or persistent storage. The program stored in the storage medium 730 may include one or more modules (not shown), each of which may include a series of instructions operating on the data transmission device 700. Further, the processor 710 may be configured to communicate with the storage medium 730, and execute a series of instruction operations in the storage medium 730 on the data transmission device 700 to implement the steps of the data transmission method.
The data transmission apparatus 700 may also include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input-output interfaces 760, and/or one or more operating systems 731, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will appreciate that the data transfer device configuration shown in fig. 6 does not constitute a limitation of the data transfer devices provided herein, and may include more or fewer components than shown, or some components may be combined, or a different arrangement of components.
The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, and may also be a volatile computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the instructions are executed on a computer, the instructions cause the computer to perform the steps of the data transmission method provided in each of the above embodiments.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, which is used for verifying the validity of the information (anti-counterfeiting) and generating a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic or optical disk, or other various media capable of storing program codes.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A data transmission method, characterized in that the data transmission method comprises:
reading a data transmission file to be verified from a server in a service system, wherein the data transmission file comprises an original data file and a first verification value;
extracting an original data file in the data transmission file, calling a preset data packaging rule, and extracting an original data group from the original data file;
calculating a second check value of the original data set according to a message digest algorithm;
comparing the first check value with the second check value to obtain a comparison result;
and if the comparison result is inconsistent, determining that the data transmission file is incomplete and the original data file is inconsistent, and informing an output end of resending the data.
2. The data transmission method according to claim 1, further comprising, before the reading of the data transmission file to be verified from the server in the service system:
acquiring data to be transmitted in the service system;
grouping and packaging the data to be transmitted according to the data packaging rule to obtain an original data file, and calculating a first check value of the data in the original data file through the message digest algorithm;
and synthesizing the first check value and the data transmission file into a data transmission file, and sending the data transmission file to the server for storage.
3. The data transmission method according to claim 2, wherein the grouping and encapsulating the data to be transmitted according to the data encapsulation rule to obtain an original data file comprises:
dividing the data to be transmitted into a plurality of data sections according to the bit length of a data block specified in the data encapsulation rule, wherein the data sections comprise N original data sections with the length equal to the bit length and 1 original data residual block, and N is a natural number;
according to a data filling algorithm in the data encapsulation rule, expanding the original data residual block to obtain an expanded data segment with the length equal to the bit length;
averagely dividing the plurality of data segments into M groups, and adding a data length record field behind each data segment in the M groups of original data groups to obtain a complete data block;
and performing grouping and cascading on the data blocks to obtain the original data file.
4. The data transmission method of claim 3, wherein the calculating the first check value of the data in the original data file by the message digest algorithm comprises:
extracting an endian type adopted by a set message digest algorithm in the data encapsulation rule;
judging whether the endian type is the same as the endian type in the currently used message digest algorithm;
if the M original data groups are the same, directly calculating the sum of the check values of the M original data groups to obtain a first check value;
if not, reversing the byte order of the M original arrays according to 32-bit bytes, and calculating the checksum of the check value after the byte order is to be reversed to obtain a first check value.
5. The data transmission method according to claim 4, wherein if the bit length of the data block is 512 bits, the dividing the data to be transmitted into a plurality of data segments according to the bit length of the data block specified in the data encapsulation rule comprises:
the data to be transmitted is complemented with 512 to obtain a calculation result;
and judging the quotient value in the calculation result, and dividing the data to be transmitted into original data segments with the number equal to the quotient value and 1 original data residual block.
6. The data transmission method of claim 5, wherein said calculating a second parity value for said original data set according to a message digest algorithm comprises:
calculating a second syndrome value of each data segment in the raw data set according to the message digest algorithm;
and calculating a second check value sum of the original data group based on the check value of each data segment.
7. The data transmission method according to claim 6, wherein the comparing the first check value with the second check value to obtain a comparison result comprises:
comparing a first checksum in the first check value with the second checksum for comparison;
and if the comparison is inconsistent, comparing the second check sub value with a first check sub value in the first check sub value, and outputting a comparison result.
8. A data transmission apparatus, characterized in that the data transmission apparatus comprises:
the system comprises a reading module, a verification module and a verification module, wherein the reading module is used for reading a data transmission file to be verified from a server in a service system, and the data transmission file comprises an original data file and a first verification value;
the extraction module is used for extracting an original data file in the data transmission file, calling a preset data packaging rule and extracting an original data group from the original data file;
the calculation module is used for calculating a second check value of the original data set according to a message digest algorithm;
the comparison module is used for comparing the first check value with the second check value to obtain a comparison result;
and the notification module is used for determining that the data transmission file is incomplete and the original data file is inconsistent when the comparison result is inconsistent, and notifying an output end of resending the data.
9. A data transmission device, characterized in that the data transmission device comprises: a memory having instructions stored therein and at least one processor, the memory and the at least one processor interconnected by a line;
the at least one processor invokes the instructions in the memory to cause the data transfer device to perform the data transfer method of any of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the data transmission method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011282966.0A CN112491989A (en) | 2020-11-17 | 2020-11-17 | Data transmission method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011282966.0A CN112491989A (en) | 2020-11-17 | 2020-11-17 | Data transmission method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112491989A true CN112491989A (en) | 2021-03-12 |
Family
ID=74930764
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011282966.0A Withdrawn CN112491989A (en) | 2020-11-17 | 2020-11-17 | Data transmission method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112491989A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113422671A (en) * | 2021-06-30 | 2021-09-21 | 北京交通大学 | Verification method for judicial public internal and external network data consistency |
| CN113468863A (en) * | 2021-05-20 | 2021-10-01 | 北京字节跳动网络技术有限公司 | File checking method and device, electronic equipment and computer readable storage medium |
| CN113645266A (en) * | 2021-06-21 | 2021-11-12 | 天津津航计算技术研究所 | File transmission method, file verification method and device |
| CN113806364A (en) * | 2021-08-28 | 2021-12-17 | 特斯联科技集团有限公司 | Big data storage system and method |
| CN114401147A (en) * | 2022-01-20 | 2022-04-26 | 山西晟视汇智科技有限公司 | New energy power station communication message comparison method and system based on abstract algorithm |
| CN114710333A (en) * | 2022-03-23 | 2022-07-05 | 未鲲(上海)科技服务有限公司 | Data transmission and verification method, system, computer equipment and storage medium |
| CN114884713A (en) * | 2022-04-26 | 2022-08-09 | 中通服慧展科技有限公司 | Information protection method, device, equipment and medium |
| CN116383896A (en) * | 2023-06-07 | 2023-07-04 | 中航金网(北京)电子商务有限公司 | File integrity verification method, cloud platform starting method, device and equipment |
-
2020
- 2020-11-17 CN CN202011282966.0A patent/CN112491989A/en not_active Withdrawn
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113468863A (en) * | 2021-05-20 | 2021-10-01 | 北京字节跳动网络技术有限公司 | File checking method and device, electronic equipment and computer readable storage medium |
| CN113645266A (en) * | 2021-06-21 | 2021-11-12 | 天津津航计算技术研究所 | File transmission method, file verification method and device |
| CN113422671A (en) * | 2021-06-30 | 2021-09-21 | 北京交通大学 | Verification method for judicial public internal and external network data consistency |
| CN113806364A (en) * | 2021-08-28 | 2021-12-17 | 特斯联科技集团有限公司 | Big data storage system and method |
| CN113806364B (en) * | 2021-08-28 | 2023-12-22 | 深圳特斯联智能科技有限公司 | Big data storage system and method |
| CN114401147A (en) * | 2022-01-20 | 2022-04-26 | 山西晟视汇智科技有限公司 | New energy power station communication message comparison method and system based on abstract algorithm |
| CN114401147B (en) * | 2022-01-20 | 2024-02-20 | 山西晟视汇智科技有限公司 | New energy power station communication message comparison method and system based on abstract algorithm |
| CN114710333A (en) * | 2022-03-23 | 2022-07-05 | 未鲲(上海)科技服务有限公司 | Data transmission and verification method, system, computer equipment and storage medium |
| CN114884713A (en) * | 2022-04-26 | 2022-08-09 | 中通服慧展科技有限公司 | Information protection method, device, equipment and medium |
| CN114884713B (en) * | 2022-04-26 | 2024-05-10 | 中通服慧展科技有限公司 | Information protection method, device, equipment and medium |
| CN116383896A (en) * | 2023-06-07 | 2023-07-04 | 中航金网(北京)电子商务有限公司 | File integrity verification method, cloud platform starting method, device and equipment |
| CN116383896B (en) * | 2023-06-07 | 2023-11-03 | 中航金网(北京)电子商务有限公司 | File integrity verification method, cloud platform starting method, device and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112491989A (en) | Data transmission method, device, equipment and storage medium | |
| CN109117097B (en) | Data storage method and system based on block chain | |
| US11907406B2 (en) | Computer-implemented method and system of tamper-evident recording of a plurality of service data items | |
| US10878248B2 (en) | Media authentication using distributed ledger | |
| WO2021082757A1 (en) | Blockchain system-based data processing method and apparatus | |
| CN111555872B (en) | Communication data processing method, device, computer system and storage medium | |
| US10938575B2 (en) | Signature compression for hash-based signature schemes | |
| CN100357901C (en) | Method for verifying data between main device and back-up device | |
| JPH06315027A (en) | Method and equipment for data authentication in data communication environment | |
| CN113468598A (en) | Block chain-based certificate-preserving and security notarization system and method | |
| CN110503434A (en) | Data verification method, device, equipment and storage medium based on hash algorithm | |
| TW202044086A (en) | File verification method, file verification system and file verification server | |
| CN112732695B (en) | Cloud storage data security deduplication method based on block chain | |
| CN112380575A (en) | Multi-party electronic signature synthesis method, device, equipment and storage medium | |
| CN111339551B (en) | Data verification method and related device and equipment | |
| CN111294209B (en) | Block chain-based intelligent terminal security verification method and device | |
| CN100481835C (en) | Data concentrated backup method, reduction method and system thereof | |
| CN114830572A (en) | Data transmission method, device, equipment, system and storage medium | |
| CN114374502A (en) | Energy data uploading and consensus system and method based on-chain and off-chain cooperation | |
| CN112668353A (en) | Bargun authentication method, device, equipment and storage medium | |
| CN112184440A (en) | Data processing method, data processing device, node equipment and storage medium | |
| CN112241513B (en) | Digital asset processing system to be authenticated | |
| CN111050218A (en) | Set top box upgrading and checking method, server, set top box and storage medium | |
| CN117200992A (en) | System and method for identifying false of data stream transmission | |
| CN111478948B (en) | Block chain access method, internet of things equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210312 |
|
| WW01 | Invention patent application withdrawn after publication |