CN111683138A - Socket-based efficient custom data transmission method - Google Patents
Socket-based efficient custom data transmission method Download PDFInfo
- Publication number
- CN111683138A CN111683138A CN202010504781.3A CN202010504781A CN111683138A CN 111683138 A CN111683138 A CN 111683138A CN 202010504781 A CN202010504781 A CN 202010504781A CN 111683138 A CN111683138 A CN 111683138A
- Authority
- CN
- China
- Prior art keywords
- file
- data
- data transmission
- socket
- upper computer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
- H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
- H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
- H04L1/1867—Arrangements specially adapted for the transmitter end
- H04L1/189—Transmission or retransmission of more than one copy of a message
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/161—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
- H04L69/162—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields involving adaptations of sockets based mechanisms
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Communication Control (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a Socket-based high-efficiency user-defined data transmission method, which adopts a data transmission system of an upper computer and a plurality of lower computers, and the upper computer and the lower computers carry out two-way communication by adopting a Socket; after the upper computer and the lower computer are connected through Socket; transmitting file data by using a file data transmission protocol, analyzing and storing the file data in a data pool according to a quadruplet; then scanning the data pool and cleaning the data pool; after the file data are transmitted, the upper computer checks the file data according to a predefined file data transmission protocol, and if the checking fails, the upper computer sends an instruction to inform the lower computer to resend the file data. The invention can realize the data transmission of a plurality of lower computers and the upper computer, can effectively process the receiving and splicing of data packets through a self-defined transmission protocol, reduces the problem of packet loss possibly caused by bidirectional communication between the upper computer and the lower computer in a weak network environment, and is particularly suitable for the transmission of larger file data on the basis of the common Socket data transmission.
Description
Technical Field
The invention relates to the technical field of computer communication and software, in particular to a Socket-based high-efficiency custom data transmission method.
Background
With the development of internet technology and sensor technology, more and more sensors can realize remote data transmission. At present, data acquisition of sensors mostly adopts MQTT or TPC/IP technology to realize data uploading. The data communicated is also mostly a few bytes of data. However, in some monitoring devices, such as ground deformation monitoring radar, data monitored at one time can reach several MB or even several tens of MB, and is stored in the form of file data. If the conventional transmission method is adopted, the transmission of one file takes a long time, and once one byte is in error, the transmission fails. And, if the time of two measurements is less than the time of one data transmission, it may cause confusion of the two measurement data. Therefore, the invention provides an efficient custom data transmission method aiming at the situations, and the method is particularly suitable for the transmission of larger file data on the basis of the ordinary Socket data transmission.
In summary, a Socket-based high-efficiency custom data transmission method is urgently needed to solve the problems in the prior art.
Disclosure of Invention
The invention aims to provide a Socket-based high-efficiency custom data transmission method to realize the transmission of remote file data.
In order to achieve the aim, the invention provides a Socket-based high-efficiency user-defined data transmission method, which adopts a data transmission system, wherein the data transmission system comprises an upper computer and a plurality of lower computers, and the upper computer and the lower computers carry out two-way communication in a Socket mode; the data transmission method comprises the following steps:
the method comprises the following steps: the upper computer and the lower computer are connected through Socket;
step two: transmitting file data by adopting a file data transmission protocol, analyzing and storing the file data in a data pool according to a quadruplet; the quadruplet is T < filename, current packet number, total packet number and file stream >;
step three: scanning the data pool and cleaning the data pool;
step four: after the file data are transmitted, the upper computer checks the file data according to a predefined file data transmission protocol, and if the checking fails, the upper computer sends an instruction to inform the lower computer to resend the file data.
Preferably, when the Socket is used for bidirectional communication, the lower computer transmits file data to the upper computer except for the instruction information.
Preferably, the file data includes one or more of a text file, an image file, and a binary file.
Preferably, the file data is named in a manner of a suffix name of yyyy-MM-dd-HH-MM-ss., wherein yyy is year, MM is month, dd is day, HH is 24-system hour, MM is minute, and ss is second.
Preferably, the file data transmission protocol is defined as follows:
the size of file data to be transmitted is M bytes, the file data is read according to a binary format, the file data is averagely divided, and each part of the file data is a K byte file stream, so that the file is divided into L partial file streams;
wherein, L ═ Floor (M/K), Floor () is an upward rounding function; in the first L-1 part, the byte number of each part of file stream is K, and the byte number of the L-th part of file stream is M- (L-1) × K.
Preferably, the file data transmission protocol includes an identifier, a file header and a file body; the mark is used for identifying the initial position of the message, comprises two characters and occupies two bytes;
the file header comprises a radar number start byte, a file name start byte, a file stream start byte, a total file packet number, a total file body byte number and a total file byte number of an int64 data type of int32 type data; the int32 data type occupies 4 bytes, the int64 data type occupies 8 bytes, and the file header occupies 36 bytes in total;
the file body comprises a radar number, a file name, a check code and a file stream which are sequentially stored in a binary format.
Preferably, the scanning frequency of the data pool is not less than 5 seconds.
Preferably, the data pool is cleaned according to different situations, specifically:
A. when only one file name exists in all the quadruples and the number of the quadruples is less than the total package number, no processing is carried out; b. When only one file name exists in all the quadruples and the number of the quadruples is equal to the total packet number, splicing the file streams in the quadruples from small to large according to the current packet number value to form a new file write-in disk;
C. when more than two file names exist in all the quadruplets, the transmission is regarded as failure, the upper computer sends a failure instruction to the lower computer, and all the quadruplets in the data pool are eliminated simultaneously;
D. when two file names exist in all four-tuple groups, and the number of the four-tuple groups of a certain file name exceeds 90% of the total number, the data pool continues to receive the file names;
E. and when two file names exist in all the quadruplets and the quadruplet quantity of a certain file name is equal to the total package quantity of the file, splicing the file streams in the quadruplets of the file name from small to large according to the current package quantity value to form a new file to be written into the disk. Eliminating the quadruple of the file name in the data pool;
F. and when two file names exist in all four-tuple groups, the number of the four-tuple groups of any file name is less than the total number of the packages of the file, and the number of the four-tuple groups of any file name is less than 60% of the total number, clearing the data pool, sending an instruction to the lower computer, and informing that the two files fail to be sent.
The technical scheme of the invention has the following beneficial effects:
the invention can realize the data transmission of a plurality of lower computers and the upper computer, can effectively process the receiving and splicing of data packets through a self-defined transmission protocol, reduces the problem of packet loss possibly caused by bidirectional communication between the upper computer and the lower computer in a weak network environment, and is particularly suitable for the transmission of larger file data on the basis of the common Socket data transmission.
In addition to the objects, features and advantages described above, other objects, features and advantages of the present invention are also provided. The present invention will be described in further detail below.
Detailed Description
The following is a detailed description of embodiments of the invention, but the invention can be implemented in many different ways, as defined and covered by the claims.
Example 1:
a Socket-based high-efficiency user-defined data transmission method adopts a data transmission system, wherein the data transmission system comprises an upper computer and a plurality of lower computers, and the upper computer and the lower computers carry out two-way communication in a Socket mode; when Socket sockets are used for bidirectional communication, the lower computer transmits file data to the upper computer except for instruction information. The transmitted file data includes text files, image files, binary files, and the like. The file data is named after yyyy-MM-dd-HH-MM-ss., where yyyy is year, MM is month, dd is day, HH is 24 hours, MM is minute, and ss is second.
The file data transmission protocol is defined as follows:
the size of file data to be transmitted is M bytes, the file data is read according to a binary format, the file data is averagely divided, and each part of the file data is a K byte file stream, so that the file is divided into L partial file streams;
wherein, L ═ Floor (M/K), Floor () is an upward rounding function; in the first L-1 part, the byte number of each part of file stream is K, and the byte number of the L-th part of file stream is M- (L-1) × K.
The file data transmission protocol comprises an identifier (Flag), a file Header (Header) and a file Body (Body); the format is described as follows:
the mark is used for identifying the starting position of the message, comprises two characters "##", and occupies two bytes;
the file header comprises a radar number start byte, a file name start byte, a file stream start byte, a total file packet number, a total file body byte number and a total file byte number of an int64 data type of int32 type data; the int32 data type occupies 4 bytes, the int64 data type occupies 8 bytes, and the file header occupies 36 bytes in total;
wherein, the starting byte of the radar number is the starting position of the radar number in the message; the initial byte of the file name is the initial position of the file name in the message; the initial byte of the check code is the initial position of the check code in the message; the initial byte of the file stream is the initial position of the divided file stream in the message; the total byte number of the file is the total byte number of the file; the current packet number of the file represents the currently sent number of packets, and the counting is started from 1; the total package number of the file indicates the total package number of the file; the number of bytes of the Body part is the number of bytes of the Body part.
The file body comprises a radar number, a file name, a check code and a file stream which are sequentially stored in a binary format. The file stream is a file stream of the divided packet.
For file data transmission, after an upper computer and a certain lower computer are connected through a Socket, the data organized according to the defined protocol are received, and each piece of data is analyzed and then stored in a data pool according to a quadruplet, wherein the quadruplet is T < file name, current packet number, total packet number and file stream >.
For the data pool, scanning the data pool according to the frequency not less than 5 seconds, and cleaning the data pool according to different situations, specifically:
A. when only one file name exists in all the quadruples and the number of the quadruples is less than the total package number, no processing is carried out; b. When only one file name exists in all the quadruples and the number of the quadruples is equal to the total packet number, splicing the file streams in the quadruples from small to large according to the current packet number value to form a new file write-in disk;
C. when more than two file names exist in all the quadruplets, the transmission is regarded as failure, the upper computer sends a failure instruction to the lower computer, and all the quadruplets in the data pool are eliminated simultaneously;
D. when two file names exist in all four-tuple groups, and the number of the four-tuple groups of a certain file name exceeds 90% of the total number, the data pool continues to receive the file names;
E. and when two file names exist in all the quadruplets and the quadruplet quantity of a certain file name is equal to the total package quantity of the file, splicing the file streams in the quadruplets of the file name from small to large according to the current package quantity value to form a new file to be written into the disk. Eliminating the quadruple of the file name in the data pool;
F. and when two file names exist in all four-tuple groups, the number of the four-tuple groups of any file name is less than the total number of the packages of the file, and the number of the four-tuple groups of any file name is less than 60% of the total number, clearing the data pool, sending an instruction to the lower computer, and informing that the two files fail to be sent.
After the file data is transmitted, the file data is checked and corrected at the upper computer end according to a predefined file data transmission protocol, and if the checking and correcting fails, the upper computer sends an instruction to inform the lower computer to resend the file data.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (8)
1. A Socket-based high-efficiency custom data transmission method is characterized in that a data transmission system is adopted, the data transmission system comprises an upper computer and a plurality of lower computers, and the upper computer and the lower computers perform two-way communication in a Socket mode; the data transmission method comprises the following steps:
the method comprises the following steps: the upper computer and the lower computer are connected through Socket;
step two: transmitting file data by adopting a file data transmission protocol, analyzing and storing the file data in a data pool according to a quadruplet; the quadruplet is T < filename, current packet number, total packet number and file stream >;
step three: scanning the data pool and cleaning the data pool;
step four: after the file data are transmitted, the upper computer checks the file data according to a predefined file data transmission protocol, and if the checking fails, the upper computer sends an instruction to inform the lower computer to resend the file data.
2. The Socket-based efficient custom data transmission method according to claim 1, wherein when Socket sockets are used for bidirectional communication, the lower computer transmits file data to the upper computer except for instruction information.
3. The Socket-based efficient custom data transmission method according to claim 2, wherein the file data comprises one or more of a text file, an image file, and a binary file.
4. The Socket-based efficient custom data transmission method according to claim 3, wherein the file data is named as yyyy-MM-dd-HH-MM-ss. suffix, wherein yyyy is year, MM is month, dd is day, HH is 24 hours, MM is minute, and ss is second.
5. The Socket-based efficient custom data transmission method according to claim 4, wherein the file data transmission protocol is defined as follows:
the size of file data to be transmitted is M bytes, the file data is read according to a binary format, the file data is averagely divided, and each part of the file data is a K byte file stream, so that the file is divided into L partial file streams;
wherein, L ═ Floor (M/K), Floor () is an upward rounding function; in the first L-1 part, the byte number of each part of file stream is K, and the byte number of the L-th part of file stream is M- (L-1) × K.
6. The Socket-based efficient custom data transmission method according to claim 5, wherein the file data transmission protocol comprises an identifier, a file header and a file body;
the mark is used for identifying the starting position of the message and comprises two characters;
the file header comprises a radar number start byte, a file name start byte, a file stream start byte, a total file packet number, a total file body byte number and a total file byte number of an int64 data type of int32 type data;
the file body comprises a radar number, a file name, a check code and a file stream which are sequentially stored in a binary format.
7. The Socket-based efficient custom data transmission method according to claim 6, wherein a scanning frequency of the data pool is not lower than 5 seconds.
8. The Socket-based efficient custom data transmission method according to any one of claims 1 to 7, wherein the data pool is cleaned according to different situations, specifically:
A. when only one file name exists in all the quadruples and the number of the quadruples is less than the total package number, no processing is carried out;
B. when only one file name exists in all the quadruples and the number of the quadruples is equal to the total packet number, splicing the file streams in the quadruples from small to large according to the current packet number value to form a new file write-in disk;
C. when more than two file names exist in all the quadruplets, the transmission is regarded as failure, the upper computer sends a failure instruction to the lower computer, and all the quadruplets in the data pool are eliminated simultaneously;
D. when two file names exist in all four-tuple groups, and the number of the four-tuple groups of a certain file name exceeds 90% of the total number, the data pool continues to receive the file names;
E. when two file names exist in all quadruples and the number of the quadruples of a certain file name is equal to the total number of packages of the file, splicing the file streams in the quadruples of the file name from small to large according to the current number of packages to form a new file written into a disk; eliminating the quadruple of the file name in the data pool;
F. and when two file names exist in all four-tuple groups, the number of the four-tuple groups of any file name is less than the total number of the packages of the file, and the number of the four-tuple groups of any file name is less than 60% of the total number, clearing the data pool, sending an instruction to the lower computer, and informing that the two files fail to be sent.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010504781.3A CN111683138B (en) | 2020-06-05 | 2020-06-05 | Socket-based efficient custom data transmission method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010504781.3A CN111683138B (en) | 2020-06-05 | 2020-06-05 | Socket-based efficient custom data transmission method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111683138A true CN111683138A (en) | 2020-09-18 |
CN111683138B CN111683138B (en) | 2023-06-16 |
Family
ID=72454272
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010504781.3A Active CN111683138B (en) | 2020-06-05 | 2020-06-05 | Socket-based efficient custom data transmission method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111683138B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113064158A (en) * | 2021-03-17 | 2021-07-02 | 中国有色金属长沙勘察设计研究院有限公司 | Engineering disaster intelligent monitoring radar communication system based on double middleware |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2009030130A1 (en) * | 2007-09-07 | 2009-03-12 | Shenzhen Coship Electronics Co., Ltd. | A method and system for transmitting data |
US20130067213A1 (en) * | 2010-04-30 | 2013-03-14 | International Business Machines Corporation | Dynamic encryption and decryption for network communication |
CN103078707A (en) * | 2013-01-03 | 2013-05-01 | 北京理工大学 | File transmission method in deep space communication |
CN103324587A (en) * | 2012-03-20 | 2013-09-25 | 安凯(广州)微电子技术有限公司 | Nandflash equipment encryption and decryption implementation method and system |
US20150067006A1 (en) * | 2011-10-20 | 2015-03-05 | Allen Miglore | System and method for transporting files between networked or connected systems and devices |
CN106411907A (en) * | 2016-10-13 | 2017-02-15 | 广西咪付网络技术有限公司 | Data transmission method and system |
CN106937287A (en) * | 2017-04-01 | 2017-07-07 | 四川艾特赢泰智能科技有限责任公司 | A kind of wireless data safe transmission method for Android device |
CN107070613A (en) * | 2017-03-22 | 2017-08-18 | 公安部交通管理科学研究所 | Reliable data transmission method under distributed network environment |
CN109525370A (en) * | 2018-11-29 | 2019-03-26 | 苏州德姆斯信息技术有限公司 | Data packet sending method based on zigbee |
-
2020
- 2020-06-05 CN CN202010504781.3A patent/CN111683138B/en active Active
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2009030130A1 (en) * | 2007-09-07 | 2009-03-12 | Shenzhen Coship Electronics Co., Ltd. | A method and system for transmitting data |
US20130067213A1 (en) * | 2010-04-30 | 2013-03-14 | International Business Machines Corporation | Dynamic encryption and decryption for network communication |
US20150067006A1 (en) * | 2011-10-20 | 2015-03-05 | Allen Miglore | System and method for transporting files between networked or connected systems and devices |
CN103324587A (en) * | 2012-03-20 | 2013-09-25 | 安凯(广州)微电子技术有限公司 | Nandflash equipment encryption and decryption implementation method and system |
CN103078707A (en) * | 2013-01-03 | 2013-05-01 | 北京理工大学 | File transmission method in deep space communication |
CN106411907A (en) * | 2016-10-13 | 2017-02-15 | 广西咪付网络技术有限公司 | Data transmission method and system |
CN107070613A (en) * | 2017-03-22 | 2017-08-18 | 公安部交通管理科学研究所 | Reliable data transmission method under distributed network environment |
CN106937287A (en) * | 2017-04-01 | 2017-07-07 | 四川艾特赢泰智能科技有限责任公司 | A kind of wireless data safe transmission method for Android device |
CN109525370A (en) * | 2018-11-29 | 2019-03-26 | 苏州德姆斯信息技术有限公司 | Data packet sending method based on zigbee |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113064158A (en) * | 2021-03-17 | 2021-07-02 | 中国有色金属长沙勘察设计研究院有限公司 | Engineering disaster intelligent monitoring radar communication system based on double middleware |
Also Published As
Publication number | Publication date |
---|---|
CN111683138B (en) | 2023-06-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8417852B2 (en) | Uploading TCP frame data to user buffers and buffers in system memory | |
EP1198926B1 (en) | Bridge for can to tcp/ip connection | |
US20060059400A1 (en) | System and method for in-line consistency checking of packetized data | |
US6965571B2 (en) | Precise error reporting | |
US20060101111A1 (en) | Method and apparatus transferring arbitrary binary data over a fieldbus network | |
EP1345382A2 (en) | System and method for identifying upper layer protocol message boundaries | |
US20060114909A1 (en) | Apparatus and method for performing cyclic redundancy check (CRC) on partial protocol data units (PDUS) | |
WO2005033960A1 (en) | Field data collection and processing system, such as for electric, gas, and water utility data | |
CN109217983B (en) | Industrial Internet of things operation terminal communication protocol design method | |
US7447976B2 (en) | Data transfer apparatus | |
CN101217429A (en) | A determination method of the initiation relationship within TCP messages based on TCP timestamp options | |
WO2011046056A1 (en) | Transmission control method for packet communication and packet communication system | |
CN113259886B (en) | Industrial edge data transmission method and device | |
CN111683138A (en) | Socket-based efficient custom data transmission method | |
CN1913531B (en) | Transmission method, compression method and device of TCP/IP packet | |
US7680944B1 (en) | Rapid transport service in a network to peripheral device servers | |
CN113254411A (en) | Cross-gatekeeper real-time database data synchronization method and system | |
EP1357721A2 (en) | System and method for identifying upper layer protocol message boundaries | |
KR100947178B1 (en) | Interface integration device that have integral constructure of camera serial interface and display serial interface | |
KR101024270B1 (en) | Method of datalink for remote control of automated system based on network | |
CN113872982B (en) | Ship monitoring data transmission method and system based on MQTT protocol | |
US8331788B2 (en) | Encapsulation scheme for optical over Ethernet | |
CN112272081B (en) | Full-duplex stateful communication protocol method for communication between robot and server | |
CN115442454A (en) | Printing data transmission method, transmission system and printing equipment | |
WO2022031901A1 (en) | Systems and methods of low latency data communication for physical link layer reliability |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
TA01 | Transfer of patent application right |
Effective date of registration: 20201230 Address after: No. 117, Tingyuan Road, Yuhua District, Changsha City, Hunan Province Applicant after: CHINESE NONFERROUS METAL SURVEY AND DESIGN INSTITUTE OF CHANGSHA Co.,Ltd. Applicant after: CHINA ALUMINUM INTERNATIONAL ENGINEERING Corp.,Ltd. Address before: No. 117, Tingyuan Road, Yuhua District, Changsha City, Hunan Province Applicant before: CHINESE NONFERROUS METAL SURVEY AND DESIGN INSTITUTE OF CHANGSHA Co.,Ltd. |
|
TA01 | Transfer of patent application right | ||
GR01 | Patent grant | ||
GR01 | Patent grant |