CN109547162B - Data communication method based on two sets of one-way boundaries - Google Patents
Data communication method based on two sets of one-way boundaries Download PDFInfo
- Publication number
- CN109547162B CN109547162B CN201811487436.2A CN201811487436A CN109547162B CN 109547162 B CN109547162 B CN 109547162B CN 201811487436 A CN201811487436 A CN 201811487436A CN 109547162 B CN109547162 B CN 109547162B
- Authority
- CN
- China
- Prior art keywords
- data
- packet
- instruction
- message
- confirmation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/164—Adaptation or special uses of UDP protocol
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/0001—Systems modifying transmission characteristics according to link quality, e.g. power backoff
- H04L1/0006—Systems modifying transmission characteristics according to link quality, e.g. power backoff by adapting the transmission format
- H04L1/0007—Systems modifying transmission characteristics according to link quality, e.g. power backoff by adapting the transmission format by modifying the frame length
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
- H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
- H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
- H04L1/1822—Automatic repetition systems, e.g. Van Duuren systems involving configuration of automatic repeat request [ARQ] with parallel processes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
Abstract
The invention relates to a communication method which solves the problems of low efficiency and high expansion cost caused by disk IO. The method comprises the following steps: on the basis of not changing the original network security level, UDP is used as a data transmission mode to realize real-time instruction query and data security transmission; in two sets of physically isolated network environments, a bidirectional UDP protocol is used for forming a communication link from and to, and data transmission confirmation, data combination and data processing are realized according to the message type, the message number and the message length; the two ends of the boundary are respectively provided with a set of sending service and a set of receiving service, the received data are directly written into a memory, the multithreading processing technology is utilized to carry out encapsulation, instruction analysis and instruction execution on the data, a confirmation data packet is sent immediately after the data are received, and a message retransmission mechanism is started for the data without the returned confirmation packet to prevent the data from being lost.
Description
Technical Field
The invention relates to a communication method, in particular to a data communication method based on two sets of one-way boundaries.
Background
In the current public security industry, two common data interaction methods based on two sets of unidirectional boundaries exist in two networks which are completely physically isolated: the first method is to transmit data through an FTP file: the requested data is put in the FTP of one network segment and is synchronized to the FTP of the other network segment from the boundary, so that the data transmission is realized, but the efficiency and concurrency are not ensured, and the transmission state of the data cannot be known. The second method, like the first, replaces FTP with DB, which, although somewhat advantageous in data query and categorization, is less efficient than the first and more costly to maintain DB.
Specifically, the prior art has the following defects 1: in a traditional boundary synchronization mode, whether FTP or DB is adopted, the phenomena of multiple landing and continuous polling of data exist, the IO reading and writing of a disk are frequent, the cost for maintaining an FTP cluster and a database is high, and the interaction efficiency is improved by increasing more hardware resources. Defect 2: due to the polling mode, the data interaction delay is serious, the data state cannot be acquired in time, and the influence on the service scene with high real-time requirement is very large.
The characteristics of the public security service determine that high requirements are placed on real-time performance and concurrency efficiency of data transmission, in order to solve the problem that real-time instruction query, real-time comparison and data packet loss cannot be achieved by crossing boundary data, the problem is solved by considering a boundary UDP protocol, the packet loss problem of the UDP is solved by using a confirmation mechanism, the problems of data storage and interaction speed are solved by using a memory, two unidirectional UDP accesses are converted into HTTP requests, the data receiving and processing processes are completed in the memory, and the speed and the timeliness of the data transmission are well solved due to the fact that no file falls to the ground.
Disclosure of Invention
The invention provides a data communication method based on two sets of one-way boundaries, aiming at providing an efficient and high-concurrency data interaction mode based on UDP (user datagram protocol), solving the problems of low efficiency and high expansion cost caused by disk IO (input/output), solving the resource consumption caused by polling and realizing the purposes of high concurrency, high efficiency and easy expansion by using the least resources and maintenance cost.
In order to achieve the purpose, the invention adopts the following technical scheme that: on the basis of not changing the original network security level, UDP is used as a data transmission mode to realize real-time instruction query and data security transmission; in two sets of physically isolated network environments, a two-way UDP protocol is used for forming a communication link between a user and a mobile terminal, and data transmission confirmation, data combination and data processing are realized according to the message type, the message number and the message length.
The two ends of the boundary are respectively provided with a set of sending service and a set of receiving service, the received data are directly written into a memory, the multithreading processing technology is utilized to carry out encapsulation, instruction analysis and instruction execution on the data, a confirmation data packet is sent immediately after the data are received, and a message retransmission mechanism is started for the data without the returned confirmation packet to prevent the data from being lost.
As a preferable embodiment of the present invention, the method comprises the following steps.
Step one, defining a packet header and a packet body of a data packet, wherein the packet header comprises a data packet type, a packet body length, a data packet ID and a data packet sequence number.
And step two, receiving the data instruction by using the HTTP gateway, and delivering the data instruction to an instruction generating end to split the data packet.
And step three, judging the length of the data instruction when the data packet is unpacked, and unpacking the data according to the standard of the step one when the length of the instruction exceeds the length of the specified single data packet.
And step four, directly storing the data packets split in the step three in a message queue, constructing a thread pool to consume and send the data in the message queue through UDP, and sending the data to an instruction execution end.
And step five, the instruction execution end starts the data receiving module and the multi-thread data processing module, and after receiving the data, the messages are classified and stored in the memory by using the multi-thread mechanism.
And step six, generating the acknowledgement packet defined in the step one while receiving the data, and sending out the acknowledgement packet by using a UDP protocol.
Step seven, after the data sending module sends the data, if the corresponding confirmation packet is not received within the specified time, starting a data retransmission mechanism until the retry times are reached; and entering a result receiving waiting state after receiving the confirmation packet.
Step eight: when the received data is an end packet, taking out the serial number of the end packet, starting to detect the integrity of the message at regular time to form a complete processing instruction, and calling a message processing module to process the message; incomplete messages are discarded after the message is overtime, and the gateway prompts the user to operate overtime.
Step nine: after the instruction is executed, the instruction result is sent back to the instruction generating end through UDP; refer to the procedure of step three and step four specifically.
Step ten: and the instruction generating end transmits the instruction result to the HTTP gateway for result display, and completes the processing of the HTTP instruction by using UDP.
As another preferred solution of the present invention, the types of the data packets are: an end packet, a normal data packet, and an acknowledgement packet.
Compared with the prior art, the invention has the beneficial effects.
The invention utilizes UDP communication in two sets of boundaries in two physically isolated networks, and has the following advantages.
1. The problem that the HTTP can be used only under the condition of TCP communication is solved, and the complexity of project development is reduced.
2. The problem of data real-time interaction which cannot be realized by the traditional boundary technology is solved.
3. And the bottleneck of disk IO is broken through by using memory calculation, and high concurrency is realized.
4. The method has low requirements on server configuration and can run in a virtualization environment.
Drawings
The invention is further described with reference to the following figures and detailed description. The scope of the invention is not limited to the following expressions.
FIG. 1 is a block diagram of the system of the present invention.
Detailed Description
The data communication method based on two sets of unidirectional boundaries provided by the invention realizes real-time instruction query and data safe transmission by using UDP as a data transmission mode on the basis of not changing the original network safety level, forms a communication link in two sets of physically isolated network environments by using a bidirectional UDP protocol, and realizes data transmission confirmation, data combination and data processing according to message types, message numbers and message lengths.
The two ends of the boundary are respectively provided with a set of sending service and a set of receiving service, the received data are directly written into a memory, the multithreading processing technology is utilized to carry out encapsulation, instruction analysis and instruction execution on the data, a confirmation data packet is sent immediately after the data are received, and a message retransmission mechanism is started for the data without the returned confirmation packet to prevent the data from being lost.
Specifically, as shown in fig. 1, it is a structural diagram of the present invention. The left end and the right end of the boundary are two physically isolated networks, the data interaction middleware is realized by the product of the invention, and a service caller can call a service cluster in the other network by calling the data interaction middleware.
The message is written into the memory computing unit after being transmitted, the scheduling of the thread is realized through a message mechanism, the unpacking and the packing of the data are realized by utilizing the data interaction center, the service gateway on the left side provides an HTTP protocol for a service caller, and the instruction processing unit on the right side performs instruction execution work.
Specifically, the data communication method based on two sets of unidirectional boundaries comprises the following steps.
Step one, defining a packet header and a packet body of a data packet, wherein the packet header comprises a data packet type, a packet body length, a data packet ID and a data packet sequence number.
And step two, receiving the data instruction by using the HTTP gateway, and delivering the data instruction to an instruction generating end to split the data packet.
And step three, judging the length of the data instruction when the data packet is unpacked, and unpacking the data according to the standard of the step one when the length of the instruction exceeds the length of the specified single data packet.
And step four, directly storing the data packets split in the step three in a message queue, constructing a thread pool to consume and send the data in the message queue through UDP, and sending the data to an instruction execution end.
And step five, the instruction execution end starts the data receiving module and the multi-thread data processing module, and after receiving the data, the messages are classified and stored in the memory by using the multi-thread mechanism.
And step six, generating the acknowledgement packet defined in the step one while receiving the data, and sending out the acknowledgement packet by using a UDP protocol.
Step seven, after the data sending module sends the data, if the corresponding confirmation packet is not received within the specified time, starting a data retransmission mechanism until the retry times are reached; and entering a result receiving waiting state after receiving the confirmation packet.
Step eight: when the received data is an end packet, taking out the serial number of the end packet, starting to detect the integrity of the message at regular time to form a complete processing instruction, and calling a message processing module to process the message; incomplete messages are discarded after the message is overtime, and the gateway prompts the user to operate overtime.
Step nine: after the instruction is executed, the instruction result is sent back to the instruction generating end through UDP; refer to the procedure of step three and step four specifically.
Step ten: and the instruction generating end transmits the instruction result to the HTTP gateway for result display, and completes the processing of the HTTP instruction by using UDP.
Preferably, the types of the data packets are: an end packet, a normal data packet, and an acknowledgement packet.
A specific embodiment is provided for a data communication method based on two sets of unidirectional UDP boundaries.
Preparation work: two physically isolated networks A and B are prepared, A, B are provided with two unidirectional boundaries A and B, each of the A and B networks deploys a SERVER SERVER1 and a SERVER SERVER2, the service of the invention is deployed in the two SERVERs, and the A network segment prepares a user machine for instruction query.
Take the example where a user sends an HTTP service to SERVER 1.
1) An HTTP request is sent to SERVER1 using the user machine.
2) SERVER1 parses the HTTP request to encapsulate and unpack.
3) The SERVER1 sends the well-made data packet to the SERVER2 through the boundary A.
4) After receiving the data packet, the SERVER2 sends an acknowledgement packet to the SERVER1 through the boundary b.
5) After receiving the acknowledgement packet, SERVER1 knows that the transmission is successful and starts to wait for the return of the query result.
6) The SERVER2 starts to analyze and merge data packets after sending the confirmation packet, and restores the data packets to the execution instruction.
7) SERVER2 sends the results after the instruction is executed to SERVER1 through boundary B.
8) Upon receipt of the data, SERVER1 sends an acknowledgement packet to SERVER2 over boundary a.
9) The SERVER2 releases the resources after receiving the acknowledgement packet.
10) The SERVER1 sorts and combines the received data packets and returns the data packets to the user machine, and the SERVER1 releases the operation resources.
11) The user machine implements the instruction query through HTTP.
The present invention has the following features.
1. The UDP protocol is used for realizing rapid data sending and receiving, the memory is used for data processing, and the performance bottleneck caused by writing files into a disk is avoided.
2. Because the UDP has the possibility of packet loss, the data can safely arrive by using an information confirmation mechanism and a message retransmission mechanism.
3. The data transmission process supports the use of ciphertext transmission, and the data security is ensured.
4. The deployment is convenient, the requirement of a single point on the server is low, and the expansion is easy.
It should be understood that the detailed description of the present invention is only for illustrating the present invention and is not limited by the technical solutions described in the embodiments of the present invention, and those skilled in the art should understand that the present invention can be modified or substituted equally to achieve the same technical effects; as long as the use requirements are met, the method is within the protection scope of the invention.
Claims (2)
1. The data communication method based on two sets of one-way boundaries is characterized by comprising the steps of realizing real-time instruction query and data safe transmission by using UDP as a data transmission mode on the basis of not changing the original network safety level; in two sets of physically isolated network environments, a bidirectional UDP protocol is used for forming a communication link from and to, and data transmission confirmation, data combination and data processing are realized according to the message type, the message number and the message length;
the two ends of the boundary are respectively provided with a set of sending service and receiving service, the received data is directly written into the memory, the multithreading processing technology is utilized to carry out encapsulation, instruction analysis and instruction execution on the data, a confirmation data packet is sent immediately after the data is received, and a message retransmission mechanism is started for the data without the returned confirmation packet to prevent the data from being lost;
the method comprises the following steps:
step one, defining a packet header and a packet body of a data packet, wherein the packet header comprises a data packet type, a packet body length, a data packet ID and a data packet sequence number;
step two, receiving a data instruction by using an HTTP gateway, and delivering the data instruction to an instruction generating end to split a data packet;
step three, judging the length of a data instruction when the data packet is unpacked, and unpacking the data according to the standard of the step one when the length of the instruction exceeds the length of a specified single data packet;
step four, directly storing the data packet split in the step three in a message queue, constructing a thread pool to consume and send UDP (user datagram protocol) data in the message queue, and sending the data to an instruction execution end;
step five, the instruction execution end starts the data receiving module and the multi-thread data processing module, and after receiving the data, messages are classified and stored in the memory by using a multi-thread mechanism;
step six, generating a confirmation packet defined in the step one while receiving the data, and sending out the confirmation packet by using a UDP protocol;
step seven, after the data sending module sends the data, if the corresponding confirmation packet is not received within the specified time, starting a data retransmission mechanism until the retry times are reached; if the acknowledgement packet is received, entering a result waiting receiving state;
step eight: when the received data is an end packet, taking out the serial number of the end packet, starting to detect the integrity of the message at regular time to form a complete processing instruction, and calling a message processing module to process the message; the incomplete message is discarded after the message is overtime, and the gateway prompts the user to operate overtime;
step nine: after the instruction is executed, the instruction result is sent back to the instruction generating end through UDP;
step ten: and the instruction generating end transmits the instruction result to the HTTP gateway for result display, and completes the processing of the HTTP instruction by using UDP.
2. The data communication method based on two sets of unidirectional boundaries of claim 1, wherein: the type of the data packet of the first step is divided into: an end packet, a normal data packet, and an acknowledgement packet.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811487436.2A CN109547162B (en) | 2018-12-06 | 2018-12-06 | Data communication method based on two sets of one-way boundaries |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811487436.2A CN109547162B (en) | 2018-12-06 | 2018-12-06 | Data communication method based on two sets of one-way boundaries |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109547162A CN109547162A (en) | 2019-03-29 |
| CN109547162B true CN109547162B (en) | 2021-06-29 |
Family
ID=65852985
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811487436.2A Active CN109547162B (en) | 2018-12-06 | 2018-12-06 | Data communication method based on two sets of one-way boundaries |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109547162B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112436998B (en) * | 2020-11-12 | 2023-05-02 | 北京天融信网络安全技术有限公司 | Data transmission method and electronic equipment |
| CN112637129A (en) * | 2020-11-30 | 2021-04-09 | 招商华软信息有限公司 | Multi-node communication method, electronic device, storage medium and system in network isolation environment |
| CN114598700B (en) * | 2022-01-25 | 2024-03-29 | 阿里巴巴（中国）有限公司 | Communication method and communication system |
| CN114528319A (en) * | 2022-02-22 | 2022-05-24 | 厦门四信通信科技有限公司 | Method, device and equipment for acquiring data of multiple PLCs (programmable logic controllers) and readable storage medium |
| CN114979233A (en) * | 2022-07-19 | 2022-08-30 | 深圳市亿联无限科技有限公司 | Method and system for realizing synchronous and asynchronous call between modules based on domain socket |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102035843A (en) * | 2010-12-17 | 2011-04-27 | 北京锐安科技有限公司 | System and method for transmitting data in one direction |
| CN108462679A (en) * | 2017-02-21 | 2018-08-28 | 杭州海康威视数字技术股份有限公司 | Data transmission method and device |
| CN108881158A (en) * | 2018-05-04 | 2018-11-23 | 北京明朝万达科技股份有限公司 | Data interaction system and method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7665118B2 (en) * | 2002-09-23 | 2010-02-16 | Credant Technologies, Inc. | Server, computer memory, and method to support security policy maintenance and distribution |
| CN102088393B (en) * | 2009-12-02 | 2013-07-03 | 南京南瑞继保电气有限公司 | Method for transmitting positive and negative data across safety zone |
-
2018
- 2018-12-06 CN CN201811487436.2A patent/CN109547162B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102035843A (en) * | 2010-12-17 | 2011-04-27 | 北京锐安科技有限公司 | System and method for transmitting data in one direction |
| CN108462679A (en) * | 2017-02-21 | 2018-08-28 | 杭州海康威视数字技术股份有限公司 | Data transmission method and device |
| CN108881158A (en) * | 2018-05-04 | 2018-11-23 | 北京明朝万达科技股份有限公司 | Data interaction system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109547162A (en) | 2019-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109547162B (en) | Data communication method based on two sets of one-way boundaries | |
| CN108270732B (en) | A kind of Streaming Media processing method and system | |
| CN106034084B (en) | Data transmission method and device | |
| CN111580995B (en) | Synchronous communication method and system of distributed cloud platform and Internet of things intelligent terminal based on MQTT asynchronous communication scene | |
| WO2017067391A1 (en) | Data sharing method and device for virtual machines | |
| CN108494817A (en) | Data transmission method, relevant apparatus and system | |
| CN102546612B (en) | Remote procedure call implementation method based on remote direct memory access (RDMA) protocol in user mode | |
| WO2014180407A1 (en) | Pushing method and device therefor | |
| US10609125B2 (en) | Method and system for transmitting communication data | |
| CN112631788B (en) | Data transmission method and data transmission server | |
| WO2022032984A1 (en) | Mqtt protocol simulation method and simulation device | |
| CN112261142B (en) | RDMA network data retransmission method, device and FPGA | |
| CN112910909B (en) | Data packet processing method based on conversion of CoAP protocol and HTTP protocol | |
| CN111522663B (en) | Data transmission method, device and system based on distributed storage system | |
| CN114095901A (en) | Communication data processing method and device | |
| CN108234595B (en) | Log transmission method and system | |
| WO2019015487A1 (en) | Data retransmission method, rlc entity and mac entity | |
| CN109656705A (en) | A kind of method and apparatus of data processing | |
| Yang et al. | Research and Design of a Real-Time Interactive Application Development Model Based on the android Platform | |
| CN111198840B (en) | GOOSE and MMS common network communication method and system suitable for dual-core system | |
| WO2016176942A1 (en) | Link multiplexing method and system based on load balancer | |
| CN110380991A (en) | A kind of IOCP mechanism and the Internet of Things Network Communication acceleration system based on eFPGA and IOCP | |
| CN113411266B (en) | Cloud data transmission method and system based on isolation device, terminal and storage medium | |
| CN114390014B (en) | Service processing method and system for improving communication concurrency in high-speed network environment | |
| CN114650279B (en) | Instance processing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |