CN110460599A - Unidirectional transmission method is realized based on multi-level buffer and active and standby mechanism - Google Patents
Unidirectional transmission method is realized based on multi-level buffer and active and standby mechanism Download PDFInfo
- Publication number
- CN110460599A CN110460599A CN201910742496.2A CN201910742496A CN110460599A CN 110460599 A CN110460599 A CN 110460599A CN 201910742496 A CN201910742496 A CN 201910742496A CN 110460599 A CN110460599 A CN 110460599A
- Authority
- CN
- China
- Prior art keywords
- server
- file
- active
- standby
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
- H04L41/0663—Performing the actions predefined by failover planning, e.g. switching to standby network elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
- H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
- H04L49/9015—Buffering arrangements for supporting a linked list
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/164—Adaptation or special uses of UDP protocol
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1074—Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
- H04L67/1078—Resource delivery mechanisms
- H04L67/108—Resource delivery mechanisms characterised by resources being split in blocks or fragments
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Environmental & Geological Engineering (AREA)
- Health & Medical Sciences (AREA)
- Cardiology (AREA)
- General Health & Medical Sciences (AREA)
- Computer Security & Cryptography (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The present invention relates to technical field of data transmission, more particularly to one kind realizes unidirectional transmission method based on multi-level buffer and active and standby mechanism, including sending server and receiving server, it sends server and one file is divided into segment one by one, it is numbered with a section serial number, each segment is assembled into a UDP message packet, and fills filename, file MD5 value, serial number, total packet number, and be sent respectively to active and standby reception server by two network interface cards.In the invention on the basis of using one-way transmission component, utilize multilevel memory cache way, ferry-boat in conjunction with conventional security isolation technology adds agent skill group, while ensureing information one-way transmission, the real-time Transmission of information and controllable is realized to greatest extent, using direct memory, in conjunction with array, pointer, Hash, chained list, distributed, the technological means such as active and standby, memory optimization layout type, solution multithreading is read at random and overtime memory overflows, while crossing filter data using distributed thread pond, improves the processing capacity of data.
Description
Technical field
The present invention relates to technical field of data transmission, more particularly to one kind to be realized unidirectionally based on multi-level buffer and active and standby mechanism
Transmission method.
Background technique
As information technology is grown rapidly, dramatic change, the big rule of information network are had occurred in people's lives, working method
Mould application greatly improves office efficiency, builds by many years, the built digitalized network with considerable scale in China, but with
Network it is continuous universal, safety problem is increasing, and security of network and information problem, which becomes, threatens country and government safety
Major hidden danger, with the continuous awareness and understanding to safety problem, in particular for the protection of classified information, China is by information
Security Construction mentions a comparable height and comes up, and the connection of internal concerning security matters network and external public network brings serious peace
Full problem, such as viral subversive, hacker attacks, and the safety products such as firewall, proxy server, intrusion detection be all based on it is soft
The logic isolation realization of part, the physical isolation safety requirements not being able to satisfy between High Security Level network department and other networks, therefore
How from physical layer complete data flow control become a problem urgently to be resolved.
When carrying out data exchange between high safety network and lower security network, how to guarantee itself peace of high safety network
Entirely, it prevents divulging a secret for information particularly important, when realizing the one-way transmission function of file, is generally transmitted using gigabit networking, UDP is passed
Defeated agreement, since udp protocol is a kind of connectionless agreement, reliability is relatively low, by traditional technology means, it is excessive according to
Rely in network speed, hard disk speed, the conditions such as CPU frequency, will cause asymmetry between mutual parameter, lead to side's data processing
The problems such as blocking, loss of data.
Summary of the invention
(1) the technical issues of solving
In view of the deficiencies of the prior art, the present invention provides one kind realizes one-way transmission side based on multi-level buffer and active and standby mechanism
Method has the real-time Transmission and controllable advantage of information, solves traditional technology means, excessive to depend on network speed, firmly
Disk speed, the conditions such as CPU frequency will cause asymmetry between mutual parameter, side's data processing caused to block, loss of data
Problem.
(2) technical solution
To realize above-mentioned technical problem, one-way transmission is realized based on multi-level buffer and active and standby mechanism the present invention provides such a
Method, including send server and receive server, a file is divided into segment one by one, is numbered with section by the transmission server
Serial number, each segment are assembled into a UDP message packet, and fill filename, file MD5 value, serial number, total packet number, and pass through
Two network interface cards are sent respectively to active and standby reception server.
Further, the reception server process process are as follows:
S1, the active and standby server that receives receive the data for sending server transmission.
S2, server memory formatting is received, each server distribution space 4G memory is for caching, according to a structure
Body formats this 4G memory headroom, and writes structural body address, and filename, file MD5, serial number, total packet are included in structural body
Number, time, correlating markings.
S3, creation receiving process, receiving process is respectively created in active and standby reception server, as from the number for sending server
According to reception.
After S4, receiving process receive UDP message packet, according to the file MD5 value and packet serial number of each packet, one is calculated
Cryptographic Hash is exactly the address of the corresponding memory of preformatting, if the address is used, using next address,
If continuous 10 addresses are all used, on the 10th address, a dynamic link table is created.
S5, receiving process create a thread after receiving first data packet of each file.
S6, queue log process, can start on two-server respectively, and task is identical, and content is fully synchronized, and two
A process judged by way of heartbeat it is active and standby, it is standby to rise master immediately when host process occurs abnormal, continue unfinished work
Make.
S7, whenever file receiving after the completion of, first remove in log to have checked whether processed same file name and MD5,
There is direct deletion, log is not written, queue is then written.
S8, two filtering thread pools, go queue log process to take the file for finishing receiving, needing to filter, execute antivirus, close
The filtering of key word is written region to be taken, the forwarding of a file is completed to this after the work such as condition code matching.
Further, described to be write the data in correspondence memory by the process of step 4, UDP buffering is not in overflow
The buffer area of UDP is directly expanded to 100,000,000 spaces or more by situation.
Further, described in step 5, thread creates file first, and this document only has the read-write of designated user to weigh
Permission does not can be performed in limit, prevents unexpected be performed.Thread calculates memory address according to the MD5 value and serial number of file, according to
Data packet receives sequence one by one, when all data packets all finish receiving, and calculate MD5 value it is errorless after, write-in queue
Log process.If there is one of data packet time-out, file stops receiving, and thread exits, and Resource recovery has been written into memory
Data, also can time-out, do not influence next write-in.
(3) beneficial effect
The present invention provides one kind to realize unidirectional transmission method based on multi-level buffer and active and standby mechanism, have it is following the utility model has the advantages that
1, unidirectional transmission method should be realized based on multi-level buffer and active and standby mechanism, optical fiber transmission is unidirectionally controlled and efficient in realization
Property, can satisfy the requirement of one-way isolation in terms of stability and economy, meanwhile, on the basis using one-way transmission component
On, using multilevel memory cache way, the ferry-boat in conjunction with conventional security isolation technology adds agent skill group, is ensureing that information is unidirectional
While transmission, the real-time Transmission of information and controllable is realized to greatest extent.
2, unidirectional transmission method should be realized based on multi-level buffer and active and standby mechanism, referred to using direct memory in conjunction with array
Needle, Hash, chained list is distributed, the technological means such as active and standby, memory optimization layout type, and solution multithreading is read at random and time-out
Memory overflows, while increasing active-standby mode, reduces emergency response mechanism when unilateral failure and data exception, in the process simultaneously
Filter data is crossed using distributed thread pond, improves the processing capacity of data.
Detailed description of the invention
Fig. 1 is structure of the invention flow chart；
Fig. 2 is that the present invention receives server process flow chart.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
It is a kind of referring to FIG. 1-2, that unidirectional transmission method is realized based on multi-level buffer and active and standby mechanism, including send server
With reception server, a file is divided into segment one by one, is numbered with a section serial number by the transmission server, and each segment is assembled into
One UDP message packet, and filename, file MD5 value, serial number, total packet number are filled, and be sent respectively to lead by two network interface cards
It is standby to receive server.
The reception server process process are as follows:
S1, the active and standby server that receives receive the data for sending server transmission.
S2, server memory formatting is received, each server distribution space 4G memory is for caching, according to a structure
Body formats this 4G memory headroom, and writes structural body address, and filename, file MD5, serial number, total packet are included in structural body
Number, time, correlating markings.
S3, creation receiving process, receiving process is respectively created in active and standby reception server, as from the number for sending server
According to reception.
After S4, receiving process receive UDP message packet, according to the file MD5 value and packet serial number of each packet, one is calculated
Cryptographic Hash is exactly the address of the corresponding memory of preformatting, if the address is used, using next address,
If continuous 10 addresses are all used, on the 10th address, a dynamic link table is created, is write the data in corresponding in this way
In depositing, UDP buffering is not in spilling situation, and the buffer area of UDP is directly expanded to 100,000,000 spaces or more.
S5, receiving process create a thread, thread is first after receiving first data packet of each file
File is created, this document only has the access limit of designated user, and permission not can be performed, and prevents unexpected be performed.Thread according to
The MD5 value and serial number of file calculate memory address, and data packet receives one by one in sequence, when all data packets are whole
Finish receiving, and calculate MD5 value it is errorless after, be written queue log process.If there is one of data packet time-out, file stops
It receiving, thread exits, and Resource recovery has been written into the data of memory, also can be overtime, next write-in is not influenced.
S6, queue log process, can start on two-server respectively, and task is identical, and content is fully synchronized, and two
A process judged by way of heartbeat it is active and standby, it is standby to rise master immediately when host process occurs abnormal, continue unfinished work
Make.
S7, whenever file receiving after the completion of, first remove in log to have checked whether processed same file name and MD5,
There is direct deletion, log is not written, queue is then written.
S8, two filtering thread pools, go queue log process to take the file for finishing receiving, needing to filter, execute antivirus, close
The filtering of key word is written region to be taken, the forwarding of a file is completed to this after the work such as condition code matching.
In conclusion unidirectional transmission method should be realized based on multi-level buffer and active and standby mechanism, optical fiber transmission is unidirectional in realization
It can satisfy the requirement of one-way isolation in terms of control and high efficiency, stability and economy, meanwhile, using one-way transmission portion
On the basis of part, using multilevel memory cache way, the ferry-boat in conjunction with conventional security isolation technology adds agent skill group, is ensureing
While information one-way transmission, the real-time Transmission of information and controllable is realized to greatest extent.
Unidirectional transmission method should be realized based on multi-level buffer and active and standby mechanism, referred to using direct memory in conjunction with array
Needle, Hash, chained list is distributed, the technological means such as active and standby, memory optimization layout type, and solution multithreading is read at random and time-out
Memory overflows, while increasing active-standby mode, reduces emergency response mechanism when unilateral failure and data exception, in the process simultaneously
Filter data is crossed using distributed thread pond, improves the processing capacity of data.
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to
Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment
Intrinsic element.
It although an embodiment of the present invention has been shown and described, for the ordinary skill in the art, can be with
A variety of variations, modification, replacement can be carried out to these embodiments without departing from the principles and spirit of the present invention by understanding
And modification, the scope of the present invention is defined by the appended.
Claims (4)
1. one kind realizes unidirectional transmission method based on multi-level buffer and active and standby mechanism, it is characterised in that: including send server and
Server is received, a file is divided into segment one by one, is numbered with a section serial number by the transmission server, and each segment is assembled into one
A UDP message packet, and fill filename, file MD5 value, serial number, total packet number, and be sent respectively to by two network interface cards active and standby
Receive server.
2. according to claim 1 realize unidirectional transmission method based on multi-level buffer and active and standby mechanism, it is characterised in that: institute
It states and receives server process process are as follows:
S1, the active and standby server that receives receive the data for sending server transmission；
S2, server memory formatting is received, each server distribution space 4G memory is for caching, according to a structure physique
This 4G memory headroom of formulaization, and write structural body address, in structural body comprising filename, file MD5, serial number, total packet number, when
Between, correlating markings；
Receiving process is respectively created in S3, creation receiving process, active and standby reception server, connects as the data from transmission server
It receives；
After S4, receiving process receive UDP message packet, according to the file MD5 value and packet serial number of each packet, a Hash is calculated
Value, is exactly the address of the corresponding memory of preformatting, if the address is used, using next address, if
Continuous 10 addresses are all used, and on the 10th address, create a dynamic link table；
S5, receiving process create a thread after receiving first data packet of each file；
S6, queue log process, can start on two-server respectively, and task is identical, and content is fully synchronized, two into
Journey judged by way of heartbeat it is active and standby, it is standby to rise master immediately when host process occurs abnormal, continue unfinished work；
S7, after the completion of file receiving, first remove in log to have checked whether processed same file name and MD5, have straight
Deletion is connect, log is not written, queue is then written；
S8, two filtering thread pools, go queue log process to take the file for finishing receiving, needing to filter, execute antivirus, keyword
Filtering is written region to be taken, the forwarding of a file is completed to this after the work such as condition code matching.
3. according to claim 2 realize unidirectional transmission method based on multi-level buffer and active and standby mechanism, it is characterised in that: institute
It states and is write the data in correspondence memory by the process of step 4, UDP buffering is not in spilling situation, the buffer area of UDP
Directly it is expanded to 100,000,000 spaces or more.
4. according to claim 2 realize unidirectional transmission method based on multi-level buffer and active and standby mechanism, it is characterised in that: institute
It states in step 5, thread creates file first, and this document only has the access limit of designated user, and permission not can be performed, and prevents
It is only unexpected to be performed；Thread calculates memory address, data packet connects one by one in sequence according to the MD5 value and serial number of file
Receive, when all data packets all finish receiving, and calculate MD5 value it is errorless after, write-in queue log process；If there is wherein one
A data packet time-out, file stop receiving, and thread exits, and Resource recovery has been written into the data of memory, also can be overtime, not shadow
Ring write-in next time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910742496.2A CN110460599B (en) | 2019-08-13 | 2019-08-13 | Method for realizing one-way transmission based on multi-level cache and main/standby mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910742496.2A CN110460599B (en) | 2019-08-13 | 2019-08-13 | Method for realizing one-way transmission based on multi-level cache and main/standby mechanism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110460599A true CN110460599A (en) | 2019-11-15 |
| CN110460599B CN110460599B (en) | 2021-11-09 |
Family
ID=68486132
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910742496.2A Active CN110460599B (en) | 2019-08-13 | 2019-08-13 | Method for realizing one-way transmission based on multi-level cache and main/standby mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110460599B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110912669A (en) * | 2019-11-15 | 2020-03-24 | 珠海市新德汇信息技术有限公司 | Two-channel data transmission method based on file stream and data stream |
| CN112003992A (en) * | 2020-08-14 | 2020-11-27 | 迅镭智能（广州）科技有限公司 | Transmission system and method based on scanning gun |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101989916A (en) * | 2009-08-04 | 2011-03-23 | 西安交大捷普网络科技有限公司 | Separating multi-stage buffer network content filtering system and method |
| CN102035843A (en) * | 2010-12-17 | 2011-04-27 | 北京锐安科技有限公司 | System and method for transmitting data in one direction |
| US8589953B1 (en) * | 2010-08-06 | 2013-11-19 | Open Invention Network, Llc | System and method for transparent consistent application-replication of multi-process multi-threaded applications |
| CN105656902A (en) * | 2016-01-15 | 2016-06-08 | 深圳市利谱信息技术有限公司 | One-way reliable transmission and control system based on light transmission |
| CN106850156A (en) * | 2016-11-28 | 2017-06-13 | 深圳市鑫之淼科技有限公司 | No-feedback one-way data transmission set and transmission method based on network interface |
| CN106992856A (en) * | 2017-03-29 | 2017-07-28 | 山西大学 | The data coordinating method of extensive continuous variable quantum key distribution based on GPU |
-
2019
- 2019-08-13 CN CN201910742496.2A patent/CN110460599B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101989916A (en) * | 2009-08-04 | 2011-03-23 | 西安交大捷普网络科技有限公司 | Separating multi-stage buffer network content filtering system and method |
| US8589953B1 (en) * | 2010-08-06 | 2013-11-19 | Open Invention Network, Llc | System and method for transparent consistent application-replication of multi-process multi-threaded applications |
| CN102035843A (en) * | 2010-12-17 | 2011-04-27 | 北京锐安科技有限公司 | System and method for transmitting data in one direction |
| CN105656902A (en) * | 2016-01-15 | 2016-06-08 | 深圳市利谱信息技术有限公司 | One-way reliable transmission and control system based on light transmission |
| CN106850156A (en) * | 2016-11-28 | 2017-06-13 | 深圳市鑫之淼科技有限公司 | No-feedback one-way data transmission set and transmission method based on network interface |
| CN106992856A (en) * | 2017-03-29 | 2017-07-28 | 山西大学 | The data coordinating method of extensive continuous variable quantum key distribution based on GPU |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110912669A (en) * | 2019-11-15 | 2020-03-24 | 珠海市新德汇信息技术有限公司 | Two-channel data transmission method based on file stream and data stream |
| CN112003992A (en) * | 2020-08-14 | 2020-11-27 | 迅镭智能（广州）科技有限公司 | Transmission system and method based on scanning gun |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110460599B (en) | 2021-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6964193B2 (en) | How to determine the main chain of the blockchain, devices, equipment and storage media | |
| US9773015B2 (en) | Dynamically varying the number of database replicas | |
| CN105138615A (en) | Method and system for building big data distributed log | |
| WO2012145963A1 (en) | Data management system and method | |
| US9245147B1 (en) | State machine reference monitor for information system security | |
| CN110460599A (en) | Unidirectional transmission method is realized based on multi-level buffer and active and standby mechanism | |
| TW201800967A (en) | Method and device for processing distributed streaming data | |
| US7080075B1 (en) | Dynamic remastering for a subset of nodes in a cluster environment | |
| CN103984768B (en) | A kind of data-base cluster manages method, node and the system of data | |
| JP2021528882A (en) | Prioritize storage of shared blockchain data | |
| CN106899643A (en) | A kind of user journal storage method and equipment | |
| Harchol et al. | FTvNF: Fault tolerant virtual network functions | |
| US7899785B2 (en) | Reconfiguring propagation streams in distributed information sharing | |
| TW202320528A (en) | Service processing method and device based on alliance chain system | |
| US8817592B2 (en) | Storage device and storage device control method | |
| US10735247B2 (en) | Spanning tree protocol traffic handling system | |
| CN106302061A (en) | A kind of communication means based on FlexRay bus, Apparatus and system | |
| CN110727636B (en) | Equipment isolation method of system on chip | |
| CN107360153A (en) | A kind of network security protection system on big data | |
| CN115587348B (en) | Configurable security control method, device and medium for access of PCIE (peripheral component interface express) equipment | |
| CN111770150A (en) | Access flow control method and device and electronic equipment | |
| CN115688089A (en) | Method, system and medium for safely extending PCIE protocol | |
| CN104023000A (en) | Network intrusion detection method | |
| CN110636090B (en) | Data synchronization method and device under narrow bandwidth condition | |
| CN105450727A (en) | Network communication method and network communication architecture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |