CN107070958A - A kind of mass data high-efficiency transmission method - Google Patents
A kind of mass data high-efficiency transmission method Download PDFInfo
- Publication number
- CN107070958A CN107070958A CN201710467056.1A CN201710467056A CN107070958A CN 107070958 A CN107070958 A CN 107070958A CN 201710467056 A CN201710467056 A CN 201710467056A CN 107070958 A CN107070958 A CN 107070958A
- Authority
- CN
- China
- Prior art keywords
- message
- packet receiving
- packet
- estype
- microcode
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/161—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
- H04L69/162—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields involving adaptations of sockets based mechanisms
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a kind of mass data high-efficiency transmission method, kernel network interface card packet receiving circle queue is mapped to User space, User space directly operates circle queue packet receiving, User space reduces scheduling overhead to the switching of kernel state when evading packet receiving；Kernel supports many running-on packet receivings based on network interface card, and LPP is resident packet receiving thread, and LPP packet receiving threads are resident in APP, message is directly received, it is to avoid message is transmitted by LPP, Socket, reduces message transmissions link and message copying；Message is directly delivered to APP, reduces LPP links, saves more LPP redundancies table inquiry.This method significantly improves router platform protocol stack packet sending and receiving ability, improves the mass data transfers ability and performance of router platform.
Description
Technical field
The present invention relates to a kind of mass data high-efficiency transmission method of router platform, belong to Computers and Communication technology neck
Domain.
Background technology
In router platform, the standard socket send and receive packets ability of general protocol stack is in 7000pps or so, and in magnanimity
, it is necessary to which protocol stack send and receive packets ability is far longer than 7000pps in data transmission procedure.
The standard socket send and receive packets framework of protocol stack is as shown in figure 1, send and receive packets path is the void in Fig. 1 in the prior art
Line, send and receive packets performance consumption is in LPP, Socket and in message procedure transmission.Pass through analysis, the standard of Current protocol stack
Each stage performance loss of socket send and receive packets framework such as table one.
The performance loss of table one
The content of the invention
Goal of the invention：For problems of the prior art, the present invention provides a kind of mass data high-efficiency transmission method,
Improve to improve message transmissions ability in terms of following 3：
1) scheduling overhead is reduced.User space is mapped to using by kernel network interface card packet receiving circle queue, User space is directly operated
Circle queue packet receiving, User space reduces scheduling overhead to the switching of kernel state when evading packet receiving.
2) message is reduced to copy in the transmission of more cross-thread and message.Kernel supports many running-on packet receivings based on network interface card, LPP
Resident packet receiving thread, LPP packet receiving threads are resident in APP, message is directly received, it is to avoid message is transmitted by LPP, Socket,
Reduce message transmissions link and message copying.
3) LPP redundancies are reduced to table look-up.Message is directly delivered to APP, reduces LPP links, saves more LPP redundancy tables and look into
Ask.
Technical scheme：A kind of mass data high-efficiency transmission method, sets up high-speed message transmission channel, and data are direct by NP
APP PM send and receive packets threads are shipped to by internal Ethernet exchange channels.
1) high-speed message passage set up mode be data by NP after, using ICP/IP protocol, by socket passages,
Into internal Ethernet exchange channels, then immediately proceed to APP PM send and receive packets threads, such data by shortest path straight
It is connected to up to application layer, is used for application layer software.
2) in message repeating process, microcode determines protocol massages paths according to message distributing list, that is, microcode according to
Message distributing list selection message walks Socket passages or high-speed message passage, realizes that message distribution is compatible in this way
Former message distribution logic.
3) microcode reports message to network interface, application process (APP) use running-on packet receiving on network interface.To different processes or not
Same agreement, microcode is created not to the different estype types of text encapsulation of above delivering newspaper, different processes according to different estype types
Same running-on realizes distributed packet receiving；Also realized simultaneously using the message of different estype types is mutually isolated between different processes
Distributed network interface packet receiving.
4) modification kernel network interface packet receiving driving, User space is mapped to by the annular buffer of packet receiving, is started in User space and is received
Envelope curve journey directly reads packet receiving caching and carries out packet receiving.
Microcode selects message to walk Socket passages or high-speed message passage according to message distributing list, and it selects path side
Formula is：Message before transmission, packet route flag byte is added in microcode, corresponding message road is added in message distributing list
The corresponding packet route of footpath flag byte, microcode uses this flag byte, with reference to message distributing list, determines message data road
Footpath, realizes the compatible former message distribution logic of message distribution in this way.
Microcode reports message to network interface, application process (APP) use running-on packet receiving on network interface.To different processes or difference
Agreement, microcode creates different to the different estype types of text encapsulation of above delivering newspaper, different processes according to different estype types
Running-on realize distributed packet receiving.Also realized simultaneously using the message of different estype types is mutually isolated between different processes
Distributed network interface packet receiving, specific implementation is：Microcode adds a byte in heading in upper send, and the value of this byte is represented
Different estype types, APP receives parsing after message by running-on and obtains this estype, is then created according to estype values
Receiving thread, is sent in same packet receiving thread the message data bag comprising identical estype values for the thread process to realize
The mutually isolated processing of message.
Kernel network interface packet receiving driving is changed, the annular buffer of packet receiving is mapped to User space, packet receiving is started in User space
Thread directly reads packet receiving caching and carries out packet receiving, and specific implementation is：A packet receiving annular is created in Installed System Memory
The mapping table of buffer addresses (pointer) and User space mark, when User space starts packet receiving thread, according to User space mark from this
Annular buffer address is read in mapping table, then directly the reading of packet is realized from the address acquisition data.
Brief description of the drawings
Fig. 1 is standard socket send and receive packets framework；
Fig. 2 is high-speed message passage send and receive packets framework.
Embodiment
With reference to specific embodiment, the present invention is furture elucidated, it should be understood that these embodiments are merely to illustrate the present invention
Rather than limitation the scope of the present invention, after the present invention has been read, various equivalences of the those skilled in the art to the present invention
The modification of form falls within the application appended claims limited range.
A kind of mass data high-efficiency transmission method, sets up the high-speed message transmission channel marked such as Fig. 2 dotted lines, data warp
Cross the PM send and receive packets threads that NP is directly shipped to APP by internal Ethernet exchange channels.
1) high-speed message passage set up mode be data by NP after, using ICP/IP protocol, by socket passages,
Into internal Ethernet exchange channels, then immediately proceed to APP PM send and receive packets threads, such data by shortest path straight
It is connected to up to application layer, is used for application layer software.
2) in message repeating process, microcode determines protocol massages paths according to message distributing list, that is, microcode according to
Message distributing list selection message walks Socket passages or high-speed message passage, and its selection path fashion is：Message is in transmission
Before, packet route flag byte is added in microcode, corresponding packet route flag byte correspondence is added in message distributing list
Packet route, microcode uses this flag byte, with reference to message distributing list, determines message data path, in this way
Realize the compatible former message distribution logic of message distribution.
3) microcode reports message to network interface, application process (APP) use running-on packet receiving on network interface.To different processes or not
Same agreement, microcode is created not to the different estype types of text encapsulation of above delivering newspaper, different processes according to different estype types
Same running-on realizes distributed packet receiving.Also realized simultaneously using the message of different estype types is mutually isolated between different processes
Distributed network interface packet receiving, specific implementation is：Microcode adds a byte, the value generation of this byte in heading in upper send
The different estype types of table, APP receives parsing after message by running-on and obtains this estype, is then created according to estype values
Receiving thread is built, the message data bag comprising identical estype values is sent in same packet receiving thread and comes real for the thread process
The mutually isolated processing of existing message.
4) modification kernel network interface packet receiving driving, User space is mapped to by the annular buffer of packet receiving, is started in User space and is received
Envelope curve journey directly reads packet receiving caching and carries out packet receiving, and specific implementation is：A packet receiving annular is created in Installed System Memory
The mapping table of buffer addresses (pointer) and User space mark, when User space starts packet receiving thread, according to User space mark from this
Annular buffer address is read in mapping table, then directly the reading of packet is realized from the address acquisition data.
Using above-mentioned technical proposal, after the method implemented in step using the present invention, by actual test with testing
Card result shows that the present invention can make the protocol stack transmitting-receiving ability of router platform reach 100000pps, be far longer than current association
The 7000pps of the standard Socket send and receive packets abilities of stack is discussed, therefore invention significantly improves router platform protocol stack message
Transmitting-receiving ability, improves the mass data transfers ability and transmission performance of router platform.
Claims (4)
1. a kind of mass data high-efficiency transmission method, it is characterised in that set up high-speed message transmission channel, data are direct by NP
APP PM send and receive packets threads are shipped to by internal Ethernet exchange channels；
1）High-speed message passage set up mode be data by NP after, using ICP/IP protocol, pass through socket passages, enter
Internal Ethernet exchange channels, then immediately proceed to APP PM send and receive packets threads, and such data are directly arrived by shortest path
Up to application layer, used for application layer software；
2）In message repeating process, microcode determines protocol massages path according to message distributing list, that is, microcode is according to message
Distributing list selection message walks Socket passages or high-speed message passage, and the compatible former report of message distribution is realized in this way
Literary distribution logic；
3）Microcode reports message to network interface, application process (APP) use running-on packet receiving on network interface；
To different processes or different agreement, microcode is to the different estype types of text encapsulation of above delivering newspaper, and different processes are according to difference
Estype types create different running-ons and realize distributed packet receivings；Simultaneously also using different estype types message mutually every
From realizing the distributed network interface packet receiving between different processes；
4）Kernel network interface packet receiving driving is changed, the annular buffer of packet receiving is mapped to User space, packet receiving line is started in User space
Journey directly reads packet receiving caching and carries out packet receiving.
2. mass data high-efficiency transmission method as claimed in claim 1, it is characterised in that microcode is selected according to message distributing list
Message walks Socket passages or high-speed message passage, and its selection path fashion is：Message before transmission, is added in microcode
Packet route flag byte, adds the corresponding packet route of corresponding packet route flag byte, microcode in message distributing list
Using this flag byte, with reference to message distributing list, message data path is determined, realizes that message distribution is compatible in this way
Former message distribution logic.
3. mass data high-efficiency transmission method as claimed in claim 1, it is characterised in that microcode reports message to network interface, should
Running-on packet receiving on network interface is used with process (APP)；
To different processes or different agreement, microcode is to the different estype types of text encapsulation of above delivering newspaper, and different processes are according to difference
Estype types create different running-ons and realize distributed packet receivings；Simultaneously also using different estype types message mutually every
From realizing the distributed network interface packet receiving between different processes, specific implementation is：Microcode adds one in heading in upper send
Byte, the value of this byte represents different estype types, and APP receives parsing after message by running-on and obtains this
Estype, then creates receiving thread according to estype values, the message data bag comprising identical estype values is sent to same
The processing that message is mutually isolated is realized in packet receiving thread for the thread process.
4. mass data high-efficiency transmission method as claimed in claim 1, it is characterised in that modification kernel network interface packet receiving driving,
The annular buffer of packet receiving is mapped to User space, starting packet receiving thread in User space directly reads packet receiving caching progress packet receiving,
Specific implementation is：A packet receiving annular buffer address (pointer) and the mapping of User space mark are created in Installed System Memory
Table, when User space starts packet receiving thread, annular buffer address, Ran Houzhi are read according to User space mark from the mapping table
Connect from the address acquisition data to realize the reading of packet.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710467056.1A CN107070958B (en) | 2017-06-19 | 2017-06-19 | High-efficiency transmission method for mass data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710467056.1A CN107070958B (en) | 2017-06-19 | 2017-06-19 | High-efficiency transmission method for mass data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107070958A true CN107070958A (en) | 2017-08-18 |
| CN107070958B CN107070958B (en) | 2020-02-21 |
Family
ID=59594295
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710467056.1A Active CN107070958B (en) | 2017-06-19 | 2017-06-19 | High-efficiency transmission method for mass data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107070958B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109831394A (en) * | 2017-11-23 | 2019-05-31 | 杭州华为数字技术有限公司 | Data processing method, terminal and computer storage medium |
| CN110602154A (en) * | 2018-06-13 | 2019-12-20 | 网宿科技股份有限公司 | WEB server and method for processing data message thereof |
| CN111371723A (en) * | 2018-12-07 | 2020-07-03 | 网宿科技股份有限公司 | Method and device for realizing PPTP VPN network isolation under DPDK framework |
| CN114201317A (en) * | 2021-12-16 | 2022-03-18 | 北京有竹居网络技术有限公司 | Data transmission method, device, storage medium and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101178694A (en) * | 2006-11-07 | 2008-05-14 | 莱克斯信息技术（北京）有限公司 | Intel e1000 zero copy method |
| CN101494661A (en) * | 2008-12-29 | 2009-07-29 | 深圳市同洲电子股份有限公司 | Method and apparatus for reading data of protocol stack of transmission control protocol/internet protocol |
| CN101917350A (en) * | 2010-09-13 | 2010-12-15 | 南京中兴特种软件有限责任公司 | Network card drive-based zero copy Ethernet message capturing and transmitting implementation method under Linux |
| EP2887256A1 (en) * | 2013-12-18 | 2015-06-24 | ContinuumBridge Limited | An apparatus for network bridging |
-
2017
- 2017-06-19 CN CN201710467056.1A patent/CN107070958B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101178694A (en) * | 2006-11-07 | 2008-05-14 | 莱克斯信息技术（北京）有限公司 | Intel e1000 zero copy method |
| CN101494661A (en) * | 2008-12-29 | 2009-07-29 | 深圳市同洲电子股份有限公司 | Method and apparatus for reading data of protocol stack of transmission control protocol/internet protocol |
| CN101917350A (en) * | 2010-09-13 | 2010-12-15 | 南京中兴特种软件有限责任公司 | Network card drive-based zero copy Ethernet message capturing and transmitting implementation method under Linux |
| EP2887256A1 (en) * | 2013-12-18 | 2015-06-24 | ContinuumBridge Limited | An apparatus for network bridging |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109831394A (en) * | 2017-11-23 | 2019-05-31 | 杭州华为数字技术有限公司 | Data processing method, terminal and computer storage medium |
| CN110602154A (en) * | 2018-06-13 | 2019-12-20 | 网宿科技股份有限公司 | WEB server and method for processing data message thereof |
| CN111371723A (en) * | 2018-12-07 | 2020-07-03 | 网宿科技股份有限公司 | Method and device for realizing PPTP VPN network isolation under DPDK framework |
| CN111371723B (en) * | 2018-12-07 | 2022-06-17 | 网宿科技股份有限公司 | Method and device for realizing PPTP VPN network isolation under DPDK framework |
| CN114201317A (en) * | 2021-12-16 | 2022-03-18 | 北京有竹居网络技术有限公司 | Data transmission method, device, storage medium and electronic equipment |
| CN114201317B (en) * | 2021-12-16 | 2024-02-02 | 北京有竹居网络技术有限公司 | Data transmission method and device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107070958B (en) | 2020-02-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107070958A (en) | A kind of mass data high-efficiency transmission method | |
| CN102859973B (en) | Method, apparatus and system for address resolution | |
| CN107079017B (en) | Message conversion method and device | |
| CN100471162C (en) | Method for releasing and processing virtual circuit information and supplier edge device | |
| CN103391296B (en) | A kind of controller, transponder and Path Setup method and system | |
| JP7403635B2 (en) | METHODS, APPARATUS AND SYSTEM FOR FORWARDING PACKETS IN SR NETWORK | |
| CN105227466A (en) | Communication processing method and device | |
| CN104486244B (en) | A kind of execution method and device of QoS policy | |
| CN103685032B (en) | Message forwarding method and network address translation services device | |
| CN112491706B (en) | Data message processing method and device, storage medium and electronic device | |
| CN102065012B (en) | Distributed system and method for processing IP message fragments | |
| US20080037558A1 (en) | Access Network System and Subscriber Data Route Control Method | |
| WO2008089667A1 (en) | Method and apparatus for reducing service loss in a link aggregation group | |
| CN101534210A (en) | A method and equipment for setting priority of multicast IP messages | |
| CN1937568B (en) | Method and device for realizing message priority-level mapping | |
| CN101115001B (en) | Host computer of ADSL router and PVC dynamic binding method | |
| CN100539551C (en) | The implementation method of static network address conversion-protocol conversion and gateway device | |
| CN102868642A (en) | Method and device for achieving NVGRE message transmission in application specific integrated circuit (ASIC) | |
| CN108696430A (en) | The route filtering method of LDP is realized in a kind of IPRAN systems | |
| CN100463430C (en) | Method for establishing a path, having a certain qos-class and a related access multiplexer | |
| CN107231316A (en) | The transmission method and device of message | |
| CN102970218B (en) | The method that MF-TDMA ICBM SHF satellite terminal carries out anonymous interconnection based on broadcast-type interface | |
| CN104579959B (en) | The method and apparatus for establishing the label switched path of SDN streams in an mpls network | |
| CN107786444A (en) | Method for message transmission and controller | |
| CN106161269A (en) | A kind of data pack transmission method, controller and switch |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |