CN109889403A - A kind of CDN network flow rate testing methods based on polling mechanism - Google Patents
A kind of CDN network flow rate testing methods based on polling mechanism Download PDFInfo
- Publication number
- CN109889403A CN109889403A CN201910093227.8A CN201910093227A CN109889403A CN 109889403 A CN109889403 A CN 109889403A CN 201910093227 A CN201910093227 A CN 201910093227A CN 109889403 A CN109889403 A CN 109889403A
- Authority
- CN
- China
- Prior art keywords
- data packet
- buffer area
- network
- equipment
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The present invention relates to a kind of CDN network flow rate testing methods based on polling mechanism, there is extremely strong actual application prospect, when first data packet in network last consignment of data packet reaches network interface card, system is notified using terminal mode, system is by the facility registration into a device polling queue, and closing is to the interrupt response of the equipment, simultaneity factor activates a traps, the network equipment registered in snoop queue is polled, therefrom read data packet, be advantageous in that compared to traditional scheme: system will check a network card equipment every certain time interval, processing routine is called if having data arrival, poll efficiency under high bandwidth or heavy traffic condition is higher, it is easy to operate, it is easily achieved, convenient for being widely popularized and popularizing.
Description
Technical field
The present invention relates to Internet technical fields, and in particular to a kind of CDN network flow detection side based on polling mechanism
Method.
Background technique
Network Traffic Monitoring has very important effect in CDN business platform, while being also that network flow characteristic is returned
It receives, the important foundation of user's behaviors analysis.Usual Network Traffic Monitoring mainly has based on packet calculation method and based on Packet capturing
Method.
SNMP agent is realized in network equipments such as routers based on packet method of counting, and manager is read by management software
SNMP mib variable obtains network traffic information.It is supervised based on the flow monitoring method that packet counts because needing to realize in the network equipment
Agency by agreement is surveyed, flow monitoring activity has a great impact to the packet forwarding performance of equipment, and network performance is caused to decline；And base
It is also limited by the MIB realized in the information that the flow monitoring method that packet counts can capture, it is extremely limited, it is difficult to meet multiple
Miscellaneous network analysis demand.
Flow monitoring technology based on Packet capturing technology is functionally by packet capture and data packet analysis two parts group
At the data packet for capturing from network and meeting contact conditions in whole or in part is realized in packet capture, and data packet analysis is then right
The data packet captured is analyzed and is counted, information required for therefrom extracting.The difficulty that Packet capturing technology is faced at present
It is that hardware processing capability and used Packet capturing mechanism are difficult to adapt to the excessively high data packet arrival rate of network interface.In hardware
Under the premise of resource is limited, Packet capturing efficiency can only be improved by improving Packet capturing mechanism.
Summary of the invention
It is in view of the drawbacks of the prior art and insufficient, the CDN network flow inspection based on polling mechanism that the invention discloses a kind of
Survey method, system will check a network card equipment every certain time interval, call processing routine if having data arrival,
Poll efficiency under high bandwidth or heavy traffic condition is higher.
The present invention is achieved by the following technical programs:
A kind of CDN network flow rate testing methods based on polling mechanism, first number in network last consignment of data packet
According to packet reach predeterminated target equipment when, using network card equipment notify operating system, system by the facility registration of the transmission data to
In one device polling queue, and the interrupt response to the equipment is closed, simultaneity factor activates a traps, to snoop queue
The network equipment of middle registration is polled, therefrom read data packet.
In the above-mentioned technical solutions, operating system is the basic environment of various application systems and business software operation, such as
Windows system, Linux system etc..
In the above-mentioned technical solutions, specific step is as follows for detection method:
Network card equipment will also connect the data packet that network interface card receives from a Circular buffer area read data packet, operating system
The data received are written or copy to Circular buffer area；
Data reading and writing in Circular buffer area, Circular buffer area is in a manner of memory mapping in Packet capturing program
It is shared between system kernel；
The data received are written or copy Circular buffer area to, system kernel copies data packet to Circular buffer area
And increase data packet write pointer；
The read data packet from Circular buffer area, Packet capturing program obtain data packet from Circular buffer area and increase data packet
Read pointer, the data packet newly received cover the buffer area being read, without distributing new memory.
In the above-mentioned technical solutions, Circular buffer area is the cyclic buffer of a first in first out in operating system, is equipped with
One read pointer and a write pointer provide the exclusive reference to buffer area to signal procedure, and read pointer is directed toward buffer circle
In readable data, write pointer is directed toward buffer area writeable in buffer circle, realized by mobile read pointer and write pointer
The reading data and write-in of buffer area.
In the above-mentioned technical solutions, to reduce overhead caused by storage allocation, the memory size in Circular buffer area exists
Socket is just distributed by operating system when creating, and avoids usually being received by network card equipment data packet and then being carried out by system
When reason, cause continually to distribute, the step of releasing memory causes and cause overhead.
As a preferred embodiment of the above technical solution, a kind of CDN network flow rate testing methods based on polling mechanism of the present invention, together
Sample supports the poll to network card equipment, i.e. predeterminated target equipment is CPU, and the sending device for receiving poll is network interface card.
As a preferred embodiment of the above technical solution, the present invention is additionally provided with quota mechanism, i.e. network card equipment sends number to CPU every time
According to the maximum number of packet, this numerical value is configurable, and is calculated as P；If after network interface card sends P data packet, data packet receives slow
Depositing still has data in area, which is just reregistered with to the end of snoop queue, and waiting is polled to the equipment next time
Shi Jixu submits data packet to CPU core；Otherwise the network card equipment is nullified from snoop queue, and opens the sound of device interrupt
It answers.
The invention has the benefit that
The present invention relates to a kind of CDN network flow rate testing methods based on polling mechanism, there is extremely strong reality to answer
With prospect, when first data packet in network last consignment of data packet reaches network interface card, system, system are notified using terminal mode
By the facility registration into a device polling queue, and the interrupt response to the equipment is closed, simultaneity factor activation one is soft
It interrupts, the network equipment registered in snoop queue is polled, therefrom read data packet.Benefit compared to traditional scheme exists
In: system will check a network card equipment every certain time interval, call processing routine, poll if having data arrival
Efficiency is higher under high bandwidth or heavy traffic condition, easy to operate, it is easy to accomplish, convenient for being widely popularized and popularizing.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is a kind of data in the prior art of CDN network flow rate testing methods based on polling mechanism of the invention
The structural schematic diagram of packet RX path；
Fig. 2 is a kind of structural schematic diagram of CDN network flow rate testing methods based on polling mechanism of the embodiment of the present invention.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
Every other embodiment obtained without creative efforts, shall fall within the protection scope of the present invention.
Embodiment
When first data packet in network last consignment of data packet reaches network interface card, system is notified using terminal mode, is
It unites by the facility registration into a device polling queue, and closes the interrupt response to the equipment.Simultaneity factor activates one
Traps are polled the network equipment registered in snoop queue, therefrom read data packet.
Furthermore in the present invention also introduce quota concept, i.e., network interface card every time to CPU send data packet maximum number, this
Numerical value is configurable, and is calculated as P.If data packet, which receives in buffer area, still data, just after P data packet is accompanied in network interface card transmission
The equipment is reregistered with to the end of snoop queue, waiting continues to submit data to kernel when being polled to the equipment next time
Packet；Otherwise it will change equipment to nullify from snoop queue, and open a year old response for device interrupt.There are multiple network interface cards in systems in this way
When, it is ensured that the equity dispatching of each network interface card.
In above scheme, polling mode can improve the treatment effeciency of data packet to a certain extent, still, in the prior art
Polling mode itself there are also unfavorable factor, be mainly shown as under low bandwidth or small traffic conditions, data packet can be given
Bring larger delay；In addition, ceaselessly poll is also the waste of cpu resource when reaching without any data packet.
This performance bottleneck is present in the data path that the data packet received is submitted to user control from network interface card buffer area.
Packet capturing program will obtain data packet from network interface card, need by two buffer areas and normal ICP/IP protocol stack.From Packet capturing
For angle, data packet experienced unnecessary memory copying and processing from network interface card to user's space, it is not necessary to which strategic point expense is
It unites the CPU time.
As shown in Fig. 2, the present invention further improves this case, the improvement to the Packet capturing technology based on polling mechanism
It is concentrated mainly on transmittance process of the system to data packet from network interface card to user's space.
The technology creates a kind of Socket type PC_CIRCLE for being exclusively used in Packet capturing, it is from a Circular buffer
Area's read data packet, system also copy the data packet that network interface card receives to this Circular buffer area.Draw to reduce storage allocation
Rise overhead, Circular buffer area in there are just being distributed when socket creation, avoid usual data packet receive when it is frequent
Overhead caused by ground distribution, releasing memory.Circular buffer area is in a manner of memory mapping in Packet capturing program (corresponding network interface card
Equipment obtains data packet) and system kernel between share (corresponding data reading and writing in Circular buffer area), in system
Core copies data packet to Circular buffer area and increases (the corresponding data write-in (copy) that will be received to ring of data packet write pointer
The step of shape buffer area), Packet capturing program from Circular buffer area obtain data packet and increase data packet read pointer (it is corresponding be from
In Circular buffer area the step of read data packet ").The data packet newly received covers the buffer area being read, without
Distribute new memory.Its step are as follows:
Q1: creating a kind of Socket type PC_CIRCLE, is applied in Packet capturing program, the effect of Packet capturing program is
By memory mirror, (reading) data packet is copied from Circular buffer area；Circular buffer area is referred to and write pointer point by read pointer
Not buffer area writeable in the data readable into buffer circle and direction buffer circle.Q2: Circular buffer area is with memory
The mode of image is shared between Packet capturing program and system kernel；Memory mirror is that internal storage data is done to two copies, respectively
It is placed in main memory and mirror memory；
Q3: when needing that data are written, system kernel, which copies data packet to Circular buffer area and increases data packet, writes finger
Needle completes the write-in of data；Because there are just distributing when socket creation, avoiding usual data packet to receive in Circular buffer area
When continually distribution, overhead caused by releasing memory.Improve the efficiency of data packet write-in.
Q4: when needing to copy (reading) data, Packet capturing program obtains data packet from Circular buffer area and increases data
Packet read pointer completes the reading of data；Because, there are just distributing when socket creation, avoiding usual data in Circular buffer area
It is continually distributed when packet receives, overhead caused by releasing memory.Improve the efficiency of data packet reading.
Each block structure body is defined as follows in Circular buffer area:
Structsr_buff{
char packet[514]；
uintpktlen；
uint offset
}
Wherein, packet is used to save data packet itself, and pktlen is the length of data packet, and offset is then notebook data
Index of the mass in Circular buffer area.
User application is using the virtual address for really using memory when memory, and network interface directly uses memory
Physical address be addressed.It realizes that data packet is directly stored in Circular buffer area by network interface, needs to solve user's sky
Between arrive kernel spacing address translation problem.Therefore exclusively for one user's physical address mapping table of Circular buffer area definition,
It is actually 32 signless integer arrays.One table element is with saving a physical memory beginning of the page in Circular buffer area
Location.Network interface card can directly access Circular buffer area in this way.
After improving to the Packet capturing of polling mode, either small data packets or big data packet, Packet capturing ability are all obtained
Great raising is arrived.Therefore practical net of the improved poll Packet capturing method proposed by the present invention in high bandwidth or big flow
Flow monitoring can be preferably realized in network.In fact, when data package size is when 128 byte, improved Packet capturing side
Method has reached linear speed Packet capturing ability.
The above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although with reference to the foregoing embodiments
Invention is explained in detail, those skilled in the art should understand that: it still can be to aforementioned each implementation
Technical solution documented by example is modified or equivalent replacement of some of the technical features；And these modification or
Replacement, the spirit and scope for technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution.
Claims (7)
1. a kind of CDN network flow rate testing methods based on polling mechanism, which is characterized in that in network last consignment of data packet
When first data packet reaches predeterminated target equipment, operating system, system is notified to set the transmission data using network card equipment
It is standby to be registered in a device polling queue, and the interrupt response to the equipment is closed, simultaneity factor activates a traps, right
The network equipment registered in snoop queue is polled, therefrom read data packet.
2. a kind of CDN network flow rate testing methods based on polling mechanism according to claim 1, which is characterized in that behaviour
It is the basic environment of various application systems and business software operation as system.
3. a kind of CDN network flow rate testing methods based on polling mechanism according to claim 1, which is characterized in that its
Specific step is as follows for detection method:
Network card equipment will also receive the data packet that network interface card receives from a Circular buffer area read data packet, operating system
Data write-in or copy Circular buffer area to；
Data reading and writing in Circular buffer area, Circular buffer area in Packet capturing program and are in a manner of memory mapping
It is shared between system kernel；
The data received are written or copy Circular buffer area to, system kernel copies data packet to Circular buffer Qu Bingzeng
Big data packet write pointer；
The read data packet from Circular buffer area, Packet capturing program, which obtains data packet from Circular buffer area and increases data packet reading, to be referred to
Needle, the data packet newly received cover the buffer area being read, without distributing new memory.
4. a kind of CDN network flow rate testing methods based on polling mechanism according to claim 3, which is characterized in that ring
Shape buffer area is the cyclic buffer of a first in first out in operating system, is equipped with a read pointer and a write pointer, Xiang Tong
Believe that program provides the exclusive reference to buffer area, read pointer is directed toward data readable in buffer circle, and write pointer is directed toward annular
Writeable buffer area in buffer area realizes the reading data and write-in of buffer area by mobile read pointer and write pointer.
5. a kind of CDN network flow rate testing methods based on polling mechanism according to claim 3, which is characterized in that be
Overhead caused by storage allocation is reduced, the memory size in Circular buffer area is just distributed by operating system when socket creates
It is good, when avoiding usually being received by network card equipment data packet, then being handled by system, cause continually to distribute, releasing memory
The step of causing and cause overhead.
6. a kind of CDN network flow rate testing methods based on polling mechanism according to claim 3, which is characterized in that this
Detection method equally supports the poll to network card equipment, i.e. predeterminated target equipment is CPU, and the sending device for receiving poll is net
Card.
7. a kind of CDN network flow rate testing methods based on polling mechanism according to claim 3, which is characterized in that this
Invention is additionally provided with quota mechanism, i.e., for network card equipment every time to the maximum number of CPU transmission data packet, this numerical value is configurable
, it is calculated as P；If after network interface card sends P data packet, data packet, which receives in buffer area, still data, just by the network card equipment weight
New registration waits to the end of snoop queue and continues to submit data packet to CPU core when being polled to the equipment next time；Otherwise will
The network card equipment is nullified from snoop queue, and opens the response of device interrupt.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910093227.8A CN109889403A (en) | 2019-01-30 | 2019-01-30 | A kind of CDN network flow rate testing methods based on polling mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910093227.8A CN109889403A (en) | 2019-01-30 | 2019-01-30 | A kind of CDN network flow rate testing methods based on polling mechanism |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109889403A true CN109889403A (en) | 2019-06-14 |
Family
ID=66927459
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910093227.8A Pending CN109889403A (en) | 2019-01-30 | 2019-01-30 | A kind of CDN network flow rate testing methods based on polling mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109889403A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111371759A (en) * | 2020-02-25 | 2020-07-03 | 深信服科技股份有限公司 | Network data packet reading method, device, equipment and readable storage medium |
| CN111431757A (en) * | 2020-02-18 | 2020-07-17 | 平安科技（深圳）有限公司 | Virtual network flow acquisition method and device |
| CN112115134A (en) * | 2020-08-04 | 2020-12-22 | 北京金山云网络技术有限公司 | Data storage method and device, electronic equipment and storage medium |
| CN113079114A (en) * | 2021-06-04 | 2021-07-06 | 鹏城实验室 | Data packet receiving method, device, terminal and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108090003A (en) * | 2017-11-20 | 2018-05-29 | 广东睿江云计算股份有限公司 | A kind of method, the system of the promotion WEB server performance based on zero-copy |
-
2019
- 2019-01-30 CN CN201910093227.8A patent/CN109889403A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108090003A (en) * | 2017-11-20 | 2018-05-29 | 广东睿江云计算股份有限公司 | A kind of method, the system of the promotion WEB server performance based on zero-copy |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111431757A (en) * | 2020-02-18 | 2020-07-17 | 平安科技（深圳）有限公司 | Virtual network flow acquisition method and device |
| CN111431757B (en) * | 2020-02-18 | 2023-03-24 | 平安科技（深圳）有限公司 | Virtual network flow acquisition method and device |
| CN111371759A (en) * | 2020-02-25 | 2020-07-03 | 深信服科技股份有限公司 | Network data packet reading method, device, equipment and readable storage medium |
| CN112115134A (en) * | 2020-08-04 | 2020-12-22 | 北京金山云网络技术有限公司 | Data storage method and device, electronic equipment and storage medium |
| CN113079114A (en) * | 2021-06-04 | 2021-07-06 | 鹏城实验室 | Data packet receiving method, device, terminal and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109889403A (en) | A kind of CDN network flow rate testing methods based on polling mechanism | |
| US7103888B1 (en) | Split model driver using a push-push messaging protocol over a channel based network | |
| CN101267361B (en) | A high-speed network data packet capturing method based on zero duplication technology | |
| Barrera III | A Fast Mach Network IPC Implementation. | |
| US8671138B2 (en) | Network adapter with shared database for message context information | |
| US7836212B2 (en) | Reflecting bandwidth and priority in network attached storage I/O | |
| CN111431757B (en) | Virtual network flow acquisition method and device | |
| CN108600053B (en) | Wireless network data packet capturing method based on zero copy technology | |
| Brightwell et al. | An analysis of NIC resource usage for offloading MPI | |
| Steenkiste | Analyzing communication latency using the nectar communication processor | |
| CN101178694A (en) | Intel e1000 zero copy method | |
| CN104506379A (en) | Method and system for capturing network data | |
| CN111182008B (en) | Establishing socket connections in user space | |
| CN108090003A (en) | A kind of method, the system of the promotion WEB server performance based on zero-copy | |
| Tucker et al. | CMMD: Active messages on the CM-5 | |
| CN109857545A (en) | A kind of data transmission method and device | |
| CN113064846A (en) | Zero-copy data transmission method based on Rsockets protocol | |
| CN111600852A (en) | Firewall design method based on programmable data plane | |
| CN110874336B (en) | Distributed block storage low-delay control method and system based on Shenwei platform | |
| US9288163B2 (en) | Low-latency packet receive method for networking devices | |
| CN110266787B (en) | Hybrid cloud management system and method and computer equipment | |
| CN102902593A (en) | Protocol distribution processing system based on cache mechanism | |
| WO2001016742A2 (en) | Network shared memory | |
| US7623538B1 (en) | Hardware-based network interface per-ring resource accounting | |
| Noronha et al. | Early cancellation: an active NIC optimization for time-warp |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190614 |
|
| WD01 | Invention patent application deemed withdrawn after publication |