CN114944996A - Data acquisition method and device and computer readable medium - Google Patents
Data acquisition method and device and computer readable medium Download PDFInfo
- Publication number
- CN114944996A CN114944996A CN202210890013.5A CN202210890013A CN114944996A CN 114944996 A CN114944996 A CN 114944996A CN 202210890013 A CN202210890013 A CN 202210890013A CN 114944996 A CN114944996 A CN 114944996A
- Authority
- CN
- China
- Prior art keywords
- dpdk
- working mode
- data acquisition
- network card
- monitoring result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/12—Network monitoring probes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0803—Configuration setting
- H04L41/0813—Configuration setting characterised by the conditions triggering a change of settings
- H04L41/082—Configuration setting characterised by the conditions triggering a change of settings the condition being updates or upgrades of network functionality
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the invention discloses a data acquisition method, a data acquisition device and a computer readable medium, and belongs to the technical field of computers. The method is applied to a first device, and one embodiment of the method comprises the following steps: acquiring a data acquisition request of a target network card; controlling a data plane development kit DPDK to carry out data acquisition on a target network card according to a corresponding DPDK working mode based on the data acquisition request; sending the collected data to a third party; therefore, the software and hardware resources of the first equipment can be monitored in real time, and the DPDK parameters are dynamically configured according to the monitoring result to update the DPDK working mode, so that the first equipment can quickly acquire network card data in a gigabit network, and the processing performance of the first equipment on the network message stream is improved.
Description
Technical Field
The invention belongs to the technical field of computers, and particularly relates to a data acquisition method, a data acquisition device and a computer readable medium.
Background
Processing network packet flows in real time is a prerequisite that intrusion detection systems, network protocol analysis, network firewalls, high performance communication systems, high performance routers, host routers, and other network monitoring systems must satisfy. At present, most network monitoring systems such as an intrusion detection system, a defense system, a firewall and the like are network message processing programs running in a Linux user space. Under a hundred-megabyte network environment, a common network card and a network data Packet Capture function library (Packet Capture library, abbreviated as libpcap) interface can enable a network message processing program running in a Linux user space to Capture a network message in real time; however, after the gigabit network era comes, the existing network message processing program is far from the requirement of capturing network data packets in real time under the general operating system. The main reasons for affecting the packet capturing performance include three aspects of system overhead, memory access and processing of the TCP/IP protocol stack, and the system overhead is the most influential factor.
Conventional Packet capture mechanisms include a Berkeley Packet Filter (abbreviated BPF) Packet capture mechanism, a libpcap Packet capture mechanism, and a pf _ ring Packet capture mechanism. The two components of the BPF data packet capturing mechanism comprise a forwarding part and a filtering part, wherein the forwarding part is responsible for extracting data packets from a link layer and forwarding the data packets to the filtering part; the filtering part decides the forwarding or discarding of the data packet according to the filtering rule, and then gives the data packet to the application layer. However, the BPF packet capture mechanism has the disadvantage that all actions are done in the kernel and portability is poor. The libpcap data packet capturing mechanism is to set the network card to be in a hybrid mode and copy all data packets flowing through the network card; then, the data packet is filtered by BPF and then delivered to a data packet buffer area in the kernel for storage; and finally, acquiring the data packet from the data packet buffer by the application program in a system calling mode. However, the drawback of the libpcap packet is that CPU cycles are wasted. The pf _ ring data packet capturing mechanism is that an application program directly accesses a kernel, firstly, a ring buffer area distinguished by a socket is added in the kernel, then, a data packet in a network card is directly copied to a buffer area corresponding to the socket, and then, the application program directly obtains the data packet of the corresponding buffer area through an interface function. However, the pf _ ring packet capture mechanism has the disadvantage that the above pre-processing is still in the kernel. Therefore, the copying, caching and system calling processes of the data packet in the kernel and user space transmission process need large system overhead, and the processing speed of the system on the network message flow is reduced.
Disclosure of Invention
The invention provides a data acquisition method, a data acquisition device and a computer readable medium. The method can save the copying and caching of the data packet in the process of transmitting the kernel and the user space and the process of system calling, thereby greatly improving the processing performance of the network message.
In order to achieve the above object, according to a first aspect of embodiments of the present application, a data acquisition method is applied to a first device, and includes: acquiring a data acquisition request of a target network card; controlling a data plane development kit DPDK to carry out data acquisition on a target network card according to a corresponding DPDK working mode based on the data acquisition request; and sending the collected data to a third party.
Optionally, the controlling, based on the data acquisition request, the DPDK to perform data acquisition on the target network card according to the corresponding DPDK working mode includes: monitoring the system resource surplus condition of the first equipment based on the data acquisition request; determining the current working mode of the DPDK based on the monitoring result; and controlling the DPDK to carry out data acquisition on the target network card based on the current working mode of the DPDK.
Optionally, the determining, based on the monitoring result, the current working mode of the DPDK includes: if the monitoring result meets a preset condition, determining a preset DPDK working mode as a DPDK current working mode; if the monitoring result does not meet the preset condition, updating the DPDK working mode based on the monitoring result; and determining the updated DPDK working mode as the current working mode of the DPDK.
Optionally, the preset DPDK working mode is obtained as follows: acquiring a DPDK configuration request; and configuring parameters of the DPDK based on the configuration request to obtain a preset DPDK working mode.
Optionally, the updating the DPDK operation mode based on the monitoring result includes: acquiring system resources required by the DPDK when acquiring target network card data; and updating the DPDK working mode by reconfiguring the DPDK parameters based on the monitoring result and the system resources required by the DPDK.
Optionally, the method further includes: monitoring the process of collecting network card data aiming at the DPDK; and if the monitoring result meets the preset condition, updating the current DPDK working mode by reconfiguring the parameters of the DPDK.
In order to achieve the above object, according to a second aspect of embodiments of the present application, there is provided a data acquisition apparatus applied to a first device, including: the acquisition module is used for acquiring a data acquisition request of the target network card; the control module is used for controlling the data plane development kit DPDK to carry out data acquisition on the target network card according to the corresponding DPDK working mode based on the data acquisition request; and the sending module is used for sending the acquired data to a third party.
Optionally, the control module includes: the monitoring unit is used for monitoring the system resource residual condition of the first equipment based on the data acquisition request; the determining unit is used for determining the current working mode of the DPDK based on the monitoring result; and the control unit is used for controlling the DPDK to carry out data acquisition on the target network card based on the current working mode of the DPDK.
Optionally, the determining unit includes: the first determining subunit is configured to determine a preset DPDK working mode as a current DPDK working mode if the monitoring result meets a preset condition; the second determining subunit is used for updating the DPDK working mode based on the monitoring result if the monitoring result does not meet the preset condition; and determining the updated DPDK working mode as the current working mode of the DPDK.
To achieve the above object, according to a third aspect of embodiments of the present application, there is provided a computer readable medium having stored thereon a computer program which, when executed by a processor, implements the method according to the first aspect.
Compared with the prior art, the embodiment of the invention provides a data acquisition method, a data acquisition device and a computer readable medium; the method is applied to first equipment, and specifically comprises the following steps: firstly, acquiring a data acquisition request of a target network card; then based on the data acquisition request, controlling a data plane development kit DPDK to acquire data of the target network card according to a corresponding DPDK working mode; and then sending the collected data to a third party. Therefore, the software resources of the first device can be monitored in real time, and the DPDK parameters are dynamically configured according to the monitoring result to update the DPDK working mode, so that the first device can rapidly acquire network card data in a gigabit network, and the processing performance of the first device on the network message stream is improved.
Drawings
Some specific embodiments of the invention will be described in detail hereinafter, by way of illustration and not limitation, with reference to the accompanying drawings. The same reference numbers in the drawings identify the same or similar elements or components. Those skilled in the art will appreciate that the drawings are not necessarily drawn to scale. In the drawings:
fig. 1 is a schematic flow chart of a data acquisition method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a data acquisition method according to another embodiment of the present invention;
fig. 3 is a schematic structural diagram of a data acquisition device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic flow chart of a data acquisition method according to an embodiment of the present invention.
A method of data acquisition, the method comprising at least the steps of:
s101, acquiring a data acquisition request of a target network card;
s102, controlling a data plane development kit DPDK to carry out data acquisition on a target network card according to a corresponding DPDK working mode based on a data acquisition request;
and S103, sending the acquired data to a third party.
In S101, a data acquisition request of the target network card is acquired based on a trigger of the user on the display interface of the first device.
In S102, Inter DPDK includes PMD driver, stream classification process, loop free queue, MBUF management, and environment abstraction layer.
The first device controls the DPDK to perform data acquisition on the target network card according to the data acquisition request and a preset DPDK working mode, or the first device controls the DPDK to perform data acquisition on the target network card according to the data acquisition request and an updated DPDK working mode.
In S103, the collected data is sent to an application program of a third party, the third party application program analyzes and processes the collected data, and finally the third party sends the processed data to the user.
In the embodiment, the DPDK is used for directly acquiring data from the target network card and sending the acquired data to the third party, so that copying, buffering and system calling required by a data packet in the process of transmitting the data packet in the first device kernel and the user space are omitted, the processing performance of the network message is greatly improved, a set of Linux-based data packet capturing system is further realized by using the DPDK, and the Linux-based data packet capturing system is successfully applied to a gigabit network security protection system.
It should be understood that, in the various embodiments of the present invention, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and the inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
Fig. 2 is a schematic flow chart of a data acquisition method according to another embodiment of the present invention. A data acquisition method at least comprises the following steps:
s201, acquiring a data acquisition request of a target network card;
s202, monitoring the system resource surplus condition of the first equipment based on the data acquisition request;
s203, determining the current working mode of the DPDK based on the monitoring result;
s204, controlling the DPDK to carry out data acquisition on the target network card based on the current working mode of the DPDK;
and S205, sending the acquired data to a third party.
The implementation processes of step S201 and step S205 are similar to the implementation processes of step S101 and step S103 in fig. 1, respectively, and here, the implementation processes are not limited too much.
In S202 and S204, here, the system resource is used to indicate the CPU resource and the large page memory. The purpose of monitoring the remaining system resources is to determine how much system resources are available for the DPDK.
If the monitoring result meets a preset condition, determining a preset DPDK working mode as a DPDK current working mode; if the monitoring result does not meet the preset condition, updating the DPDK working mode based on the monitoring result; and determining the updated DPDK working mode as the current working mode of the DPDK. Specifically, if the monitoring result indicates that the CPU resource is smaller than a first preset threshold and the large-page memory is smaller than a second preset threshold, it is determined that the remaining amount of the system resource does not meet the preset condition, and the DPDK working mode is automatically updated. And if the monitoring result indicates that the CPU resource is smaller than the first preset threshold, determining that the residual quantity of the system resource does not meet the preset condition, automatically updating the DPDK working mode, and determining the updated DPDK working mode as the current DPDK working mode. And if the monitoring result represents that the large-page memory is smaller than a second preset threshold, determining that the residual quantity of the system resources does not meet the preset condition, and automatically updating the DPDK working mode. And determining the updated DPDK working mode as the current working mode of the DPDK.
And if the monitoring result indicates that the CPU resource is greater than a first preset threshold and the large-page memory is greater than a second preset threshold, determining that the surplus of the system resource meets a preset condition, and determining a preset DPDK working mode before the data acquisition request is acquired as a DPDK current working mode. Before acquiring the data acquisition request, configuring parameters of the DPDK based on key parameters to obtain a plurality of preset DPDK working modes; and then switching the preset DPDK working mode among a plurality of preset DPDK working modes based on the updating request of the DPDK working mode to obtain the updated DPDK working mode. The DPDK mode of operation may also be updated by reconfiguring DPDK parameters.
In this embodiment, the remaining condition of the system resources is monitored based on the data acquisition request, the DPDK working mode is automatically updated based on the monitoring result, and then the data acquisition is performed on the target network card based on the updated DPDK working mode; therefore, the software and hardware resources of the system can be monitored in real time, the Inter Dpdk is dynamically configured, and the Inter Dpdk is utilized to realize a high-performance network data packet capturing system, so that the system is successfully applied to a gigabit network security protection system.
In a preferred embodiment, the updating the DPDK operation mode based on the monitoring result includes: acquiring system resources required by the DPDK when acquiring target network card data; and updating the DPDK working mode by reconfiguring DPDK parameters based on the monitoring result and the system resources required by the DPDK.
Therefore, the working mode of the DPDK in the first device can be effectively updated by reconfiguring the DPDK parameters, and the first device can acquire data of the target network card conveniently.
In another preferred embodiment, the preset DPDK mode of operation is obtained as follows: acquiring a DPDK configuration request; and configuring the parameters of the DPDK based on the configuration request to obtain a preset DPDK working mode. Therefore, before the data acquisition request is acquired, the DPDK working parameters are automatically configured based on the configuration request, so that the DPDK working mode of the first device is acquired.
In a preferred further embodiment, the data acquisition method further comprises: monitoring the process of DPDK acquiring network card data; and if the monitoring result meets the preset condition, updating the current DPDK working mode by reconfiguring the parameters of the DPDK.
For example, when the DPDK is running, if the monitoring result indicates that the acquired network card data is smaller than a third preset threshold, it is determined that the network card is down, the DPDK is automatically initialized by reconfiguring parameters of the DPDK, and the network card is removed from the monitoring list.
Therefore, the network card data acquisition process can be monitored in real time, and the DPDK working mode can be adjusted in real time according to the monitoring result, so that the effective configuration of system resources is facilitated, the network card data acquisition efficiency is improved, and the requirement of capturing network data packets in real time based on a general system in the gigabit network era is met.
The interactive process of the data acquisition method is specifically realized as follows:
s301, the first equipment management platform sends a data acquisition request to the DPDK;
s302, the DPDK acquires data of the target network card according to the data acquisition request;
and S303, the DPDK sends the acquired data to an application layer.
The management platform of the first device has the following functions: DPDK configuration, DPDK control, network card monitoring, and system monitoring. The DPDK configuration mainly refers to the DPDK configuration through a management platform, and the configuration process of the traditional manual DPDK is greatly simplified. The DPDK control is mainly that the management platform can flexibly select the operating mode of the DPDK through information obtained from the system or the network. The network card monitoring mainly monitors data information from a network in real time and calls a system and DPDK resources. The system monitoring mainly controls the DPDK to work in real time by analyzing the condition of system resources.
Inter DPDK includes PMD driver, stream classification, loop free queue, MBUF management, and environment abstraction layer. The Inter DPDK round-robin Mode Driver (PMD) improves the efficiency of transmitting or receiving data frames by a zero-copy mechanism without interruption and with the data frames going in and out of the application buffer. Flow Classification (Flow Classification) of Inter DPDK provides an optimized lookup algorithm for N-tuple matching and LPM (longest prefix matching). The Ring Queue of Inter DPDK provides a lock-free mechanism aiming at the Queue and the Queue of a single or a plurality of data packet producers and a single data packet consumer, thereby effectively reducing the system overhead. The Mbuf buffer management of Inter DPDK is used to allocate memory buffers and encapsulate actual data frames for use by applications by creating Mbuf objects. An Environment Abstraction Layer (EAL) of the Inter DPDK is used for system initialization such as PMD initialization, CPU kernel and DPDK thread configuration or binding, and large Page Huge Page memory setting.
Therefore, in the embodiment, a set of Linux-based data packet capturing system is realized by using the DPDK, the system can monitor software and hardware resources of the system in real time, and configure Inter DPDK dynamically, so that the processing performance of network messages is improved, and high-speed traffic collection of the network card is realized by using the Inter DPDK
It should be understood that, in various embodiments of the present invention, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and the inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
Fig. 3 is a schematic structural diagram of a data acquisition device according to an embodiment of the present invention. The data acquisition device, the device 300 is applied to the first equipment, specifically includes: an obtaining module 301, configured to obtain a data acquisition request of a target network card; the control module 302 is configured to control the data plane development kit DPDK to perform data acquisition on the target network card according to a corresponding DPDK working mode based on the data acquisition request; a sending module 303, configured to send the acquired data to a third party.
In an alternative embodiment, the control module 302 includes: the monitoring unit is used for monitoring the system resource residual condition of the first equipment based on the data acquisition request; the determining unit is used for determining the current working mode of the DPDK based on the monitoring result; and the control unit is used for controlling the DPDK to carry out data acquisition on the target network card based on the current working mode of the DPDK.
In an alternative embodiment, the determining unit comprises: the first determining subunit is configured to determine a preset DPDK working mode as a current DPDK working mode if the monitoring result meets a preset condition; the second determining subunit is configured to update the DPDK working mode based on the monitoring result if the monitoring result does not satisfy a preset condition; and the third determining subunit is used for determining the updated DPDK working mode as the current DPDK working mode.
In an optional embodiment, the preset DPDK operation mode is obtained as follows: acquiring a DPDK configuration request; and configuring the parameters of the DPDK based on the configuration request to obtain a preset DPDK working mode.
In an alternative embodiment, the second determining subunit includes: the acquiring unit is used for acquiring system resources required by the DPDK when acquiring the data of the target network card; and the updating unit is used for updating the DPDK working mode by reconfiguring the DPDK parameters based on the monitoring result and the system resources required by the DPDK.
In an alternative embodiment, the data acquisition device further comprises: the monitoring module is used for monitoring the process of collecting network card data aiming at the DPDK; and the updating module is used for updating the current DPDK working mode by reconfiguring the parameters of the DPDK if the monitoring result meets the preset conditions.
The device can execute the data acquisition method provided by the embodiment of the invention, and has the corresponding functional modules and beneficial effects of executing the data acquisition method. For technical details that are not described in detail in this embodiment, reference may be made to the data acquisition method provided in the embodiment of the present invention.
The present invention also provides an electronic device comprising: a processor; a memory for storing the processor-executable instructions; the processor is used for reading the executable instruction from the memory and executing the instruction to realize the data acquisition method.
In addition to the above-described methods and apparatus, embodiments of the present application may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to the various embodiments of the present application described in the "exemplary methods" section of this specification, above.
The computer program product may be written with program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present application may also be a computer readable storage medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform steps in a method according to embodiments of the present application described in the "exemplary methods" section above in the present specification.
The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The basic principles of the present application have been described above with reference to specific embodiments, but it should be noted that advantages, effects, etc. mentioned in the present application are only examples and are not limiting, and the advantages, effects, etc. must not be considered to be possessed by various embodiments of the present application. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the foregoing disclosure is not intended to be exhaustive or to limit the disclosure to the precise details disclosed.
The block diagrams of devices, apparatuses, systems referred to in this application are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
It should also be noted that in the devices, apparatuses, and methods of the present application, the components or steps may be decomposed and/or recombined. These decompositions and/or recombinations are to be considered as equivalents of the present application.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the application to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. A data acquisition method is applied to first equipment and comprises the following steps:
acquiring a data acquisition request of a target network card;
controlling a data plane development kit DPDK to carry out data acquisition on a target network card according to a corresponding DPDK working mode based on the data acquisition request;
and sending the collected data to a third party.
2. The method of claim 1, wherein the controlling, based on the data acquisition request, the DPDK to perform data acquisition on the target network card according to the corresponding DPDK working mode comprises:
monitoring the system resource residual condition of the first equipment based on the data acquisition request;
determining the current working mode of the DPDK based on the monitoring result;
and controlling the DPDK to carry out data acquisition on the target network card based on the current working mode of the DPDK.
3. The method of claim 2, wherein the determining the current operating mode of the DPDK based on the monitoring result comprises:
if the monitoring result meets a preset condition, determining a preset DPDK working mode as a DPDK current working mode;
if the monitoring result does not meet the preset condition, updating the DPDK working mode based on the monitoring result; and determining the updated DPDK working mode as the current working mode of the DPDK.
4. The method of claim 3, wherein the preset DPDK working mode is obtained by:
acquiring a DPDK configuration request;
and configuring the parameters of the DPDK based on the configuration request to obtain a preset DPDK working mode.
5. The method of claim 3, wherein the updating the DPDK operation mode based on the monitoring result comprises:
acquiring system resources required by the DPDK when acquiring target network card data;
and updating the DPDK working mode by reconfiguring the DPDK parameters based on the monitoring result and the system resources required by the DPDK.
6. The method of claim 1, further comprising:
monitoring the process of collecting network card data aiming at the DPDK;
and if the monitoring result meets the preset condition, updating the current DPDK working mode by reconfiguring the parameters of the DPDK.
7. A data acquisition device, applied to a first device, comprising:
the acquisition module is used for acquiring a data acquisition request of the target network card;
the control module is used for controlling the data plane development kit DPDK to carry out data acquisition on the target network card according to the corresponding DPDK working mode based on the data acquisition request;
and the sending module is used for sending the acquired data to a third party.
8. The apparatus of claim 7, wherein the control module comprises:
the monitoring unit is used for monitoring the system resource residual condition of the first equipment based on the data acquisition request;
the determining unit is used for determining the current working mode of the DPDK based on the monitoring result;
and the control unit is used for controlling the DPDK to carry out data acquisition on the target network card based on the current working mode of the DPDK.
9. The apparatus of claim 8, wherein the determining unit comprises:
the first determining subunit is configured to determine a preset DPDK working mode as a current DPDK working mode if the monitoring result meets a preset condition;
the second determining subunit is configured to update the DPDK working mode based on the monitoring result if the monitoring result does not satisfy a preset condition; and determining the updated DPDK working mode as the current working mode of the DPDK.
10. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210890013.5A CN114944996B (en) | 2022-07-27 | 2022-07-27 | Data acquisition method and device and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210890013.5A CN114944996B (en) | 2022-07-27 | 2022-07-27 | Data acquisition method and device and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114944996A true CN114944996A (en) | 2022-08-26 |
| CN114944996B CN114944996B (en) | 2022-09-30 |
Family
ID=82910646
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210890013.5A Active CN114944996B (en) | 2022-07-27 | 2022-07-27 | Data acquisition method and device and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114944996B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107911237A (en) * | 2017-11-10 | 2018-04-13 | 南京邮电大学 | Data packet quick determination method in a kind of user's space based on DPDK |
| CN108062269A (en) * | 2017-12-05 | 2018-05-22 | 上海交通大学 | A kind of computing resource elastic telescopic method and system based on DPDK |
| US20180331976A1 (en) * | 2017-05-12 | 2018-11-15 | Solarflare Communications, Inc. | Data processing system |
| CN109309626A (en) * | 2018-09-10 | 2019-02-05 | 南京知常容信息技术有限公司 | A kind of high-speed network data packet capturing shunting and caching method based on DPDK |
| CN109981403A (en) * | 2019-03-05 | 2019-07-05 | 北京勤慕数据科技有限公司 | Virtual machine network data traffic monitoring method and device |
| CN111107081A (en) * | 2019-12-17 | 2020-05-05 | 互联网域名系统北京市工程研究中心有限公司 | DPDK-based multi-process DNS service method and system |
| JP2021048505A (en) * | 2019-09-19 | 2021-03-25 | 株式会社Pfu | Information processing device, information processing method, and program |
| CN114125015A (en) * | 2021-11-30 | 2022-03-01 | 上海斗象信息科技有限公司 | Data acquisition method and system |
-
2022
- 2022-07-27 CN CN202210890013.5A patent/CN114944996B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180331976A1 (en) * | 2017-05-12 | 2018-11-15 | Solarflare Communications, Inc. | Data processing system |
| CN107911237A (en) * | 2017-11-10 | 2018-04-13 | 南京邮电大学 | Data packet quick determination method in a kind of user's space based on DPDK |
| CN108062269A (en) * | 2017-12-05 | 2018-05-22 | 上海交通大学 | A kind of computing resource elastic telescopic method and system based on DPDK |
| CN109309626A (en) * | 2018-09-10 | 2019-02-05 | 南京知常容信息技术有限公司 | A kind of high-speed network data packet capturing shunting and caching method based on DPDK |
| CN109981403A (en) * | 2019-03-05 | 2019-07-05 | 北京勤慕数据科技有限公司 | Virtual machine network data traffic monitoring method and device |
| JP2021048505A (en) * | 2019-09-19 | 2021-03-25 | 株式会社Pfu | Information processing device, information processing method, and program |
| CN111107081A (en) * | 2019-12-17 | 2020-05-05 | 互联网域名系统北京市工程研究中心有限公司 | DPDK-based multi-process DNS service method and system |
| CN114125015A (en) * | 2021-11-30 | 2022-03-01 | 上海斗象信息科技有限公司 | Data acquisition method and system |
Non-Patent Citations (1)
| Title |
|---|
| 曾理等: "DPDK技术应用研究综述", 《网络新媒体技术》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114944996B (en) | 2022-09-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8176300B2 (en) | Method and apparatus for content based searching | |
| US8095675B2 (en) | Priority and bandwidth specification at mount time of NAS device volume | |
| CN114244560B (en) | Flow processing method and device, electronic equipment and storage medium | |
| US9356844B2 (en) | Efficient application recognition in network traffic | |
| CN111385147B (en) | Fault simulation method, device and computer readable storage medium | |
| CN113364804B (en) | Method and device for processing flow data | |
| CN113037687A (en) | Flow identification method and electronic equipment | |
| US20170295193A1 (en) | Adaptive anomaly context description | |
| CN113014571B (en) | Method, device and storage medium for processing access request | |
| CN111181850B (en) | Data packet flooding suppression method, device and equipment and computer storage medium | |
| KR20110083084A (en) | Apparatus and method for operating server by using virtualization technology | |
| KR20190029486A (en) | Elastic honeynet system and method for managing the same | |
| KR101916676B1 (en) | Method for collecting cyber threat intelligence data and system thereof | |
| CN112929376A (en) | Flow data processing method and device, computer equipment and storage medium | |
| CN113595927A (en) | Method and device for processing mirror flow in bypass mode | |
| US20190273720A1 (en) | Firewall rule remediation for improved network security and performance | |
| CN115023699A (en) | Malicious process detection method and device, electronic device and storage medium | |
| CN116233018A (en) | Message processing method and device, electronic equipment and storage medium | |
| CN114567481A (en) | Data transmission method and device, electronic equipment and storage medium | |
| CN114944996B (en) | Data acquisition method and device and computer readable medium | |
| US20210306357A1 (en) | Sorting device, communication system, and sorting method | |
| CN106528267A (en) | Xen privileged domain-based network communication monitoring system and method | |
| WO2008121690A2 (en) | Data and control plane architecture for network application traffic management device | |
| US10341299B2 (en) | Collecting firewall flow records of a virtual infrastructure | |
| US8149709B2 (en) | Serialization queue framework for transmitting packets |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |