CN114884845A - Data integration-based multi-dimensional network traffic statistical method and device - Google Patents
Data integration-based multi-dimensional network traffic statistical method and device Download PDFInfo
- Publication number
- CN114884845A CN114884845A CN202210678781.4A CN202210678781A CN114884845A CN 114884845 A CN114884845 A CN 114884845A CN 202210678781 A CN202210678781 A CN 202210678781A CN 114884845 A CN114884845 A CN 114884845A
- Authority
- CN
- China
- Prior art keywords
- data
- flow
- network card
- network
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000010354 integration Effects 0.000 title claims abstract description 40
- 238000007619 statistical method Methods 0.000 title claims abstract description 23
- 238000000034 method Methods 0.000 claims abstract description 30
- 238000012546 transfer Methods 0.000 claims abstract description 23
- 238000004590 computer program Methods 0.000 claims description 18
- 238000003860 storage Methods 0.000 claims description 17
- 238000004140 cleaning Methods 0.000 claims description 12
- 238000010586 diagram Methods 0.000 description 17
- 238000004458 analytical method Methods 0.000 description 13
- 238000013515 script Methods 0.000 description 12
- 238000012545 processing Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000002354 daily effect Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000012216 screening Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 238000005206 flow analysis Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/04—Processing captured monitoring data, e.g. for logfile generation
Abstract
The invention provides a data integration-based multi-dimensional network traffic statistical method and device, which can be used in the technical field of big data. The method comprises the following steps: acquiring flow and network card data on a leaf server under a switch according to a preset data acquisition strategy; storing the flow and the network card data into a data file preset on a leaf server; and sending the flow and the network card data in the data file to a transfer server according to the data acquisition request so that the transfer server stores the flow and the network card data in a preset data table, and carrying out flow statistics according to the flow and the network card data of each leaf server stored in the preset data table. The device is used for executing the method. The data integration-based multi-dimensional network traffic statistical method and device provided by the embodiment of the invention can automatically integrate the network cards and traffic data of a plurality of leaf servers to carry out multi-dimensional network traffic statistics and realize diversification and automation of the network traffic statistics.
Description
Technical Field
The invention relates to the technical field of big data, in particular to a multidimensional network traffic statistical method and device based on data integration.
Background
The current network communication and flow data are collected based on the switch, and are stored in a cache server in a mirror image mode for users to use. However, the storage period of the collected data is short due to the limited memory of the cache server, and the leaf switch and the leaf server data are not collected into the cache server. For network cards and traffic data of LEAF servers under the LEAF switch, a common method is to rely on a server deployment tool to collect and download the data and then display the data locally, such as NMON and other tools. However, the tools have the disadvantages that the data granularity is fixed, only the network condition of a single server can be displayed, multidimensional aggregation display cannot be performed, and diversified analysis requirements cannot be met.
Disclosure of Invention
For solving the problems in the prior art, embodiments of the present invention provide a data integration-based multidimensional network traffic statistical method and apparatus, which can at least partially solve the problems in the prior art.
In a first aspect, the present invention provides a data integration-based multidimensional network traffic statistical method, including:
acquiring flow and network card data on a leaf server under a switch according to a preset data acquisition strategy;
storing the flow and the network card data into a data file preset on the leaf server;
and sending the flow and the network card data in the data file to a transfer server according to the data acquisition request so that the transfer server stores the flow and the network card data in a preset data table, and carrying out flow statistics according to the flow and the network card data of each leaf server stored in the preset data table.
Optionally, the obtaining, according to a preset data obtaining policy, traffic and network card data on a leaf server under a switch includes:
and acquiring the flow and network card data of the current time point on the leaf server under the switch at preset intervals.
Optionally, the method further includes:
and cleaning the flow and the network card data in the data file according to a preset data cleaning strategy.
In a second aspect, the present invention provides a data integration-based multidimensional network traffic statistical method, including:
acquiring flow and network card data in data files of leaf servers under each switch;
storing the traffic and network card data acquired from each leaf server into a preset data table, wherein the number of the preset data table is at least 1;
and carrying out flow statistics according to the flow of each leaf server and the network card data stored in the preset data table.
Optionally, the storing the traffic and the network card data acquired from each leaf server into a preset data table includes:
acquiring field values of fields in a preset data table from each flow and network card data;
and correspondingly storing the field values of all the fields acquired from each piece of flow and network card data into the preset data table.
Optionally, the performing traffic statistics according to the traffic of each leaf server and the network card data stored in the preset data table includes:
searching corresponding network data in the preset data table according to the IP or network card, date and time period requested to be inquired in the single-day traffic inquiry request;
according to the found network data, calculating the network flow of the IP or the network card requested to be inquired at preset time intervals in the time period under the date; and/or
Searching corresponding network data in the preset data table according to the IP or network card and date range requested to be searched in the flow trend query request;
and calculating the daily network flow of the IP or the network card requested to be inquired in the date range according to the searched network data.
In a third aspect, the present invention provides a data integration-based multidimensional network traffic statistic apparatus, including:
the acquisition module is used for acquiring the flow and the network card data on the leaf server under the switch according to a preset data acquisition strategy;
the storage module is used for storing the flow and the network card data into a data file preset on the leaf server;
and the sending module is used for sending the flow and the network card data in the data file to a transfer server according to a data acquisition request so that the transfer server stores the flow and the network card data in a preset data table, and carrying out flow statistics according to the flow and the network card data of each leaf server stored in the preset data table.
Optionally, the obtaining module is specifically configured to: and acquiring the flow and network card data of the current time point on the leaf server under the switch at preset intervals.
Optionally, the apparatus further comprises:
and the cleaning module is used for cleaning the flow and the network card data in the data file according to a preset data cleaning strategy.
In a fourth aspect, the present invention provides a data integration-based multidimensional network traffic statistic apparatus, including:
the acquisition module is used for acquiring the flow and network card data in the data file of the leaf server under each switch;
the storage module is used for storing the traffic and network card data acquired from each leaf server into a preset data table, wherein the number of the preset data table is at least 1;
and the statistical module is used for carrying out traffic statistics according to the traffic of each leaf server and the network card data stored in the preset data table.
Optionally, the saving module is specifically configured to:
acquiring field values of fields in a preset data table from each flow and network card data;
and correspondingly storing the field values of all the fields acquired from each piece of flow and network card data into the preset data table.
Optionally, the statistical module is specifically configured to:
searching corresponding network data in the preset data table according to the IP or network card, date and time period requested to be inquired in the single-day traffic inquiry request;
according to the found network data, calculating the network flow of the IP or the network card requested to be inquired at preset time intervals in the time period under the date; and/or
Searching corresponding network data in the preset data table according to the IP or network card and date range requested to be inquired in the flow trend inquiry request;
and calculating the daily network flow of the IP or the network card requested to be inquired in the date range according to the searched network data.
In a fifth aspect, the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the steps of the data integration-based multidimensional network traffic statistical method according to any of the above embodiments.
In a sixth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the data integration-based multidimensional network traffic statistical method according to any one of the above embodiments.
According to the data integration-based multi-dimensional network traffic statistical method and device, the network card and traffic data are collected by deploying the access tool at the lower leaf server of each switch; then, the flow and the network data of each leaf server are downloaded to a transfer server in a unified way, and the flow data and the corresponding network card data are recorded into a preset data table at the transfer server part; and deploying a network traffic analysis tool on the transit server, and carrying out multi-dimensional traffic statistics according to the traffic of each leaf server and the network card data stored in the preset data table. Therefore, network cards and flow data of a plurality of leaf servers can be automatically integrated to carry out multidimensional network flow statistics, and diversification and automation of the network flow statistics are realized.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts. In the drawings:
fig. 1 is a schematic flowchart of a data integration-based multidimensional network traffic statistical method according to an embodiment of the present invention.
Fig. 2 is a schematic flowchart of a data integration-based multidimensional network traffic statistical method according to another embodiment of the present invention.
Fig. 3 is a schematic diagram of a front-end display interface for single-day traffic query according to an embodiment of the present invention.
Fig. 4 is a schematic diagram of a front-end display interface of a traffic trend query according to an embodiment of the present invention.
Fig. 5 is a schematic structural diagram of a multidimensional network traffic statistic device based on data integration according to an embodiment of the present invention.
Fig. 6 is a schematic structural diagram of a multidimensional network traffic statistic device based on data integration according to an embodiment of the present invention.
Fig. 7 is a schematic physical structure diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
As used herein, the terms "first," "second," … …, etc. do not denote any order or order, nor are they used to limit the invention, but rather are used to distinguish one element from another element or operation described by the same technical terms.
As used herein, the terms "comprising," "including," "having," "containing," and the like are open-ended terms that mean including, but not limited to.
As used herein, "and/or" includes any and all combinations of the described items.
The execution subject of the data integration-based multi-dimensional network traffic statistical method provided by the embodiment of the invention includes but is not limited to a computer.
Fig. 1 is a schematic flow diagram of a data integration-based multidimensional network traffic statistical method according to an embodiment of the present invention, and as shown in fig. 1, the data integration-based multidimensional network traffic statistical method according to the embodiment of the present invention may be applied to a leaf server under a switch, where the method includes:
s101, acquiring flow and network card data on a leaf server under a switch according to a preset data acquisition strategy;
the method comprises the steps that a data acquisition strategy is preset aiming at a network card and flow data of a LEAF server under the LEAF switch, so that the flow and the network card data on the LEAF server under the switch are acquired according to the preset data acquisition strategy; the preset data acquisition strategy can be specifically timing acquisition and the like, a shell script can be deployed in each leaf server, and the flow and network card data (including fields such as the current time point, ip, network card name, network card state, inflow, outflow, and the number of receiving and sending packets) of the server are acquired according to the preset data acquisition strategy.
S102, storing the flow and the network card data into a data file preset on the leaf server;
in this step, the traffic and the network card data acquired from each leaf server are stored in a data file preset on the server, for example, a txt file.
S103, sending the flow and the network card data in the data file to a transfer server according to a data acquisition request, so that the transfer server stores the flow and the network card data in a preset data table, and carrying out flow statistics according to the flow and the network card data of each leaf server stored in the preset data table.
In this step, the data acquisition request may be sent by the transit server, and the transit server may specifically be an operation and maintenance server used for integrating traffic data; each leaf server uniformly sends the traffic and the network card data to the transfer server, specifically, the traffic and the network card data on each leaf server (for example, linux server) can be downloaded to the transfer server at regular time by using a remote transmission technology, and the forwarding server can be specifically realized by deploying timing and ftp scripts.
The transit server can be deployed with an analysis program, records the flow data and the corresponding network card data into a preset data table, is deployed with a network flow analysis tool, and performs multi-dimensional flow statistics according to the flow of each leaf server and the network card data stored in the preset data table.
According to the data integration-based multi-dimensional network flow statistical method provided by the embodiment of the invention, network cards and flow data are collected by deploying an access tool at a leaf server under each switch; then, the flow and the network data of each leaf server are downloaded to a transfer server in a unified way, and the flow data and the corresponding network card data are recorded into a preset data table at the transfer server part; and deploying a network traffic analysis tool on the transit server, and carrying out multi-dimensional traffic statistics according to the traffic of each leaf server and the network card data stored in the preset data table. Therefore, network cards and flow data of a plurality of leaf servers can be automatically integrated to carry out multidimensional network flow statistics, and diversification and automation of the network flow statistics are realized.
Optionally, the obtaining, according to a preset data obtaining policy, traffic and network card data on a leaf server under a switch includes: and acquiring the flow and network card data of the current time point on the leaf server under the switch at preset intervals.
In this embodiment, a crontab timing task may be added to the server, and the shell script is automatically executed every 1 minute to obtain the traffic and the network card data of the current time point on the leaf server under the switch. After a period of time, the corresponding traffic and network card data are recorded in a data file (e.g., txt file) on the leaf server, one data per minute.
Optionally, the method may further include: and cleaning the flow and the network card data in the data file according to a preset data cleaning strategy.
In this embodiment, in order to avoid the problem of server memory caused by an excessively large data amount of a data file (e.g., a txt file) on a leaf server, a shell script and a crottab timing task may be deployed, the shell script is executed every 24 hours, and the txt text content is emptied.
Fig. 2 is a schematic flow chart of a data integration-based multidimensional network traffic statistical method according to an embodiment of the present invention, and as shown in fig. 2, the data integration-based multidimensional network traffic statistical method according to the embodiment of the present invention may be applied to a transit server, where the method includes:
s201, obtaining flow and network card data in data files of leaf servers under all switches;
in this step, a bat script may be deployed on the relay server, and traffic and network card data may be obtained from each leaf server by an ftp command, for example, text content (i.e., traffic and network card data) of the txt file may be obtained; specifically, a windows timing task can be set in the transit server, and bat scripts are executed once per minute to obtain the latest traffic and network card data on the server.
S202, storing the flow and network card data acquired from each leaf server into a preset data table, wherein the number of the preset data table is at least 1;
in this step, an analysis tool can be deployed on the transfer server, the downloaded traffic and the network card data are analyzed, and the preset at least one data table is inserted, so that the network data integration of the plurality of leaf servers is completed.
And S203, carrying out flow statistics according to the flow of each leaf server and the network card data stored in the preset data table.
In this step, a network traffic analysis tool can be deployed on the transit server to form a multidimensional and visual traffic analysis page. When the tool is executed, the tool can make a data request to the transit server and receive and return the data request, and finally, an analysis result is displayed at the client. Specifically, a python script can be deployed on the transit server, and the python script comprises a front-end display code of the network traffic tool and a background data calling and calculating code, wherein the background calculating code is mainly used for realizing connection with a data table and carrying out data screening and calculation.
The data integration-based multi-dimensional network flow statistical method provided by the embodiment of the invention comprises the steps of collecting flow and network data of each leaf server, and inputting the collected flow data and corresponding network card data into a preset data table; and carrying out multi-dimensional traffic statistics according to the traffic of each leaf server and network card data stored in the preset data table by using a deployed network traffic analysis tool. Therefore, network cards and flow data of a plurality of leaf servers can be automatically integrated to carry out multidimensional network flow statistics, and diversification and automation of the network flow statistics are realized.
Optionally, the storing the traffic and the network card data acquired from each leaf server into a preset data table includes: acquiring field values of fields in a preset data table from each flow and network card data; and correspondingly storing the field values of all the fields acquired from each piece of flow and network card data into the preset data table.
In this embodiment, an analysis module may be deployed on the transit server to form a multi-type screening mode, and field combinations may be freely configured, so as to analyze, screen, and aggregate the original data collected from multiple leaf servers, decouple and recombine data elements, implement multi-dimensional integration of data, and then write data tables of different functional types respectively, thereby facilitating data management and improving the flexibility of later-stage network traffic pages.
For example, (1) a data table is first created in the oracle database, and is used to store the network data of each server, including fields such as time, ip, network card name, network card status, inflow, outflow, and number of packets received/transmitted. (2) Then, a python script is deployed on the transit server, and the source data downloaded by each leaf server is analyzed: and (3) according to the source data text format, removing corresponding values of fields such as IP, network cards, flow and the like, and inserting the numerical values into the newly-built data table in the step (1) item by item through a dictionary format. So far, the oracle newly-created data table contains the summary of the network history data of each server.
Optionally, the performing traffic statistics according to the traffic of each leaf server and the network card data stored in the preset data table includes:
searching corresponding network data in the preset data table according to the IP or network card, date and time period requested to be inquired in the single-day traffic inquiry request;
according to the searched network data, calculating the network flow of the IP or the network card requested to be inquired at preset time intervals in the time period under the date; and/or
Searching corresponding network data in the preset data table according to the IP or network card and date range requested to be inquired in the flow trend inquiry request;
and calculating the daily network flow of the IP or the network card requested to be inquired in the date range according to the searched network data.
In this embodiment, a python script may be deployed on the transit server, where the python script includes a front-end display code of the network traffic analysis tool and a background data call and calculation code, and the background calculation code is mainly used to implement connection of a data table and perform data screening and calculation. The front end display pages are shown in fig. 3 and 4.
As shown in fig. 3, a user may input an IP or network card name, a date, and a time period to be queried through the front-end display interface, click a query button to perform a single-day traffic query, call a tool code in the background, access an oracle data table, obtain and calculate network data corresponding to the input IP, network card, and date, and return a graph of network data such as an inflow amount and an outflow amount at the front end. The method supports the summary display of network data of a plurality of IP or network cards, and background codes can calculate and return results for flow statistics of a cluster or a plurality of network cards. The result data of the "single day traffic query" page may be in minutes, i.e., one data per minute, for outlier localization.
As shown in FIG. 4, the results data of the "traffic trend query" page may be in "days", i.e., one data per day, for overall trend analysis. The background query principle is the same as the single-day flow query, but the background query principle can be used for summarizing and calculating the data every day and then returning a result, and the background query principle also supports the summarizing query of multiple IPs and multiple network cards.
According to the above description, in the embodiment of the present invention, a self-research tool is deployed on a plurality of leaf servers (e.g., linux servers), and network cards and traffic data are collected; then, all server flow data are uniformly and immediately transmitted to a transfer server, an analysis program is deployed in the transfer server, and the flow data and corresponding network card data are recorded into a preset data table; and finally, a webpage tool is developed, and multi-dimensional flow display (including multi-IP or cluster total flow, time-sharing flow trend and daily flow trend) and abnormal comparison are visual and effective. The whole process involves three stages, including a data source end, an integration end and a client end, data of a plurality of data source ends are gathered to the integration end through an integration technology, and distributed to a plurality of client ends for use after being subjected to customized processing.
Fig. 5 is a schematic structural diagram of a data integration-based multidimensional network traffic statistic apparatus according to an embodiment of the present invention, and as shown in fig. 5, the data integration-based multidimensional network traffic statistic apparatus according to the embodiment of the present invention includes:
the acquiring module 31 is configured to acquire traffic and network card data on a leaf server under a switch according to a preset data acquiring policy;
the storage module 32 is configured to store the traffic and the network card data into a data file preset on the leaf server;
the sending module 33 is configured to send the traffic and the network card data in the data file to a transit server according to a data acquisition request, so that the transit server stores the traffic and the network card data in a preset data table, and performs traffic statistics according to the traffic and the network card data of each leaf server stored in the preset data table.
The data integration-based multi-dimensional network flow statistical device provided by the embodiment of the invention collects network card and flow data by deploying an access tool at a leaf server under each switch; then, the flow and the network data of each leaf server are downloaded to a transfer server in a unified way, and the flow data and the corresponding network card data are recorded into a preset data table at the transfer server part; and deploying a network traffic analysis tool on the transit server, and carrying out multi-dimensional traffic statistics according to the traffic of each leaf server and the network card data stored in the preset data table. Therefore, network cards and flow data of a plurality of leaf servers can be automatically integrated to carry out multidimensional network flow statistics, and diversification and automation of the network flow statistics are realized.
Optionally, the obtaining module is specifically configured to: and acquiring the flow and network card data of the current time point on the leaf server under the switch at preset intervals.
Optionally, the apparatus further comprises:
and the cleaning module is used for cleaning the flow and the network card data in the data file according to a preset data cleaning strategy.
The embodiment of the apparatus provided in the embodiment of the present invention may be specifically configured to execute the processing flows of the above method embodiments, and the functions of the apparatus are not described herein again, and refer to the detailed description of the above method embodiments.
Fig. 6 is a schematic structural diagram of a data integration-based multidimensional network traffic statistic apparatus according to an embodiment of the present invention, and as shown in fig. 6, the present invention provides a data integration-based multidimensional network traffic statistic apparatus, including:
an obtaining module 41, configured to obtain traffic and network card data in a data file of a leaf server under each switch;
a storage module 42, configured to store the traffic and the network card data acquired from each leaf server into a preset data table, where the number of the preset data table is at least 1;
and the statistical module 43 is configured to perform traffic statistics according to the traffic of each leaf server and the network card data stored in the preset data table.
According to the data integration-based multi-dimensional network flow statistical device provided by the embodiment of the invention, the collected flow data and the corresponding network card data are input into a preset data table by collecting the flow and the network data of each leaf server; and carrying out multi-dimensional traffic statistics according to the traffic of each leaf server and network card data stored in the preset data table by using a deployed network traffic analysis tool. Therefore, network cards and flow data of a plurality of leaf servers can be automatically integrated to carry out multidimensional network flow statistics, and diversification and automation of the network flow statistics are realized.
Optionally, the saving module is specifically configured to:
acquiring field values of fields in a preset data table from each flow and network card data;
and correspondingly storing the field values of all the fields acquired from each piece of flow and network card data into the preset data table.
Optionally, the statistical module is specifically configured to:
searching corresponding network data in the preset data table according to the IP or network card, date and time period requested to be inquired in the single-day traffic inquiry request;
according to the found network data, calculating the network flow of the IP or the network card requested to be inquired at preset time intervals in the time period under the date; and/or
Searching corresponding network data in the preset data table according to the IP or network card and date range requested to be inquired in the flow trend inquiry request;
and calculating the daily network flow of the IP or the network card requested to be inquired in the date range according to the searched network data.
The embodiment of the apparatus provided in the embodiment of the present invention may be specifically configured to execute the processing flows of the above method embodiments, and the functions of the apparatus are not described herein again, and refer to the detailed description of the above method embodiments.
It should be noted that the data integration-based multidimensional network traffic statistical method and apparatus provided by the embodiment of the present invention can be used in the financial field, and can also be used in any technical field except the financial field.
Fig. 7 is a schematic physical structure diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 7, the electronic device may include: a processor (processor)501, a communication Interface (Communications Interface)502, a memory (memory)503, and a communication bus 504, wherein the processor 501, the communication Interface 502, and the memory 503 are configured to communicate with each other via the communication bus 504. The processor 501 may call logic instructions in the memory 503 to perform the method described in any of the above embodiments.
In addition, the logic instructions in the memory 503 may be implemented in the form of software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The present embodiments disclose a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the methods provided by the above-described method embodiments.
The present embodiment provides a computer-readable storage medium storing a computer program that causes a computer to execute the method provided by the above-described method embodiments.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In the description herein, reference to the description of the terms "one embodiment," "a particular embodiment," "some embodiments," "for example," "an example," "a particular example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. A multidimensional network flow statistical method based on data integration is characterized by comprising the following steps:
acquiring flow and network card data on a leaf server under a switch according to a preset data acquisition strategy;
storing the flow and the network card data into a data file preset on the leaf server;
and sending the flow and the network card data in the data file to a transfer server according to the data acquisition request so that the transfer server stores the flow and the network card data in a preset data table, and carrying out flow statistics according to the flow and the network card data of each leaf server stored in the preset data table.
2. The method of claim 1, wherein the obtaining traffic and network card data on a leaf server under a switch according to a preset data obtaining policy comprises:
and acquiring the flow and network card data of the current time point on the leaf server under the switch at preset intervals.
3. The method according to claim 1 or 2, characterized in that the method further comprises:
and cleaning the flow and the network card data in the data file according to a preset data cleaning strategy.
4. A multidimensional network flow statistical method based on data integration is characterized by comprising the following steps:
acquiring flow and network card data in data files of leaf servers under each switch;
storing the traffic and network card data acquired from each leaf server into a preset data table, wherein the number of the preset data table is at least 1;
and carrying out flow statistics according to the flow of each leaf server and the network card data stored in the preset data table.
5. The method of claim 4, wherein the storing the traffic and network card data obtained from each of the leaf servers into a predetermined data table comprises:
acquiring field values of fields in a preset data table from each flow and network card data;
and correspondingly storing the field values of all the fields acquired from each piece of flow and network card data into the preset data table.
6. The method according to claim 4 or 5, wherein the performing traffic statistics according to the traffic and the network card data of each leaf server stored in the preset data table comprises:
searching corresponding network data in the preset data table according to the IP or network card, date and time period requested to be inquired in the single-day traffic inquiry request;
according to the found network data, calculating the network flow of the IP or the network card requested to be inquired at preset time intervals in the time period under the date; and/or
Searching corresponding network data in the preset data table according to the IP or network card and date range requested to be inquired in the flow trend inquiry request;
and calculating the daily network flow of the IP or the network card requested to be inquired in the date range according to the searched network data.
7. A multidimensional network flow statistic device based on data integration is characterized by comprising:
the acquisition module is used for acquiring the flow and the network card data on the leaf server under the switch according to a preset data acquisition strategy;
the storage module is used for storing the flow and the network card data into a data file preset on the leaf server;
and the sending module is used for sending the flow and the network card data in the data file to a transfer server according to a data acquisition request so that the transfer server stores the flow and the network card data in a preset data table, and carrying out flow statistics according to the flow and the network card data of each leaf server stored in the preset data table.
8. A multidimensional network flow statistic device based on data integration is characterized by comprising:
the acquisition module is used for acquiring the flow and network card data in the data file of the leaf server under each switch;
the storage module is used for storing the traffic and network card data acquired from each leaf server into a preset data table, wherein the number of the preset data table is at least 1;
and the statistical module is used for carrying out traffic statistics according to the traffic of each leaf server and the network card data stored in the preset data table.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the method of any one of claims 1 to 3 or 4 to 6 are implemented when the computer program is executed by the processor.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 3 or 4 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210678781.4A CN114884845A (en) | 2022-06-16 | 2022-06-16 | Data integration-based multi-dimensional network traffic statistical method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210678781.4A CN114884845A (en) | 2022-06-16 | 2022-06-16 | Data integration-based multi-dimensional network traffic statistical method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114884845A true CN114884845A (en) | 2022-08-09 |
Family
ID=82680765
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210678781.4A Pending CN114884845A (en) | 2022-06-16 | 2022-06-16 | Data integration-based multi-dimensional network traffic statistical method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114884845A (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114513428A (en) * | 2022-01-06 | 2022-05-17 | 新华三技术有限公司 | Data processing method and device |
-
2022
- 2022-06-16 CN CN202210678781.4A patent/CN114884845A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114513428A (en) * | 2022-01-06 | 2022-05-17 | 新华三技术有限公司 | Data processing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104838377B (en) | It is handled using mapping reduction integration events | |
| CN103942210A (en) | Processing method, device and system of mass log information | |
| CN103186834A (en) | Method and device of business process configuration | |
| CN106997431B (en) | Data processing method and device | |
| CN111611207B (en) | State data processing method and device and computer equipment | |
| CN113553310B (en) | Data acquisition method and device, storage medium and electronic equipment | |
| CN113360554A (en) | Method and equipment for extracting, converting and loading ETL (extract transform load) data | |
| CN110096521A (en) | Log information processing method and device | |
| CN110928681A (en) | Data processing method and device, storage medium and electronic device | |
| CN108809704B (en) | Data deduplication statistical method and device based on dynamic time window | |
| CN111125518A (en) | System and method for recommending household appliance information | |
| CN110580293A (en) | Entity relationship storage method and device | |
| CN114153862A (en) | Service data processing method, device, equipment and storage medium | |
| CN100413257C (en) | Network element warning data obtaining method | |
| CN107330031B (en) | Data storage method and device and electronic equipment | |
| CN112527824B (en) | Paging query method, paging query device, electronic equipment and computer-readable storage medium | |
| CN111026796B (en) | Multi-source heterogeneous data acquisition method, device, system, medium and equipment | |
| CN111459987A (en) | Cache updating method and device | |
| CN111814041A (en) | NPM package recommendation method and device, storage medium and computer equipment | |
| CN114884845A (en) | Data integration-based multi-dimensional network traffic statistical method and device | |
| CN106202302A (en) | Collecting method, Apparatus and system | |
| CN109684279B (en) | Data processing method and system | |
| CN115629848A (en) | Business processing method and device, electronic equipment and storage medium | |
| CN115794545A (en) | Automatic processing method of operation and maintenance data and related equipment thereof | |
| CN104317820A (en) | Statistical method and device of report |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |