CN113986564A - Application data flow monitoring method and device, computer equipment and medium - Google Patents
Application data flow monitoring method and device, computer equipment and medium Download PDFInfo
- Publication number
- CN113986564A CN113986564A CN202111108992.6A CN202111108992A CN113986564A CN 113986564 A CN113986564 A CN 113986564A CN 202111108992 A CN202111108992 A CN 202111108992A CN 113986564 A CN113986564 A CN 113986564A
- Authority
- CN
- China
- Prior art keywords
- peak value
- early warning
- load information
- preset
- application
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/543—User-generated data transfer, e.g. clipboards, dynamic data exchange [DDE], object linking and embedding [OLE]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
Abstract
The invention discloses a method and a device for monitoring the flow of application data, a computer device and a storage medium, wherein the method comprises the following steps: the method comprises the steps of acquiring flow data of each application in a preset period in real time based on an IP data flow information output protocol IPFIX, analyzing the basic performance data to obtain load information of a node server corresponding to the application as current load information, obtaining historical load information corresponding to the application for each application, inputting the historical load information and the current load information into a reinforcement learning model to predict a peak value to obtain a predicted peak value, judging whether the predicted peak value exceeds a preset early warning threshold value, and executing a preset flow management and control measure if the predicted peak value exceeds the preset early warning threshold value.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a method and an apparatus for monitoring traffic of application data, a computer device, and a medium.
Background
With the rapid development of computer technology, learning, working, shopping, information query and service handling are more and more convenient through the internet, internet transactions of some large enterprises are more and more, in order to meet the increasing user access requirements, the internet enterprises need to adopt a cluster mode to perform data access response, but the cluster relates to more node servers, each node server manages different application services, and due to the difference of the application services, the traffic control modes of different applications of the cluster cannot be unified and standardized, when the data traffic of the applications has potential safety hazards, how to discover and process in time becomes a difficult problem to be solved urgently.
Disclosure of Invention
The embodiment of the invention provides a method and a device for monitoring the flow of application data, computer equipment and a storage medium, which are used for improving the safety of the flow monitoring of the application data.
In order to solve the foregoing technical problem, an embodiment of the present application provides a method for monitoring traffic of application data, including:
acquiring flow data of each application in a preset period in real time based on an IP data flow information output protocol IPFIX as basic performance data;
analyzing the basic performance data to obtain load information of a node server corresponding to the application, wherein the load information is used as current load information;
acquiring historical load information corresponding to each application, and inputting the historical load information and the current load information into a reinforcement learning model for peak value prediction to obtain a predicted peak value;
and judging whether the predicted peak value exceeds a preset early warning threshold value, and if the predicted peak value exceeds the preset early warning peak value, executing a preset flow control measure.
Optionally, the basic performance data comprises one or more of an IP address, an access port, a network protocol, throughput, access duration, network latency, network jitter, and layer 2 to layer 7 protocol.
Optionally, the performing peak prediction on the historical load information and the current load information in a reinforcement learning model to obtain a predicted peak value includes:
inputting the historical load information into a prediction unit in the reinforcement learning model for prediction to obtain a first load peak prediction value, wherein the prediction unit is a neural network model;
acquiring a real peak value corresponding to the historical load information as a second load peak value;
inputting the first load peak value and the second load peak value into the reinforcement learning model, calculating a difference value between the first load peak value and the second load peak value based on a loss function of the reinforcement learning model, and taking the difference value as the reward function;
updating the reinforcement learning model based on a gradient ascent method and the reward function;
and inputting the current load information and the reward function into the updated reinforcement learning model for decision making to obtain a decision making result, and taking a peak value in the decision making result as the prediction peak value.
Optionally, if the predicted peak value exceeds a preset early warning peak value, executing a preset flow management and control measure includes:
if the predicted peak value exceeds a preset early warning peak value and is in a first early warning range, sending early warning information;
and if the predicted peak value exceeds the preset early warning peak value and is within a second early warning range, executing corresponding emergency early warning measures and limiting the access of data traffic.
Optionally, after the determining whether the predicted peak value exceeds a preset early warning threshold value and executing a preset traffic control measure if the predicted peak value exceeds the preset early warning peak value, the method for monitoring the traffic of the application data further includes:
constructing a visual chart according to the flow data collected in real time and the predicted peak value, wherein the visual chart comprises at least one of a trend chart, a frequency chart, a proportion chart or a data table;
displaying the visualization chart in a visualization interface.
In order to solve the foregoing technical problem, an embodiment of the present application further provides a traffic monitoring apparatus for application data, including:
the data acquisition module is used for acquiring flow data of each application in a preset period in real time based on an IP data flow information output protocol IPFIX as basic performance data;
the data analysis module is used for analyzing the basic performance data to obtain load information of a node server corresponding to the application as current load information;
the peak value prediction module is used for acquiring historical load information corresponding to each application, and inputting the historical load information and the current load information into a reinforcement learning model for peak value prediction to obtain a predicted peak value;
and the early warning module is used for judging whether the predicted peak value exceeds a preset early warning threshold value or not, and if the predicted peak value exceeds the preset early warning peak value, executing a preset flow control measure.
Optionally, the peak prediction module comprises:
the first peak prediction unit is used for inputting the historical load information into a prediction unit in the reinforcement learning model for prediction to obtain a first load peak prediction value, wherein the prediction unit is a neural network model;
the second peak value acquisition unit is used for taking a real peak value corresponding to the historical load information as a second load peak value;
a reward function calculation unit, configured to input the first load peak value and the second load peak value into the reinforcement learning model, calculate a difference value between the first load peak value and the second load peak value based on a loss function of the reinforcement learning model, and use the difference value as the reward function;
a model updating unit for updating the reinforcement learning model based on a gradient ascent method and the reward function;
and the prediction peak value determining unit is used for inputting the current load information and the reward function into the updated reinforcement learning model for decision making to obtain a decision making result, and taking a peak value in the decision making result as the prediction peak value.
Optionally, the early warning module includes:
the first early warning unit is used for sending early warning information if the predicted peak value exceeds a preset early warning peak value and is in a first early warning range;
and the second early warning unit is used for executing corresponding emergency early warning measures and limiting the access of data traffic if the predicted peak value exceeds the preset early warning peak value and is within a second early warning range.
Optionally, the device for monitoring the flow of the application data further includes:
the visual chart construction module is used for constructing a visual chart according to the flow data collected in real time and the prediction peak value, wherein the visual chart comprises at least one of a trend chart, a frequency chart, a proportion chart or a data table;
and the visual chart display module is used for displaying the visual chart in a visual interface.
In order to solve the above technical problem, an embodiment of the present application further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the method for monitoring traffic of application data when executing the computer program.
In order to solve the above technical problem, an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the steps of the method for monitoring traffic of application data are implemented.
The method, the device, the computer equipment and the storage medium for monitoring the flow of the application data, provided by the embodiment of the invention, are based on an IP data flow information output protocol IPFIX, collect the flow data of each application in a preset period in real time to serve as basic performance data, analyze the basic performance data to obtain load information of a node server corresponding to the application to serve as current load information, obtain historical load information corresponding to the application for each application, input the historical load information and the current load information into a reinforcement learning model to perform peak value prediction to obtain a prediction peak value, judge whether the prediction peak value exceeds a preset early warning threshold value, and execute a preset flow control measure if the prediction peak value exceeds the preset early warning peak value, so that the real-time monitoring of the data flow is realized, and the safety of data flow response and the timeliness of early warning are improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
FIG. 1 is a schematic diagram of an application environment of the present application;
FIG. 2 is a flow diagram of one embodiment of a method for traffic monitoring of application data of the present application;
FIG. 3 is a schematic block diagram of one embodiment of a traffic monitoring apparatus for application data according to the present application;
FIG. 4 is a schematic block diagram of one embodiment of a computer device according to the present application.
Detailed Description
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "including" and "having," and any variations thereof, in the description and claims of this application and the description of the above figures are intended to cover non-exclusive inclusions. The terms "first," "second," and the like in the description and claims of this application or in the above-described drawings are used for distinguishing between different objects and not for describing a particular order.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, as shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like.
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture experts Group Audio Layer III, motion Picture experts compression standard Audio Layer 3), MP4 players (Moving Picture experts Group Audio Layer IV, motion Picture experts compression standard Audio Layer 4), laptop portable computers, desktop computers, and the like.
The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.
The information access control method provided by the embodiment of the present application is executed by a server, and accordingly, an information access control device is provided in the server.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. Any number of terminal devices, networks and servers may be provided according to implementation needs, and the terminal devices 101, 102 and 103 in this embodiment may specifically correspond to an application system in actual production.
Referring to fig. 2, fig. 2 shows a method for monitoring traffic of application data according to an embodiment of the present invention, which is detailed as follows:
s201: and acquiring the flow data of each application in a preset period in real time based on an IP data flow information output protocol IPFIX as basic performance data.
Among them, IPFIX is called IP Flow Information Export, which is a standard protocol published by IETF for Flow Information measurement in a network.
In this embodiment, IPFIX protocol is used, which only occupies 1/250 of actual throughput, and compared with conventional packet capture, the processing performance and efficiency are improved.
Optionally, the base performance data includes one or more of an IP address, an access port, a network protocol, throughput, access duration, network latency, network jitter, and layer 2 to layer 7 protocol.
S202: and analyzing the basic performance data to obtain the load information of the node server corresponding to the application as the current load information.
S203: and acquiring historical load information corresponding to each application, and inputting the historical load information and the current load information into a reinforcement learning model to perform peak value prediction to obtain a predicted peak value.
In a specific optional embodiment, in step S203, performing peak prediction on the historical load information and the current load information in a reinforcement learning model, and obtaining a predicted peak includes:
inputting historical load information into a prediction unit in a reinforcement learning model for prediction to obtain a first load peak value prediction value, wherein the prediction unit is a neural network model;
acquiring a real peak value corresponding to the historical load information as a second load peak value;
inputting the first load peak value and the second load peak value into a reinforcement learning model, calculating to obtain a difference value of the first load peak value and the second load peak value based on a loss function of the reinforcement learning model, and taking the difference value as a reward function;
updating the reinforcement learning model based on the gradient ascending method and the reward function;
and inputting the current load information and the reward function into the updated reinforcement learning model for decision making to obtain a decision making result, and taking a peak value in the decision making result as a prediction peak value.
Wherein, the Loss function Loss is as formula (1):
LOSS(y,y')＝-(ylog(y')+(1-y)log(1-y')) (1)
wherein y is the first load peak value and y' is the second load peak value.
The gradient ascent method is to find the local maximum of the function, and in the application, the optimization formula for optimizing the reinforcement learning model is argmax (loss).
Wherein, the decision unit is a perceptron model.
The decision unit model formed by the five-layer perceptron is explained, and the concrete steps are as follows:
the decision unit model formed by the five layers of perceptrons comprises an input layer, a hidden layer and an output layer, wherein the unit of each layer is connected with all units of the adjacent layer, the units of the same layer are not connected, load information is input into the input layer, the decision is output by the output layer after passing through the hidden layer, and the specific calculation process is as follows:
input Xi＝(xi1,xi2,...,xin)，XiAnd a feature vector representing the load information of the ith preset period of the historical load information, where n may be set according to an actual application scenario, for example, n is 4, and there are 4 feature vectors in the same preset period.
According to the formula h1＝sigmoid(W1Xi+b1) Outputting the result of the first hidden layer, where h1Output result, W, representing the first hidden layer1Coefficient matrix representing the first hidden layer, b1Indicating the bias of the first hidden layer.
H is to be1Input to a second hidden layer by the formula h2＝sigmoid(W2h1+b2) Calculating the result of the second hidden layer, wherein h2Representing the output result of the second hidden layer, W2Coefficient matrix representing the second hidden layer, b2Indicating the bias of the second hidden layer.
H is to be2And a prediction result y 'obtained from the long-term and short-term memory model of the last passing prediction unit'i-1Splicing and inputting the three hidden layers, and calculating by a formula (2) to obtain an output result (a)1,a2) Wherein a is1，a2Respectively, the probability of 0 and 1, W3Coefficient matrix representing a third hidden layer, b3Indicating the biasing of the third hidden layer.
(a1,a2)＝soft(sigmoid(W3(h2⊕y'i-1)+b3)) (2)
According to the output result (a) of the output1,a2) And sampling to obtain a final decision result.
S204: and judging whether the predicted peak value exceeds a preset early warning threshold value, and if the predicted peak value exceeds the preset early warning peak value, executing a preset flow control measure.
In a specific optional implementation manner, in step S204, if the predicted peak value exceeds the preset warning peak value, the executing the preset flow management and control measure includes:
if the predicted peak value exceeds the preset early warning peak value and is in a first early warning range, sending early warning information;
and if the predicted peak value exceeds the preset early warning peak value and is within the second early warning range, executing corresponding emergency early warning measures and limiting the access of the data flow.
Specifically, when the value of the predicted peak value exceeding the preset early warning peak value is in a first early warning range, early warning information is sent to the monitoring end, so that the monitoring end can check the reason in time, and when the value of the predicted peak value exceeding the preset early warning peak value is in a second early warning range, preset emergency measures and early warning measures are taken, and the safety of data access is ensured.
Emergency early warning measures include, but are not limited to: refusing the access request of the IP address, deleting the server node corresponding to the application, and the like, and setting corresponding early warning measures for each application in advance.
It should be understood that the first warning range and the second warning range are both numerical ranges, and the second warning range is larger in value than the first warning range, for example, the first warning range is (0, 10), and the second warning range is (10, 100).
In a specific optional implementation manner, after determining whether the predicted peak value exceeds the preset early warning threshold value, and if the predicted peak value exceeds the preset early warning peak value, executing a preset traffic control measure, the method for monitoring traffic of application data further includes:
constructing a visual chart according to the flow data and the predicted peak value which are collected in real time, wherein the visual chart comprises at least one of a trend chart, a frequency chart, a proportion chart or a data table;
and displaying the visual chart in the visual interface.
In the embodiment of the year, based on an IP data flow information output protocol IPFIX, flow data of each application in a preset period are collected in real time and used as basic performance data, the basic performance data are analyzed to obtain load information of a node server corresponding to the application and used as current load information, historical load information corresponding to the application is obtained for each application, the historical load information and the current load information are input into a reinforcement learning model to predict a peak value, a predicted peak value is obtained, whether the predicted peak value exceeds a preset early warning threshold value or not is judged, if the predicted peak value exceeds the preset early warning peak value, preset flow management and control measures are executed, real-time monitoring of data flow is achieved, and safety of data flow response and timeliness of early warning are improved.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
Fig. 3 is a schematic block diagram of an application data traffic monitoring device corresponding to the above-described application data traffic monitoring method according to one-to-one embodiment. As shown in fig. 3, the flow monitoring apparatus for application data includes a data acquisition module 31, a data analysis module 32, a peak prediction module 33, and an early warning module 34. The functional modules are explained in detail as follows:
the data acquisition module 31 is configured to acquire flow data of each application in a preset period in real time based on an IP data flow information output protocol IPFIX, and use the flow data as basic performance data;
the data analysis module 32 is configured to analyze the basic performance data to obtain load information of the node server corresponding to the application, which is used as current load information;
the peak value prediction module 33 is configured to obtain, for each application, historical load information corresponding to the application, and input the historical load information and current load information into the reinforcement learning model to perform peak value prediction, so as to obtain a predicted peak value;
and the early warning module 34 is configured to determine whether the predicted peak value exceeds a preset early warning threshold, and if the predicted peak value exceeds the preset early warning peak value, execute a preset traffic control measure.
Optionally, the peak prediction module 33 comprises:
the first peak prediction unit is used for inputting the historical load information into a prediction unit in the reinforcement learning model for prediction to obtain a first load peak prediction value, wherein the prediction unit is a neural network model;
the second peak value acquisition unit is used for taking a real peak value corresponding to the historical load information as a second load peak value;
the reward function calculation unit is used for inputting the first load peak value and the second load peak value into the reinforcement learning model, calculating a difference value between the first load peak value and the second load peak value based on a loss function of the reinforcement learning model, and taking the difference value as a reward function;
the model updating unit is used for updating the reinforcement learning model based on the gradient ascending method and the reward function;
and the prediction peak value determining unit is used for inputting the current load information and the reward function into the updated reinforcement learning model for decision making to obtain a decision making result, and taking a peak value in the decision making result as a prediction peak value.
Optionally, the early warning module 34 includes:
the first early warning unit is used for sending early warning information if the predicted peak value exceeds a preset early warning peak value and is in a first early warning range;
and the second early warning unit is used for executing corresponding emergency early warning measures and limiting the access of the data traffic if the predicted peak value exceeds the preset early warning peak value and is within a second early warning range.
Optionally, the traffic monitoring apparatus for application data further includes:
the visual chart construction module is used for constructing a visual chart according to the flow data and the prediction peak value which are collected in real time, wherein the visual chart comprises at least one of a trend chart, a frequency chart, a proportion chart or a data table;
and the visual chart display module is used for displaying the visual chart in the visual interface.
For specific limitations of the application data flow monitoring device, reference may be made to the above limitations of the application data flow monitoring method, which are not described herein again. All or part of the modules in the flow monitoring device for application data can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In order to solve the technical problem, an embodiment of the present application further provides a computer device. Referring to fig. 4, fig. 4 is a block diagram of a basic structure of a computer device according to the present embodiment.
The computer device 4 comprises a memory 41, a processor 42, a network interface 43 communicatively connected to each other via a system bus. It is noted that only the computer device 4 having the components connection memory 41, processor 42, network interface 43 is shown, but it is understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead. As will be understood by those skilled in the art, the computer device is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and the hardware includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.
The computer device can be a desktop computer, a notebook, a palm computer, a cloud server and other computing devices. The computer equipment can carry out man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch panel or voice control equipment and the like.
The memory 41 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or D interface display memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the computer device 4. Of course, the memory 41 may also include both internal and external storage devices of the computer device 4. In this embodiment, the memory 41 is generally used for storing an operating system installed in the computer device 4 and various types of application software, such as program codes for controlling electronic files. Further, the memory 41 may also be used to temporarily store various types of data that have been output or are to be output.
The processor 42 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is configured to execute the program code stored in the memory 41 or process data, for example, execute the program code for data access.
The network interface 43 may comprise a wireless network interface or a wired network interface, and the network interface 43 is generally used for establishing communication connection between the computer device 4 and other electronic devices.
The present application provides another embodiment, which is to provide a computer-readable storage medium storing a data access program, which is executable by at least one processor to cause the at least one processor to perform the steps of the method for monitoring traffic of application data as described above.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present application.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present application.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present application.
It is to be understood that the above-described embodiments are merely illustrative of some, but not restrictive, of the broad invention, and that the appended drawings illustrate preferred embodiments of the invention and do not limit the scope of the invention. This application is capable of embodiments in many different forms and is provided for the purpose of enabling a thorough understanding of the disclosure of the application. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to one skilled in the art that the present application may be practiced without modification or with equivalents of some of the features described in the foregoing embodiments. All equivalent structures made by using the contents of the specification and the drawings of the present application are directly or indirectly applied to other related technical fields and are within the protection scope of the present application.
Claims (10)
1. A traffic monitoring method of application data is characterized in that the traffic monitoring method of the application data comprises the following steps:
acquiring flow data of each application in a preset period in real time based on an IP data flow information output protocol IPFIX as basic performance data;
analyzing the basic performance data to obtain load information of a node server corresponding to the application, wherein the load information is used as current load information;
acquiring historical load information corresponding to each application, and inputting the historical load information and the current load information into a reinforcement learning model for peak value prediction to obtain a predicted peak value;
and judging whether the predicted peak value exceeds a preset early warning threshold value, and if the predicted peak value exceeds the preset early warning peak value, executing a preset flow control measure.
2. The method of application data traffic monitoring according to claim 1, wherein the basic performance data comprises one or more of IP address, access port, network protocol, throughput, access duration, network latency, network jitter, and layer 2 to layer 7 protocol.
3. The method for monitoring the flow of the application data according to claim 1, wherein the step of predicting the peak value in a reinforcement learning model according to the historical load information and the current load information to obtain the predicted peak value comprises:
inputting the historical load information into a prediction unit in the reinforcement learning model for prediction to obtain a first load peak prediction value, wherein the prediction unit is a neural network model;
acquiring a real peak value corresponding to the historical load information as a second load peak value;
inputting the first load peak value and the second load peak value into the reinforcement learning model, calculating a difference value between the first load peak value and the second load peak value based on a loss function of the reinforcement learning model, and taking the difference value as the reward function;
updating the reinforcement learning model based on a gradient ascent method and the reward function;
and inputting the current load information and the reward function into the updated reinforcement learning model for decision making to obtain a decision making result, and taking a peak value in the decision making result as the prediction peak value.
4. The method for monitoring the flow of the application data according to claim 1, wherein if the predicted peak value exceeds a preset early warning peak value, performing a preset flow control measure comprises:
if the predicted peak value exceeds a preset early warning peak value and is in a first early warning range, sending early warning information;
and if the predicted peak value exceeds the preset early warning peak value and is within a second early warning range, executing corresponding emergency early warning measures and limiting the access of data traffic.
5. The method for monitoring traffic of application data according to any one of claims 1 to 4, wherein after the determining whether the predicted peak value exceeds a preset warning threshold value and executing a preset traffic control measure if the predicted peak value exceeds a preset warning peak value, the method for monitoring traffic of application data further comprises:
constructing a visual chart according to the flow data collected in real time and the predicted peak value, wherein the visual chart comprises at least one of a trend chart, a frequency chart, a proportion chart or a data table;
displaying the visualization chart in a visualization interface.
6. An application data flow monitoring device, comprising:
the data acquisition module is used for acquiring flow data of each application in a preset period in real time based on an IP data flow information output protocol IPFIX as basic performance data;
the data analysis module is used for analyzing the basic performance data to obtain load information of a node server corresponding to the application as current load information;
the peak value prediction module is used for acquiring historical load information corresponding to each application, and inputting the historical load information and the current load information into a reinforcement learning model for peak value prediction to obtain a predicted peak value;
and the early warning module is used for judging whether the predicted peak value exceeds a preset early warning threshold value or not, and if the predicted peak value exceeds the preset early warning peak value, executing a preset flow control measure.
7. The traffic monitoring apparatus of application data according to claim 6, wherein the peak prediction module comprises:
the first peak prediction unit is used for inputting the historical load information into a prediction unit in the reinforcement learning model for prediction to obtain a first load peak prediction value, wherein the prediction unit is a neural network model;
the second peak value acquisition unit is used for taking a real peak value corresponding to the historical load information as a second load peak value;
a reward function calculation unit, configured to input the first load peak value and the second load peak value into the reinforcement learning model, calculate a difference value between the first load peak value and the second load peak value based on a loss function of the reinforcement learning model, and use the difference value as the reward function;
a model updating unit for updating the reinforcement learning model based on a gradient ascent method and the reward function;
and the prediction peak value determining unit is used for inputting the current load information and the reward function into the updated reinforcement learning model for decision making to obtain a decision making result, and taking a peak value in the decision making result as the prediction peak value.
8. The traffic monitoring device of application data according to claim 6, wherein the early warning module comprises:
the first early warning unit is used for sending early warning information if the predicted peak value exceeds a preset early warning peak value and is in a first early warning range;
and the second early warning unit is used for executing corresponding emergency early warning measures and limiting the access of data traffic if the predicted peak value exceeds the preset early warning peak value and is within a second early warning range.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method of traffic monitoring of application data according to any of claims 1 to 5 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out a method for traffic monitoring of application data according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111108992.6A CN113986564A (en) | 2021-09-22 | 2021-09-22 | Application data flow monitoring method and device, computer equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111108992.6A CN113986564A (en) | 2021-09-22 | 2021-09-22 | Application data flow monitoring method and device, computer equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113986564A true CN113986564A (en) | 2022-01-28 |
Family
ID=79736261
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111108992.6A Pending CN113986564A (en) | 2021-09-22 | 2021-09-22 | Application data flow monitoring method and device, computer equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113986564A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115002042A (en) * | 2022-05-25 | 2022-09-02 | 中国平安财产保险股份有限公司 | Special line flow management and control method and device based on machine learning and computer equipment |
| CN115658701A (en) * | 2022-12-27 | 2023-01-31 | 北京仁科互动网络技术有限公司 | Database flow control method, device, equipment and storage medium |
| CN117041072A (en) * | 2023-06-25 | 2023-11-10 | 兴容(上海)信息技术股份有限公司 | Multi-mode data management system and method based on artificial intelligence |
-
2021
- 2021-09-22 CN CN202111108992.6A patent/CN113986564A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115002042A (en) * | 2022-05-25 | 2022-09-02 | 中国平安财产保险股份有限公司 | Special line flow management and control method and device based on machine learning and computer equipment |
| CN115002042B (en) * | 2022-05-25 | 2023-06-20 | 中国平安财产保险股份有限公司 | Special line flow control method and device based on machine learning and computer equipment |
| CN115658701A (en) * | 2022-12-27 | 2023-01-31 | 北京仁科互动网络技术有限公司 | Database flow control method, device, equipment and storage medium |
| CN115658701B (en) * | 2022-12-27 | 2023-03-14 | 北京仁科互动网络技术有限公司 | Database flow control method, device, equipment and storage medium |
| CN117041072A (en) * | 2023-06-25 | 2023-11-10 | 兴容(上海)信息技术股份有限公司 | Multi-mode data management system and method based on artificial intelligence |
| CN117041072B (en) * | 2023-06-25 | 2024-02-13 | 兴容(上海)信息技术股份有限公司 | Multi-mode data management system and method based on artificial intelligence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113986564A (en) | Application data flow monitoring method and device, computer equipment and medium | |
| CN110826071A (en) | Software vulnerability risk prediction method, device, equipment and storage medium | |
| CN113326991B (en) | Automatic authorization method, device, computer equipment and storage medium | |
| CN112491602A (en) | Behavior data monitoring method and device, computer equipment and medium | |
| CN112631910A (en) | Front-end testing method and device, computer equipment and storage medium | |
| CN112463422A (en) | Internet of things fault operation and maintenance method and device, computer equipment and storage medium | |
| CN114095567A (en) | Data access request processing method and device, computer equipment and medium | |
| CN114564294A (en) | Intelligent service arranging method and device, computer equipment and storage medium | |
| CN114070583A (en) | Information access control method, information access control device, computer equipment and medium | |
| CN112631911A (en) | Automatic testing method and device, computer equipment and storage medium | |
| CN112395351A (en) | Visual identification group complaint risk method, device, computer equipment and medium | |
| CN110807050B (en) | Performance analysis method, device, computer equipment and storage medium | |
| CN113886721B (en) | Personalized interest point recommendation method and device, computer equipment and storage medium | |
| CN115936895A (en) | Risk assessment method, device and equipment based on artificial intelligence and storage medium | |
| CN110795554A (en) | Target information analysis method, device, equipment and storage medium | |
| CN110598093A (en) | Business rule management method and device | |
| WO2023066258A1 (en) | Data processing method and apparatus for private data, computer device and medium | |
| CN115545753A (en) | Partner prediction method based on Bayesian algorithm and related equipment | |
| CN115328764A (en) | Test code optimization method based on automatic test and related equipment thereof | |
| CN114995914A (en) | Picture data processing method and device, computer equipment and storage medium | |
| CN114281817A (en) | Data cleaning method and device, computer equipment and storage medium | |
| CN114549053A (en) | Data analysis method and device, computer equipment and storage medium | |
| CN114265835A (en) | Data analysis method and device based on graph mining and related equipment | |
| CN114615177A (en) | Load detection method and device of cloud platform, electronic equipment and storage medium | |
| CN114385694A (en) | Data processing method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |