CN117749486A - Method and system for carrying out abnormal alarm based on correlation of running state data - Google Patents
Method and system for carrying out abnormal alarm based on correlation of running state data Download PDFInfo
- Publication number
- CN117749486A CN117749486A CN202311763692.0A CN202311763692A CN117749486A CN 117749486 A CN117749486 A CN 117749486A CN 202311763692 A CN202311763692 A CN 202311763692A CN 117749486 A CN117749486 A CN 117749486A
- Authority
- CN
- China
- Prior art keywords
- sequence
- cpu
- flow
- data
- correlation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 41
- 230000005856 abnormality Effects 0.000 claims abstract description 50
- 238000004422 calculation algorithm Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 238000007621 cluster analysis Methods 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 abstract description 10
- 238000005457 optimization Methods 0.000 abstract description 9
- 238000012098 association analyses Methods 0.000 abstract description 8
- 238000007405 data analysis Methods 0.000 abstract description 4
- 238000004891 communication Methods 0.000 description 16
- 230000000875 corresponding effect Effects 0.000 description 16
- 238000004364 calculation method Methods 0.000 description 8
- 230000004044 response Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000012887 quadratic function Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a method and a system for carrying out abnormality warning based on correlation of running state data, which relate to the technical field of network security and comprise the steps of obtaining a flow sequence and a CPU occupation sequence of network equipment in a time window, determining the degree of correlation of the CPU based on a historical flow sequence and a historical resource occupation sequence when the CPU real-time occupancy rate of the equipment is higher than a threshold value, and determining whether the equipment is abnormal or not based on the correlation of the fitted flow sequence and the CPU occupation sequence. According to the invention, through real-time monitoring of CPU occupancy rate and flow rate and combination of historical data analysis, the system has stronger real-time performance and predictability, and potential problems can be found earlier. By utilizing the historical flow and the resource occupation sequence, intelligent association analysis can be performed, the relation between the CPU occupation rate abnormality and the flow can be accurately judged, and the judgment accuracy is improved. By fitting the association of the flow sequence and the CPU occupation sequence, the performance optimization can be more comprehensively carried out, the network resources are effectively managed, and the overall efficiency is improved.
Description
Technical Field
The invention relates to the field of network security, in particular to a method and a system for carrying out abnormal alarm based on correlation of running state data.
Background
As the dependence of society on networks increases, such as enterprise operations, online services, remote offices, etc., it becomes critical to ensure high availability and stability of network devices, many critical services in society rely on the proper operation of network devices. By monitoring CPU occupancy and traffic in real-time, the system can detect potential failures and resource bottlenecks in time to prevent possible network outages or service unavailability. And managing the resource usage of the network device is helpful for resource optimization and planning, and ensures that the network resources are effectively utilized. This has important roles in controlling operating costs, improving efficiency and promoting sustainable development. Where abnormal traffic and CPU occupancy patterns may be signs of network attacks or malicious activity.
In the current data center security operation and maintenance process, various security products are often required to be purchased, and the security products monitor various network traffic inside and outside the data center, and the traffic is usually used for traffic analysis in one data center. However, these traffic analyses tend to be focused on units and are not associated with a specific traffic type, resulting in some useful information not being found and utilized, i.e. data resources being wasted.
Disclosure of Invention
The invention is provided in view of the existing problems in the system and the abnormality warning based on the correlation of the running state data.
The problem to be solved by the invention is therefore how to discover potential risks in a data center by mining the data.
In order to solve the technical problems, the invention provides the following technical scheme:
in a first aspect, an embodiment of the present invention provides a method for performing an anomaly alarm based on correlation of running state data, which includes acquiring a traffic sequence and a CPU occupation sequence of a network device in a time window. And when the CPU real-time occupancy rate of the equipment is higher than a threshold value, determining the CPU association degree based on the historical traffic sequence and the historical resource occupancy sequence. And determining whether the device has an abnormality based on the association of the fitted traffic sequence and the CPU occupation sequence.
As a preferable scheme of the abnormality warning method based on the correlation of the running state data, the invention comprises the following steps: the flow sequence is a flow data sequence which is collected by firewall equipment and flows through a firewall, and the flow sequence comprises a protocol type and frame information. And the CPU occupation sequence is an occupation sequence of a server in the data center. Determining a characteristic sequence of the flow based on the flow sequence, determining a characteristic sequence of a CPU of the device based on a CPU occupation sequence of the device, aligning the characteristic sequence of the flow with the characteristic sequence of the CPU based on a time stamp to obtain historical state information distinguished according to the time stamp, and performing cluster analysis on the historical state information distinguished according to the time stamp by using a K-Means method to obtain a plurality of clusters, and determining a correlation model of the flow sequence and the CPU occupation sequence by fitting; and taking the flow sequence as the input of the association model to acquire the expected CPU occupation sequence.
As a preferable scheme of the abnormality warning method based on the correlation of the running state data, the invention comprises the following steps: if the expected CPU occupation sequence exceeds a threshold value, early warning is carried out; the early warning further comprises determining through fitting a correlation model of the flow sequence and the CPU occupation sequence, specifically taking the CPU occupation sequence as input of the correlation model, and acquiring an expected flow sequence; and when the expected flow sequence exceeds a threshold value, early warning is carried out.
As a preferable scheme of the abnormality warning method based on the correlation of the running state data, the invention comprises the following steps: the fitting is a quadratic fitting of the number of threads and the occupation of CPU resources, and specifically comprises the following steps: the input parameters of the secondary fitting of the thread number comprise the number of protocols corresponding to the flow sequence, and the output is the actual process or handle number. The input parameters of the secondary fitting of the CPU resource occupation comprise the number of frames corresponding to the flow sequence, and the number of frames is output as an actual CPU occupation value.
As a preferable scheme of the abnormality warning method based on the correlation of the running state data, the invention comprises the following steps: and classifying each history state by the early warning, acquiring the classification of the abnormal data and the normal data, further acquiring corresponding cluster members, and judging whether the abnormality occurs or not based on fitting of the cluster members. And calculating CPU load values of all clusters obtained by clustering, performing fitting by using the cluster with the lowest CPU load value mean value, determining whether abnormality exists or not by using the association degree under the normal state, and performing fitting by using the cluster corresponding to the latest history state information.
As a preferable scheme of the abnormality warning method based on the correlation of the running state data, the invention comprises the following steps: the abnormal judgment is carried out, the K-Means algorithm and the Euclidean distance are used for calculating the interval between adjacent points for clustering, the clustering center is set as m, and normal and abnormal data are obtained; fitting the abnormal data points to obtain a binomial, wherein if the input of the binomial is data quantity and the output is CPU occupancy rate, if the CPU value under the current flow is calculated by using the binomial, the abnormality is considered to occur when the deviation exceeds x%; if the data amount under the current CPU occupancy is calculated by using binomial terms, the exception is considered to occur when the deviation exceeds y%.
As a preferable scheme of the abnormality warning method based on the correlation of the running state data, the invention comprises the following steps: the abnormal judgment is carried out, the K-Means algorithm and the Euclidean distance are used for calculating the interval between adjacent points for clustering, the clustering center is set as n, and normal and abnormal data are obtained; the average CPU occupancy rate of each cluster is calculated respectively, the cluster with the minimum CPU occupancy rate is obtained for fitting, a binomial is obtained, the input of the binomial is data quantity, the output is the CPU occupancy rate, and at the moment, if the binomial is used for calculating the CPU occupancy rate under the current flow, the abnormality is considered to occur when the deviation exceeds p%. If the CPU occupancy rate at the current flow is calculated using binomial terms, an abnormality is considered to occur when the deviation exceeds q%.
In a second aspect, an embodiment of the present invention provides an abnormality alert system based on an operational status data correlation, including: and the state sequence acquisition module is used for acquiring the flow sequence and the CPU occupation sequence of the network equipment in a time window. And the association degree determining module is used for determining the association degree of the CPU based on the historical flow sequence and the historical resource occupation sequence when the real-time CPU occupancy rate of the equipment is higher than the threshold value. And the abnormality alarming module is used for determining whether the equipment is abnormal or not based on the association of the fitted flow sequence and the CPU occupation sequence.
In a third aspect, embodiments of the present invention provide a computer apparatus comprising a memory and a processor, the memory storing a computer program, wherein: and the processor realizes any step of the method for carrying out abnormal alarm based on the correlation of the running state data when executing the computer program.
In a fourth aspect, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein: the computer program when executed by the processor implements any step of the above-described method for performing an anomaly alert based on operational state data correlation.
The method has the advantages that by monitoring the CPU occupancy rate and the flow rate and combining with historical data analysis, the system has stronger instantaneity and predictability, potential problems can be found earlier, intelligent association analysis can be performed by utilizing the historical flow rate and the resource occupancy sequence, the relationship between the abnormality of the CPU occupancy rate and the flow rate can be judged more accurately, and the judgment accuracy is improved. Based on the conclusion obtained by the association analysis, automatic response can be realized, measures can be quickly taken to cope with abnormality, manual intervention time is reduced, the association of a flow sequence and a CPU occupation sequence is fitted, performance optimization is more comprehensively carried out, network resources are effectively managed, and overall efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. Wherein:
FIG. 1 is a flow chart of the present invention.
Detailed Description
So that the manner in which the above recited objects, features and advantages of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to the embodiments, some of which are illustrated in the appended drawings. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways other than those described herein, and persons skilled in the art will readily appreciate that the present invention is not limited to the specific embodiments disclosed below.
Further, reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic can be included in at least one implementation of the invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.
Example 1
Referring to fig. 1, a first embodiment of the present invention provides a method for performing abnormality alert based on correlation of operation status data, including:
s1: and acquiring a flow sequence and a CPU occupation sequence of the network equipment in a time window.
The sequence of the traffic of the network device in a time window is the sequence of the switch in an event, which may be a network slice or the traffic of the whole data center, including some information except specific data, where the collected information is information such as source, forwarding destination, protocol type and the like of a data packet, and the number of times of occurrence of the corresponding information in a sequence is only processed because of more data types, data packets, forwarding types and protocol types. The CPU occupancy sequence is an occupancy sequence of servers in a data center, which is periodically recorded and archived by journals to a journal server and analyzed.
According to one embodiment of the invention, the traffic sequence is a traffic data sequence collected by the firewall device and flowing through the firewall, the traffic sequence comprises protocol type and frame information, and in this way, a network information abstract in each time window (5 s or 15 s) can be provided, and the analysis cost per frame is high. The CPU occupation sequence comprises a process name, a CPU duty ratio corresponding to the process name, a CPU event and a period. And determining the related information of the process by acquiring the occupation of the process and the PID information. Each server can collect and analyze the physical machine and the virtual machine independently, and each container is regarded as a separate application when analyzing the physical machine.
Determining a characteristic sequence of the historical flow based on the historical flow sequence, determining a characteristic sequence of a CPU of the device based on a CPU occupation sequence of the device, aligning the characteristic sequence of the historical flow with the characteristic sequence of the CPU based on a time stamp to obtain historical state information distinguished according to the time stamp, and performing cluster analysis on the historical state information distinguished according to the time stamp by using a K-Means method to obtain a plurality of clusters, wherein the clusters are used for determining a correlation model of the flow sequence and the CPU occupation sequence through fitting; and taking the flow sequence as the input of the association model to acquire the expected CPU occupation sequence.
S2: and when the CPU real-time occupancy rate of the equipment is higher than a threshold value, determining the CPU association degree based on the historical flow sequence and the historical resource occupancy sequence.
And when the expected CPU occupation sequence exceeds a threshold value, early warning is carried out.
Wherein the threshold is typically set to 50% -60%, less than 50% has no monitoring value, as abnormal connections typically result in rapid elevation of dynamic threshold set at one point resource occupancy, which is typically not of good use, as new task deployments are involved. Taking a 50% threshold as an example, if it is below the threshold, no treatment is performed; when the threshold value is equal to the threshold value, processing is started, and an alarm program can be initialized at the moment; at this time, a waiting time period, for example, an occupancy rate exceeding 10s continuously, may be set, and when the tracking is greater than the threshold value, the correlation is analyzed according to the historical sequence and the current sequence.
Or carrying out early warning by fitting an association model for determining the flow sequence and the CPU occupation sequence, taking the CPU occupation sequence as the input of the association model, and obtaining an expected flow sequence; and when the expected flow sequence exceeds a threshold value, early warning is carried out.
The early warning process classifies each history state through an unsupervised algorithm, so that abnormal data and normal data can be classified, normal data and abnormal data can be classified, corresponding cluster members can be obtained, and whether abnormality occurs can be judged by fitting based on the cluster members.
Further, the fitting is a secondary fitting, the fitted input parameters comprise the length corresponding to the flow sequence, and the fitted output is the load value of the CPU. At this time, if a network device such as a firewall F is provided in the network, the flow data sequence FF flowing through the firewall device in real time may be obtained through the SNMP protocol, and meanwhile, the real-time duty ratio data sequences such as FC and FM of the CPU and the memory of the firewall may be collected by using the SNMP protocol. The larger FF data means that more CPU and memory are required, the larger FC and FM. Therefore, FF and FC, FF and FM are positively correlated.
FF and FC, FF and FM were fitted using the fitting function numpy. The set deg parameter is set to 2, i.e., a quadratic function, is called as follows:
L1[k1,k2,k3]＝numpy.polyfit(FF,FC,2)
obtaining a fitting function of the flow and the CPU through fitting, wherein the fitting function is expressed as:
c1＝PC(f1)c1＝k1*f12+k2*f1+k3
and monitoring the real-time flow fn, the CPU real-time duty ratio cn and the memory real-time duty ratio mn of the equipment through SNMP. If the CPU real-time duty ratio is larger than the early warning value, substituting the real-time flow into a fitting function PC of the flow and the CPU to obtain fitting data c1, and if the deviation between c1 and cn is larger than a fitting deviation value D, warning is carried out.
Fitting is performed by performing quadratic fitting on the number of threads and the occupation of CPU resources, wherein input parameters of the quadratic fitting on the number of threads comprise the number of protocols corresponding to a flow sequence, and the number of protocols is output as the actual number of processes or handles; the input parameters of the secondary fitting of the CPU resource occupation comprise the number of frames corresponding to the flow sequence, and the output is the actual CPU occupation value. The number of threads, processes, or handles are distinguished herein and are associated based on the protocol type and amount of data, respectively.
Further, CPU load values of all clusters obtained by clustering are calculated, the cluster with the lowest CPU load value average value is used for fitting, whether abnormality exists or not is determined through the association degree in the normal state, and the cluster corresponding to the latest history state information is used for fitting. In this way, the association between the abnormal states is fitted, and whether or not there is an abnormality is determined by the degree of association in the abnormal state. The abnormal state corresponding cluster here includes a normal state, thereby making its sensitivity higher.
S3: and determining whether the device has an abnormality based on the association of the fitted traffic sequence and the CPU occupation sequence.
The access switch of the data center acquires a flow sequence (total data quantity) and a CPU occupation sequence (CPU actual occupancy rate) in a time window every 1 s; when the CPU real-time occupancy rate of the equipment is higher than a threshold value by 30%, determining the CPU association degree based on the historical flow sequence and the historical resource occupancy sequence, wherein the method specifically comprises the following steps: acquiring all flow data and CPU occupancy rate data of 15 days before the current time point; aligning the data to obtain data having the following format:
{ timestamp: data traffic: CPU occupancy }
Clustering by using a K-Means algorithm and the intervals of the Euclidean distance calculation adjacent points, wherein the clustering center is set to be 2, and normal and abnormal data are obtained; fitting the abnormal data points to obtain a binomial, wherein the input of the binomial is the data quantity, and the output of the binomial is the CPU occupancy rate; the CPU value at the current flow rate was calculated using binomial expression, and when the deviation exceeded 10%, an abnormality was considered to occur.
The access switch of the data center acquires a flow sequence (total data quantity) and a CPU occupation sequence (CPU actual occupancy rate) in a time window every 1 s; when the CPU real-time occupancy rate of the equipment is higher than a threshold value by 30%, determining the CPU association degree based on the historical flow sequence and the historical resource occupancy sequence, wherein the method specifically comprises the following steps: acquiring all flow data and CPU occupancy rate data of 15 days before the current time point; aligning the data to obtain data having the following format:
{ timestamp: data traffic: CPU occupancy }
Clustering by using a K-Means algorithm and the intervals of the Euclidean distance calculation adjacent points, wherein the clustering center is set to be 2, and normal and abnormal data are obtained; fitting the abnormal data points to obtain a binomial, wherein the input of the binomial is the data quantity, and the output of the binomial is the CPU occupancy rate; the amount of data at the current CPU occupancy is calculated using binomials, and when the deviation exceeds 10%, an abnormality is considered to occur.
The access exchanger of the data center acquires a flow sequence (total data quantity and communication protocol number) and a CPU occupation sequence (CPU actual occupancy rate and process number) in a time window every 10 s; when the CPU real-time occupancy rate of the equipment is higher than a threshold value by 30%, determining the CPU association degree based on the historical flow sequence and the historical resource occupancy sequence, wherein the method specifically comprises the following steps: acquiring all flow data and CPU occupancy rate data of 15 days before the current time point; aligning the data to obtain data having the following format:
{ timestamp: data traffic: number of communication protocols: CPU occupancy rate: number of processes })
Clustering by using a K-Means algorithm and the intervals of the Euclidean distance calculation adjacent points, wherein the clustering center is set to be 6, and normal and abnormal data are obtained; calculating average CPU occupancy rate of each cluster respectively, obtaining a cluster with minimum CPU occupancy rate, fitting, obtaining a binomial input as data quantity, and outputting as CPU occupancy rate; the CPU value at the current flow rate was calculated using binomial expression, and when the deviation exceeded 10%, an abnormality was considered to occur.
The access exchanger of the data center acquires a flow sequence (total data quantity and communication protocol number) and a CPU occupation sequence (CPU actual occupancy rate and process number) in a time window every 10 s; when the CPU real-time occupancy rate of the equipment is higher than a threshold value by 30%, determining the CPU association degree based on the historical flow sequence and the historical resource occupancy sequence, wherein the method specifically comprises the following steps: acquiring all flow data and CPU occupancy rate data of 15 days before the current time point; aligning the data to obtain data having the following format:
{ timestamp: data traffic: number of communication protocols: CPU occupancy rate: number of processes })
Clustering by using a K-Means algorithm and the intervals of the Euclidean distance calculation adjacent points, wherein the clustering center is set to be 6, and normal and abnormal data are obtained; calculating average CPU occupancy rate of each cluster respectively, obtaining a cluster with minimum CPU occupancy rate, fitting, obtaining a binomial, wherein the input of the binomial is the number of communication protocols, and the output is the CPU occupancy rate; the CPU occupancy at the current flow is calculated using binomial terms, and when the deviation exceeds 10%, an abnormality is considered to occur.
The access exchanger of the data center acquires a flow sequence (total data quantity and communication protocol number) and a CPU occupation sequence (CPU actual occupancy rate and process number) in a time window every 10 s; when the CPU real-time occupancy rate of the equipment is higher than a threshold value by 50%, judging whether a cached traffic sequence and a model of the CPU occupancy sequence exist or not, and if not, determining the CPU association degree based on the historical traffic sequence and the historical resource occupancy sequence, wherein the method specifically comprises the following steps: acquiring all flow data and CPU occupancy rate data of 15 days before the current time point; aligning the data to obtain data having the following format:
{ timestamp: data traffic: number of communication protocols: CPU occupancy rate: number of processes })
Clustering by using a K-Means algorithm and the intervals of the Euclidean distance calculation adjacent points, wherein the clustering center is set to be 6, and normal and abnormal data are obtained; calculating average CPU occupancy rate of each cluster respectively, obtaining a cluster with minimum CPU occupancy rate, fitting, obtaining a binomial model, caching the model, inputting the binomial model as the number of communication protocols, and outputting the binomial model as the CPU occupancy rate; the CPU occupancy at the current flow is calculated using binomial terms, and when the deviation exceeds 10%, an abnormality is considered to occur.
The access exchanger of the data center acquires a flow sequence (total data quantity and communication protocol number) and a CPU occupation sequence (CPU actual occupancy rate and process number) in a time window every 10 s; judging whether a cached flow sequence and a model of a CPU occupation sequence exist or not when the CPU real-time occupancy rate of the equipment is higher than a threshold value by 50%; when the CPU association degree is not existed, the CPU association degree is determined based on the historical traffic sequence and the historical resource occupation sequence, and the CPU association degree specifically comprises the following steps: acquiring all flow data and CPU occupancy rate data of 15 days before the current time point; aligning the data to obtain data having the following format:
{ timestamp: data traffic: number of communication protocols: CPU occupancy rate: number of processes })
Clustering by using a K-Means algorithm and the intervals of the Euclidean distance calculation adjacent points, wherein the clustering center is set to be 5, and normal and abnormal data are obtained; respectively calculating the average CPU occupancy rate of each cluster, fitting the cluster with the minimum CPU occupancy rate to obtain two binomial parameters, namely, the input parameters of the quadratic fit of the number of lines comprise the number of protocols corresponding to the flow sequence, and outputting the number of protocols as the actual process or handle number; the input parameters of the secondary fitting of the CPU resource occupation comprise the number of frames corresponding to the flow sequence, and the number of frames is output as an actual CPU occupation value; and the buffer model uses binomial formula to calculate the CPU occupancy rate and thread number under the current flow, and when the deviation exceeds 10%, the abnormality is considered to happen.
The access exchanger of the data center acquires a flow sequence (total data quantity and communication protocol number) and a CPU occupation sequence (CPU actual occupancy rate and process number) in a time window every 10 s; judging whether a cached flow sequence and a model of a CPU occupation sequence exist or not when the CPU real-time occupancy rate of the equipment is higher than a threshold value by 50%; when the CPU association degree is not existed, the CPU association degree is determined based on the historical traffic sequence and the historical resource occupation sequence, and the CPU association degree specifically comprises the following steps: acquiring all flow data and CPU occupancy rate data of 15 days before the current time point; aligning the data to obtain data having the following format:
{ timestamp: data traffic: number of communication protocols: CPU occupancy rate: number of processes })
Clustering by using a K-Means algorithm and the intervals of the Euclidean distance calculation adjacent points, wherein the clustering center is set to be 5, and normal and abnormal data are obtained; respectively calculating the average CPU occupancy rate of each cluster, obtaining the cluster with the minimum CPU occupancy rate, fitting, and obtaining two binomial parameters, namely, input parameters of secondary fitting of the number of lines, including the number of actual processes or handles, and outputting the number of the protocols corresponding to the flow sequence; the input parameters of the secondary fitting of the CPU resource occupation comprise actual CPU occupation values, and the actual CPU occupation values are output as the number of frames corresponding to the flow sequence; and caching the model, calculating a flow sequence under the current flow by using binomial terms, and considering that an abnormality occurs when the deviation exceeds 10%.
According to the invention, by monitoring the CPU occupancy rate and the flow rate and combining with historical data analysis, the system has stronger instantaneity and predictability, and can discover potential problems earlier. By utilizing the historical flow and the resource occupation sequence, the system can perform intelligent association analysis, more accurately judge the relationship between the abnormal CPU occupancy rate and the flow, and improve the judgment accuracy. By fitting the association of the flow sequence and the CPU occupation sequence, the performance optimization can be more comprehensively carried out, the network resources are effectively managed, and the overall efficiency is improved.
In conclusion, the method and the device have stronger instantaneity and predictability by monitoring the CPU occupancy rate and the flow and combining with the historical data analysis, can discover potential problems earlier, can perform intelligent association analysis by utilizing the historical flow and the resource occupancy sequence, judge the relationship between the abnormality of the CPU occupancy rate and the flow more accurately, and improve the judgment accuracy. Based on the conclusion obtained by the association analysis, automatic response can be realized, measures can be rapidly taken to cope with abnormality, manual intervention time is reduced, the association of a flow sequence and a CPU occupation sequence is fitted, performance optimization is comprehensively carried out, network resources are effectively managed, and overall efficiency is improved.
Example 2
On the basis of the first embodiment, the present embodiment further provides an abnormality alert system based on the correlation of the operational status data, including:
the state sequence acquisition module is used for acquiring a flow sequence and a CPU occupation sequence of the network equipment in a time window;
the association degree determining module is used for determining the association degree of the CPU based on the historical flow sequence and the historical resource occupation sequence when the real-time CPU occupancy rate of the equipment is higher than a threshold value;
and the abnormality alarming module is used for determining whether the equipment is abnormal or not based on the association of the fitted flow sequence and the CPU occupation sequence.
The embodiment also provides a computer device, which is suitable for the situation of carrying out an abnormality alarming method based on the correlation of running state data, and comprises a memory and a processor; the memory is used for storing computer executable instructions, and the processor is used for executing the computer executable instructions to realize the method for carrying out abnormality warning based on the correlation of the running state data as proposed by the embodiment.
The computer device may be a terminal comprising a processor, a memory, a communication interface, a display screen and input means connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
The present embodiment also provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the method for implementing abnormality alert based on correlation of operational status data as proposed in the above embodiments.
The storage medium according to the present embodiment belongs to the same inventive concept as the data storage method according to the above embodiment, and technical details not described in detail in the present embodiment can be seen in the above embodiment, and the present embodiment has the same advantageous effects as the above embodiment.
Example 3
On the basis of the first two embodiments, the embodiment provides a method for carrying out abnormal alarm based on the correlation of running state data, and in order to verify the beneficial effects of the invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.
Compared with the traditional network performance monitoring method, the method has stronger real-time performance, intelligent association analysis, automatic response and comprehensive performance optimization, and specific advantages are shown in the table 1:
table 1 comparative table of dominance
Compared with the traditional technology, the invention has more outstanding performance in real-time performance, intelligent association analysis, automatic response and comprehensive performance optimization, emphasizes faster, intelligent and automatic network performance monitoring and management, so as to better adapt to the complex and fast-changing network environment at present and improve the availability and efficiency of the network.
Specific values are used as a comparison and different time spans are used as reference basis, and specific comparison data are shown in table 2:
table 2 comparison table for network performance monitoring technique
It can be seen from Table 2 that the present invention is excellent in both the last 24 hours and the week, and the real-time property is significantly higher than that of the conventional method. The correlation accuracy of the new design is significantly better than the traditional method over all time spans, especially the most significant performance in the last 24 hours.
Second, the last 24 hours and the automated response time within a week are shorter than in the traditional method, indicating that the system is more rapid in coping with anomalies.
The invention has better performance optimization, in particular to the performance optimization efficiency which is higher than that of the traditional method in the time span of one week and one month.
It should be noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention may be modified or substituted without departing from the spirit and scope of the technical solution of the present invention, which is intended to be covered in the scope of the claims of the present invention.
Claims (10)
1. A method for carrying out abnormal alarm based on the correlation of running state data is characterized in that: comprising the steps of (a) a step of,
acquiring a flow sequence and a CPU occupation sequence of network equipment in a time window;
when the CPU real-time occupancy rate of the equipment is higher than a threshold value, determining the CPU association degree based on the historical flow sequence and the historical resource occupancy sequence;
and determining whether the device has an abnormality based on the association of the fitted traffic sequence and the CPU occupation sequence.
2. The abnormality warning method based on the correlation of the operation state data according to claim 1, characterized in that: the flow sequence is a flow data sequence which is collected by firewall equipment and flows through a firewall, and the flow sequence comprises a protocol type and frame information;
the CPU occupation sequence is an occupation sequence of a server in the data center;
determining a characteristic sequence of the flow based on the flow sequence, determining a characteristic sequence of a CPU of the device based on a CPU occupation sequence of the device, aligning the characteristic sequence of the flow with the characteristic sequence of the CPU based on a time stamp to obtain historical state information distinguished according to the time stamp, and performing cluster analysis on the historical state information distinguished according to the time stamp by using a K-Means method to obtain a plurality of clusters, and determining a correlation model of the flow sequence and the CPU occupation sequence by fitting; and taking the flow sequence as the input of the association model to acquire the expected CPU occupation sequence.
3. The abnormality warning method based on the correlation of the operation state data according to claim 2, characterized in that: if the expected CPU occupation sequence exceeds a threshold value, early warning is carried out; the early warning further comprises determining through fitting a correlation model of the flow sequence and the CPU occupation sequence, specifically taking the CPU occupation sequence as input of the correlation model, and acquiring an expected flow sequence; and when the expected flow sequence exceeds a threshold value, early warning is carried out.
4. The method for performing abnormality alert based on correlation of operation state data as claimed in claim 3, wherein: the fitting is a quadratic fit to the number of lines and the occupation of CPU resources, and specifically comprises,
the input parameters of the secondary fitting of the number of threads comprise the number of protocols corresponding to the flow sequence, and the number of the protocols is output as the actual number of processes or handles;
the input parameters of the secondary fitting of the CPU resource occupation comprise the number of frames corresponding to the flow sequence, and the number of frames is output as an actual CPU occupation value.
5. The method for performing anomaly alert based on operational status data correlation as claimed in claim 4, wherein: the early warning classifies each history state, obtains abnormal data and normal data, further obtains corresponding cluster members, and judges whether abnormality occurs based on member fitting of the clusters;
and calculating CPU load values of all clusters obtained by clustering, performing fitting by using the cluster with the lowest CPU load value mean value, determining whether abnormality exists or not by using the association degree under the normal state, and performing fitting by using the cluster corresponding to the latest history state information.
6. The method for performing anomaly alert based on operational status data correlation as claimed in claim 5, wherein: the abnormal judgment is carried out, the K-Means algorithm and the Euclidean distance are used for calculating the interval between adjacent points for clustering, the clustering center is set as m, and normal and abnormal data are obtained; fitting the abnormal data points to obtain a binomial, if the input of the binomial is data quantity, the output is CPU occupancy rate,
if the CPU value under the current flow is calculated by using the binomial method, the CPU value is regarded as abnormal when the deviation exceeds x%;
if the data amount under the current CPU occupancy is calculated by using binomial terms, the exception is considered to occur when the deviation exceeds y%.
7. The method for performing anomaly alert based on operational status data correlation as claimed in claim 6, wherein: the abnormal judgment is carried out, the K-Means algorithm and the Euclidean distance are used for calculating the interval between adjacent points for clustering, the clustering center is set as n, and normal and abnormal data are obtained; calculating average CPU occupancy rate of each cluster, fitting the cluster with the minimum CPU occupancy rate to obtain a binomial input as data volume and output as CPU occupancy rate,
if the CPU occupancy rate under the current flow is calculated by using binomial terms, the CPU occupancy rate is regarded as abnormal when the deviation exceeds p%;
if the CPU occupancy rate at the current flow is calculated using binomial terms, an abnormality is considered to occur when the deviation exceeds q%.
8. An abnormality warning system based on the correlation of operation state data, the abnormality warning method based on the correlation of operation state data according to any one of claims 1 to 7, characterized in that: comprising the steps of (a) a step of,
the state sequence acquisition module is used for acquiring a flow sequence and a CPU occupation sequence of the network equipment in a time window;
the association degree determining module is used for determining the association degree of the CPU based on the historical flow sequence and the historical resource occupation sequence when the real-time CPU occupancy rate of the equipment is higher than a threshold value;
and the abnormality alarming module is used for determining whether the equipment is abnormal or not based on the association of the fitted flow sequence and the CPU occupation sequence.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that: the processor, when executing the computer program, implements the steps of the abnormality alert method according to any one of claims 1 to 7 based on the correlation of the running state data.
10. A computer-readable storage medium having stored thereon a computer program, characterized by: the computer program when executed by a processor implements the steps of the abnormality alert method according to any one of claims 1 to 7 based on the correlation of the operation state data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311763692.0A CN117749486A (en) | 2023-12-20 | 2023-12-20 | Method and system for carrying out abnormal alarm based on correlation of running state data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311763692.0A CN117749486A (en) | 2023-12-20 | 2023-12-20 | Method and system for carrying out abnormal alarm based on correlation of running state data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117749486A true CN117749486A (en) | 2024-03-22 |
Family
ID=90252366
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311763692.0A Pending CN117749486A (en) | 2023-12-20 | 2023-12-20 | Method and system for carrying out abnormal alarm based on correlation of running state data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117749486A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118069895A (en) * | 2024-04-19 | 2024-05-24 | 临沂大学 | Teenager physique big data optimal storage method and system |
-
2023
- 2023-12-20 CN CN202311763692.0A patent/CN117749486A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118069895A (en) * | 2024-04-19 | 2024-05-24 | 临沂大学 | Teenager physique big data optimal storage method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11140056B2 (en) | Flexible and safe monitoring of computers | |
| CN110708204B (en) | Abnormity processing method, system, terminal and medium based on operation and maintenance knowledge base | |
| CN106844161B (en) | Abnormity monitoring and predicting method and system in calculation system with state flow | |
| US10498628B2 (en) | Adaptive metric collection, storage, and alert thresholds | |
| CN1194316C (en) | Remote network monitor method for computer network | |
| WO2021057382A1 (en) | Abnormality detection method and apparatus, terminal, and storage medium | |
| US20160359685A1 (en) | Method and apparatus for computing cell density based rareness for use in anomaly detection | |
| US10346756B2 (en) | Machine discovery and rapid agglomeration of similar states | |
| US20200134421A1 (en) | Assurance of policy based alerting | |
| CN110213125A (en) | Abnormality detection system based on time series data under a kind of cloud environment | |
| CN102929773A (en) | Information collection method and device | |
| CN117749486A (en) | Method and system for carrying out abnormal alarm based on correlation of running state data | |
| WO2016017208A1 (en) | Monitoring system, monitoring device, and inspection device | |
| Zeufack et al. | An unsupervised anomaly detection framework for detecting anomalies in real time through network system’s log files analysis | |
| US20080186876A1 (en) | Method for classifying applications and detecting network abnormality by statistical information of packets and apparatus therefor | |
| WO2022151680A1 (en) | Automata-based internet of things device flow anomaly detection method and apparatus | |
| US10680919B2 (en) | Eliminating bad rankers and dynamically recruiting rankers in a network assurance system | |
| US20160364467A1 (en) | Event notification system with cluster classification | |
| CN104574219A (en) | System and method for monitoring and early warning of operation conditions of power grid service information system | |
| JP2002342182A (en) | Support system for operation management in network system | |
| CN112596975A (en) | Method, system, equipment and storage medium for monitoring network equipment | |
| CN107094086A (en) | A kind of information acquisition method and device | |
| Solmaz et al. | ALACA: A platform for dynamic alarm collection and alert notification in network management systems | |
| US20200042373A1 (en) | Device operation anomaly identification and reporting system | |
| JP2012181744A (en) | Operation monitoring system and operation monitoring method for distributed file system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |