CN110535716B - Service stability monitoring method and system for converged media - Google Patents
Service stability monitoring method and system for converged media Download PDFInfo
- Publication number
- CN110535716B CN110535716B CN201910667901.9A CN201910667901A CN110535716B CN 110535716 B CN110535716 B CN 110535716B CN 201910667901 A CN201910667901 A CN 201910667901A CN 110535716 B CN110535716 B CN 110535716B
- Authority
- CN
- China
- Prior art keywords
- real
- time
- data
- characteristic
- dimension
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24147—Distances to closest patterns, e.g. nearest neighbour classification
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/02—Capturing of monitoring data
- H04L43/026—Capturing of monitoring data using flow identification
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0852—Delays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0852—Delays
- H04L43/0864—Round trip delays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/12—Network monitoring probes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/16—Threshold monitoring
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0681—Configuration of triggering conditions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0823—Errors, e.g. transmission errors
- H04L43/0829—Packet loss
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
- H04L43/0894—Packet rate
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Environmental & Geological Engineering (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a method and a system for monitoring service stability of a converged medium, which relate to the technical field of communication and comprise the following steps: preprocessing the real-time flow data; extracting feature dimensions of the real-time flow data to obtain a real-time feature data group; comparing the real-time delay in the real-time characteristic data group with a preset time delay threshold value: if the real-time delay is not smaller than the delay threshold, analyzing according to a pre-generated feature dimension model to obtain key feature data which has the largest influence on the delay in the real-time feature data, and generating an alarm event; and if the real-time delay is smaller than the delay threshold, analyzing the real-time characteristic data, judging that a safety event exists according to the analysis result, analyzing the safety event according to a pre-generated characteristic dimension model to obtain the potential risk of the safety event on the service stability, and generating an early warning event according to the potential risk. The invention can quickly discover and handle the unstable factors of the service.
Description
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a method and a system for monitoring service stability of a converged media.
Background
The integrated media is a novel media which makes full use of media carriers, integrates different media with common points and complementarity such as broadcasting, television, newspaper and the like comprehensively in the aspects of manpower, content, propaganda and the like, and realizes 'resource integration, content integration, propaganda integration and benefit integration'. The convergence medium is an industry with particularly high requirements on service stability.
In the prior art, monitoring of service stability mainly depends on real-time flow monitoring, that is, real-time flow data is acquired in a bypass mode in core exchange or aggregation exchange, and service operation conditions are judged according to time delay, flow rate and the like of real-time flow data analysis, but the real-time flow data can only reflect changes of bottom-layer factors such as time delay and the like of unstable services, so that specific problems cannot be confirmed; meanwhile, only the problem points which cause the unstable service continuously can be monitored and checked, and the factors which cause the unstable service transiently cannot be accurately positioned; because the allowed influence factors only include time delay, flow rate, packet loss and the like, and the fusion consideration of safety factors is lacked, the root cause of unstable service cannot be quickly found, and the influence degree caused by the safety factors cannot be predicted; in addition, because no behavior model sediment is formed after the current problems are examined through real-time flow data analysis, the problems of unstable service need to be analyzed again every time, and the quick matching of historical experience and the autonomous positioning of the problems are lacked.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a method for monitoring the service stability of a converged medium, which comprises the steps of presetting a flow acquisition probe in a network link of the converged medium, and setting a cloud platform remotely connected with the flow acquisition probe, wherein the cloud platform analyzes real-time flow data in the network link acquired by the flow acquisition probe so as to monitor the service stability of the converged medium;
the service stability monitoring method specifically comprises the following steps:
step S1, the flow acquisition probe carries out data preprocessing on the acquired real-time flow data to obtain and output a flow data log file;
the flow data log file comprises the real-time flow data with a preset data format;
step S2, the cloud platform performs feature extraction on the real-time traffic data in the traffic data log file according to a preset rule to obtain a real-time feature data group corresponding to the real-time traffic data;
the real-time characteristic data group comprises a flow dimension characteristic, a service dimension characteristic and a safety dimension characteristic which correspond to the real-time flow data;
step S3, the cloud platform extracts the real-time delay in the traffic dimensional feature from the real-time feature data set, and compares the real-time delay with a preset delay threshold:
if the real-time delay is not less than the delay threshold, turning to step S4;
if the real-time delay is smaller than the delay threshold, turning to step S5;
step S4, the cloud platform analyzes the real-time feature data set according to a pre-generated feature dimension model to obtain key feature data causing the real-time delay in the real-time feature data set, generates and outputs a corresponding alarm event according to the key feature, and then exits;
the key feature data comprises the business dimension features, and/or the security dimension features;
step S5, the cloud platform queries the real-time feature data set, and determines whether a security event exists in the security dimension data according to a query result:
if yes, go to step S6;
if not, exiting;
and step S6, the cloud platform analyzes the security event according to the characteristic dimension model to obtain the potential risk of the security event to the service stability, and generates and outputs a corresponding early warning event according to the potential risk.
Preferably, a switch is further disposed in the network link of the converged media, and a mirror image port of the switch is connected to the traffic collection probe;
then the step S1 specifically includes:
step S11, the switch sends the real-time traffic data in the network link to the traffic collection probe through the mirror image port by the switch mirror image drainage;
and step S12, the flow acquisition probe processes the real-time flow data into the preset data format to generate and output a flow data log file.
Preferably, in step S2, the flow dimension feature includes a real-time delay, and the method for extracting the real-time delay specifically includes:
step S211, the cloud platform extracts the real-time traffic data to obtain TCP protocol traffic;
step S212, the cloud platform records the time stamp of each data packet of each connection of the TCP protocol flow;
the time stamp comprises a data request time stamp and a data return time stamp;
step S213, the cloud platform calculates a time difference between the data request timestamp and the data return timestamp to obtain the real-time delay corresponding to each connection;
step S214, the cloud platform takes the real-time delay as the flow dimension characteristic of the real-time flow data and stores the flow dimension characteristic.
Preferably, in step S2, the method for extracting the security dimension feature specifically includes:
step S221, the cloud platform matches the real-time flow data with a preset safety product rule base according to the real-time flow data:
if the matching is successful, generating and outputting a corresponding safety event, and then turning to the step S6;
if the matching is not successful, turning to step S222;
step S222, the cloud platform extracts a corresponding characteristic value from the real-time traffic data and compares the characteristic value with a preset safety threshold:
if the characteristic value is not smaller than the safety threshold value, generating a corresponding safety event, and storing the safety event as the safety dimension characteristic;
and if the characteristic value is smaller than the safety threshold value, generating a normal behavior record, and storing the normal behavior record as the safety dimension characteristic.
Preferably, the security threshold comprises a threshold of the number of packets of the ddos attack, and/or a threshold of the number of sessions.
Preferably, the generation method of the feature dimension model is a K-neighborhood classification algorithm.
Preferably, the method for generating the feature dimension model specifically includes:
step A1, the cloud platform acquires a plurality of historical flow data, and performs feature extraction on each historical flow data to obtain a corresponding historical flow feature group;
step a2, the cloud platform analyzes each historical traffic feature set:
if the traffic dimension characteristic which represents that the service is unstable exists in the historical traffic characteristic group, the cloud platform processes the historical traffic characteristic group to obtain the key characteristic data which cause that the service is unstable;
if the safety dimension characteristics potentially influencing the service stability exist in the historical flow characteristic group, the cloud platform processes the historical flow characteristic group to obtain the potential risk of the safety dimension characteristics on the service stability;
step A3, the cloud platform saves each historical flow characteristic group and the analysis result of the step A2 to generate a sample library;
step A4, the cloud platform trains the sample library to obtain the feature dimension model.
Preferably, the method further includes a process of updating the feature dimension model, specifically including:
and storing the real-time feature data set and the analysis result of the cloud platform on the real-time feature data set according to the feature dimension model into the sample library, and updating the feature dimension model according to the updated sample library.
A system for monitoring the service stability of converged media applies any one of the above methods for monitoring the service stability of converged media, which specifically comprises:
the flow acquisition probe is used for carrying out data preprocessing on the acquired real-time flow data to obtain and output a flow data log file;
the flow data log file comprises the real-time flow data with a preset data format;
the cloud platform is connected the flow acquisition probe, the cloud platform specifically includes:
the characteristic extraction module is used for extracting the characteristics of the real-time flow data in the flow data log file according to a preset rule to obtain a real-time characteristic data group corresponding to the real-time flow data;
the real-time characteristic data group comprises a flow dimension characteristic, a service dimension characteristic and a safety dimension characteristic which correspond to the real-time flow data;
a comparison module connected with the feature extraction module and used for extracting the real-time delay in the flow dimension feature from the real-time feature data group, generating and outputting a corresponding first comparison result when the real-time delay is not less than a preset time delay threshold value, and
when the real-time delay is smaller than the delay threshold, generating and outputting a corresponding second comparison result;
the first processing module is connected with the comparison module and used for analyzing the real-time characteristic data set according to the first comparison result and a pre-generated characteristic dimension model to obtain key characteristic data causing the real-time delay in the real-time characteristic data set, and generating and outputting a corresponding alarm event according to the key characteristic;
the key feature data comprises the business dimension features, and/or the security dimension features;
the query module is connected with the comparison module and used for querying in the real-time feature data group according to the second comparison result, and generating and outputting a corresponding query result when a security event exists in the security dimension data;
and the second processing module is connected with the query module and used for analyzing the safety event according to the query result and the characteristic dimension model to obtain the potential risk of the safety event on the service stability and generating a corresponding early warning event according to the potential risk.
The technical scheme has the following advantages or beneficial effects: the characteristic dimension model can be quickly found and disposed when the service is unstable, a large amount of manual analysis is not needed, and meanwhile, the configuration prevention can be carried out on the network equipment and the safety equipment when the service is not unstable, so that the environmental requirements of different customers are met.
Drawings
Fig. 1 is a schematic flow chart illustrating a method for monitoring service stability of a converged media in a preferred embodiment of the present invention;
FIG. 2 is a flow chart illustrating data preprocessing for a flow acquisition probe according to a preferred embodiment of the present invention;
FIG. 3 is a flowchart illustrating a method for extracting real-time delay according to a preferred embodiment of the present invention;
FIG. 4 is a flow chart illustrating a method for extracting security dimension features according to a preferred embodiment of the present invention;
FIG. 5 is a diagram illustrating a K-neighbor classification algorithm according to a preferred embodiment of the present invention;
FIG. 6 is a flow chart illustrating a method for generating a feature dimension model according to a preferred embodiment of the present invention;
fig. 7 is a schematic structural diagram of a system for monitoring service stability of converged media according to a preferred embodiment of the present invention.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments. The present invention is not limited to the embodiment, and other embodiments may be included in the scope of the present invention as long as the gist of the present invention is satisfied.
In a preferred embodiment of the present invention, based on the above problems in the prior art, a method for monitoring service stability of a converged media is provided, in which a traffic acquisition probe is preset in a network link of the converged media, and a cloud platform remotely connected to the traffic acquisition probe is provided, and the cloud platform analyzes real-time traffic data in the network link acquired by the traffic acquisition probe to monitor service stability of the converged media;
as shown in fig. 1, the method for monitoring service stability specifically includes the following steps:
step S1, the flow acquisition probe carries out data preprocessing on the acquired real-time flow data to obtain and output a flow data log file;
the flow data log file comprises real-time flow data with a preset data format;
step S2, the cloud platform performs feature extraction on the real-time traffic data in the traffic data log file according to a preset rule to obtain a real-time feature data group corresponding to the real-time traffic data;
the real-time characteristic data group comprises a flow dimension characteristic, a service dimension characteristic and a safety dimension characteristic which correspond to the real-time flow data;
step S3, the cloud platform extracts the real-time delay in the traffic dimension feature from the real-time feature data set, and compares the real-time delay with a preset delay threshold:
if the real-time delay is not less than the delay threshold, go to step S4;
if the real-time delay is smaller than the delay threshold, turning to step S5;
step S4, the cloud platform analyzes the real-time characteristic data group according to the pre-generated characteristic dimension model to obtain key characteristic data causing real-time delay influence in the real-time characteristic data group, generates and outputs a corresponding alarm event according to the key characteristic, and then quits;
the key feature data comprises business dimension features and/or safety dimension features;
step S5, the cloud platform queries in the real-time characteristic data set, and judges whether a security event exists in the security dimension data according to the query result:
if yes, go to step S6;
if not, exiting;
and step S6, the cloud platform analyzes the security events according to the characteristic dimension model to obtain the potential risks of the security events to the service stability, and generates and outputs corresponding early warning events according to the potential risks.
Specifically, in this embodiment, the cloud platform is a data processing center and is mainly responsible for processing data log files acquired by the flow acquisition probe, and the cloud platform may be an upper computer or a cloud server. Performing multi-dimensional feature extraction on the real-time flow data to respectively obtain flow dimension features, service dimension features and safety dimension features and form a real-time feature data group; the flow dimension characteristics reflect the stability of the current service, and for the converged media, the time delay data is the most obvious characteristic for judging the stability of the service of the converged media. And if the comparison result shows that the current service is in an unstable state, sending the real-time feature data set into a pre-generated feature dimension model for analysis to obtain key feature data causing current service instability. The key characteristic data may be service dimension characteristics, such as unstable service caused by too high access frequency or service burst; the key feature data may also be a security dimension feature, for example, after the key feature data is analyzed and obtained, a corresponding alarm event is generated according to the key feature data and output, so that a worker can perform corresponding processing on the current problem of unstable service according to the alarm event.
Further, if the comparison result indicates that the current service is in a stable state, the security dimension characteristics of the real-time traffic data are further analyzed to determine whether a potential risk affecting the current service stable state exists. The potential risk is taken as a judgment basis according to whether a security event exists in the security dimension characteristics, and if the security event does not exist in the security dimension characteristics, the potential risk influencing the current service stable state does not exist in the network; if a security event exists in the security dimension, the potential risk of the current service stable state exists in the network, at the moment, the real-time feature data set is sent into a pre-generated feature dimension model for analysis, so that the potential risk of the security event on the service stability is obtained, the subsequent risk of the service is sensed in advance, and then a corresponding early warning event is generated according to the influence degree and the possible consequences, so that the staff can take a specific precaution measure on the subsequent risk of the service. The potential risks include the extent to which the security event affects business stability and the possible consequences that have not yet occurred, but will occur with a high probability, to the extent that the impact does not yet occur.
The technical scheme of the invention integrates the flow quality dimension, the application behavior dimension and the safety dimension, constructs an integral continuous service stability monitoring system, continuously models and evaluates the characteristics of flow, liquid level, safety and the like which cause service influence, and can configure and prevent network equipment and safety equipment through characteristic modeling when no safety problem occurs, such as making more reasonable flow control and access strategies. When a problem occurs, the problem is quickly judged according to the historical model, a large amount of manual analysis is not needed, and the problem is quickly found and quickly treated. Specific key technical support, specific technology and special technical support are not required, and mining modeling is established on the existing data. After the occurrence of a business stability problem or the occurrence of potential risk features is monitored, the deep analysis of the feature dimension model is automatically utilized, the root cause causing business influence is identified, and the transmission and marginal effect of the business influence are combed. And the deep analysis result of the characteristic dimension model can be labeled for the real-time characteristic data set, and the labeled characteristic data set is brought into a sample library, continuously self-learned and evolved and automatically attached to different client environments. The characteristic dimension model is a three-dimensional integrated model which is continuously constructed for flow, service and safety based on characteristic analysis of historical flow samples, the historical flow is used for extracting characteristic dimension information, samples are classified in a manual labeling or self-learning mode, and therefore a model base is built and is used for judging which type of characteristics affect service abnormity most, and therefore problems are located quickly.
In this embodiment, the traffic dimension characteristics include characteristic data such as real-time delay, flow rate, packet loss, and the like; the service dimension characteristics comprise service access success rate, service access client IP, access application, access frequency and the like; the security dimension features include security risk behaviors initiated to the business and the host, or initiated security risk behaviors. The time delay threshold is set by the user on the page independently and is regarded as a time delay threshold with abnormal service as a judged baseline index. The statistical process of the real-time delay comprises the following steps: and (3) carrying out flow extraction and restoration through the mirror image drainage of the switch, recording the timestamp of each data packet of each connection for the TCP flow, and calculating the time difference by using the timestamps of the requested and returned data packets according to the matching property of the id of the ack in each data packet to obtain the real-time delay.
In a preferred embodiment of the present invention, a switch is further disposed in the network link of the converged media, and a mirror image port of the switch is connected to the traffic collection probe;
as shown in fig. 2, step S1 specifically includes:
step S11, the switch sends the real-time traffic data in the network link to the traffic collection probe through the mirror port by the mirror drainage of the switch;
in step S12, the flow collection probe processes the real-time flow data into a preset data format to generate and output a flow data log file.
In a preferred embodiment of the present invention, in step S2, the flow dimension feature includes a real-time delay, as shown in fig. 3, the method for extracting the real-time delay specifically includes:
step S211, the cloud platform extracts the real-time traffic data to obtain TCP protocol traffic;
step S212, the cloud platform records the time stamp of each data packet of each connection of the TCP protocol flow;
the time stamp comprises a data request time stamp and a data return time stamp;
step S213, the cloud platform calculates the time difference between the data request timestamp and the data return timestamp to obtain the real-time delay corresponding to each connection;
step S214, the cloud platform takes the real-time delay as a flow dimension characteristic of the real-time flow data and stores the flow dimension characteristic.
In a preferred embodiment of the present invention, in step S2, as shown in fig. 4, the method for extracting the security dimension feature specifically includes:
step S221, the cloud platform matches the real-time flow data with a preset safety product rule base:
if the matching is successful, generating and outputting a corresponding safety event, and then turning to the step S6;
if the matching is not successful, turning to step S222;
step S222, the cloud platform extracts a corresponding characteristic value from the real-time traffic data and compares the characteristic value with a preset safety threshold:
if the characteristic value is not less than the safety threshold value, generating a corresponding safety event, and storing the safety event as a safety dimension characteristic;
and if the characteristic value is smaller than the safety threshold value, generating a normal behavior record, and storing the normal behavior record as a safety dimension characteristic.
In a preferred embodiment of the present invention, the security threshold comprises a threshold for the number of packets, and/or a threshold for the number of sessions, of a ddos attack.
In a preferred embodiment of the present invention, the feature dimension model is generated by a K-neighborhood classification algorithm.
Specifically, in this embodiment, the generation method of the feature dimension model includes, but is not limited to, a K-neighborhood classification algorithm. As a preferred embodiment of the present invention, the feature dimension model is further described in detail according to the K-neighborhood classification algorithm as follows:
the K-neighborhood classification algorithm classifies by measuring the distance between different feature values. Comparing the features in the real-time feature data group obtained by extracting the features of the real-time flow data with the corresponding features in the feature dimension model, finding the first K data which are most similar to the features in the feature dimension model, wherein the category corresponding to the test data is the category with the largest occurrence frequency in the K data, and the value of K is adjusted according to the actual data distribution condition. Wherein the similarity is expressed by distance, can be calculated by using calculation formulas including but not limited to Euclidean distance, Manhattan distance and the like,
take the Euclidean distance formula as an example:
the symbols in the drawings are for illustration purposes and not intended to be exhaustive, and are used to explain the examples.
As shown in fig. 5, the red triangle in the figure represents class a of the traffic burst, and the blue square represents class B of the security event, which are not described herein.
Firstly, analyzing real-time flow data in real time to obtain a flow dimension characteristic (circle), and when K is 3, taking 3 points (such as an inner circle in fig. 5) which are most similar to (closest to) the flow dimension characteristic, wherein an A-type proportion 2/3 in the 3 points, the circle belongs to the A-type, which indicates that the service is unstable due to service burst;
when k is 5, the 5 points (such as the outer circle of fig. 5) most similar to (closest to) the traffic dimension feature are taken, wherein B-class accounts for 3/5 in the 5 points, and the circle belongs to B-class, which indicates that the traffic is unstable due to the security event.
In a preferred embodiment of the present invention, as shown in fig. 6, the method for generating the feature dimension model specifically includes:
a1, the cloud platform acquires a plurality of historical flow data, and performs feature extraction on each historical flow data to obtain a corresponding historical flow feature group;
step A2, the cloud platform analyzes each historical traffic feature group:
if the historical flow feature group has flow dimension features representing unstable service, the cloud platform processes the flow dimension features according to the historical flow feature group to obtain key feature data causing unstable service;
if the safety dimension characteristics potentially influencing the service stability exist in the historical flow characteristic group, the cloud platform processes the safety dimension characteristics according to the historical flow characteristic group to obtain the potential risk of the safety dimension characteristics on the service stability;
step A3, the cloud platform stores each historical flow characteristic group and the analysis result of the step A2 to generate a sample library;
and A4, training the cloud platform according to the sample library to obtain a feature dimension model.
In a preferred embodiment of the present invention, the method further includes a process of updating the feature dimension model, specifically including:
and storing the real-time characteristic data group and the analysis result of the cloud platform on the real-time characteristic data group according to the characteristic dimension model into a sample library, and updating the characteristic dimension model according to the updated sample library.
A system for monitoring service stability of converged media, which applies any one of the above methods for monitoring service stability of converged media, as shown in fig. 7, specifically includes:
the flow acquisition probe 1 is used for carrying out data preprocessing on the acquired real-time flow data to obtain and output a flow data log file;
the flow data log file comprises real-time flow data with a preset data format;
the feature extraction module 21 is configured to perform feature extraction on the real-time traffic data in the traffic data log file according to a preset rule to obtain a real-time feature data group corresponding to the real-time traffic data;
the real-time characteristic data group comprises a flow dimension characteristic, a service dimension characteristic and a safety dimension characteristic which correspond to real-time flow data;
a comparison module 22 connected to the feature extraction module 21 and configured to extract a real-time delay in the flow dimension feature from the real-time feature data set, generate and output a corresponding first comparison result when the real-time delay is not less than a preset delay threshold, and
generating and outputting a corresponding second comparison result when the real-time delay is smaller than the delay threshold;
the first processing module 23 is connected to the comparison module 22, and is configured to analyze the real-time feature data set according to the first comparison result and a pre-generated feature dimension model, obtain key feature data causing real-time delay in the real-time feature data set, generate and output a corresponding alarm event according to the key feature;
the key feature data comprises business dimension features and/or safety dimension features;
the query module 24 is connected to the comparison module 22, and is configured to query the real-time feature data set according to the second comparison result, and generate and output a corresponding query result when a security event exists in the security dimension data;
and the second processing module 25 is connected to the query module 24, and is configured to analyze the security event according to the query result and the feature dimension model, obtain a potential risk of the security event on the service stability, and generate a corresponding early warning event according to the potential risk.
While the invention has been described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention.
Claims (8)
1. A service stability monitoring method for a converged medium is characterized in that a flow acquisition probe is preset in a network link of the converged medium, and a cloud platform remotely connected with the flow acquisition probe is arranged, wherein the cloud platform analyzes real-time flow data in the network link acquired by the flow acquisition probe so as to monitor the service stability of the converged medium;
the service stability monitoring method specifically comprises the following steps:
step S1, the flow acquisition probe carries out data preprocessing on the acquired real-time flow data to obtain and output a flow data log file;
the flow data log file comprises the real-time flow data with a preset data format;
step S2, the cloud platform performs feature extraction on the real-time traffic data in the traffic data log file according to a preset rule to obtain a real-time feature data group corresponding to the real-time traffic data;
the real-time characteristic data group comprises a flow dimension characteristic, a service dimension characteristic and a safety dimension characteristic which correspond to the real-time flow data;
step S3, the cloud platform extracts the real-time delay in the traffic dimensional feature from the real-time feature data set, and compares the real-time delay with a preset delay threshold:
if the real-time delay is not less than the delay threshold, turning to step S4;
if the real-time delay is smaller than the delay threshold, turning to step S5;
step S4, the cloud platform analyzes the real-time feature data set according to a pre-generated feature dimension model to obtain key feature data causing the real-time delay in the real-time feature data set, generates and outputs a corresponding alarm event according to the key feature data, and then exits;
the key feature data comprises the business dimension features, and/or the security dimension features;
step S5, the cloud platform queries the real-time feature data set, and determines whether a security event exists in the security dimension data according to a query result:
if yes, go to step S6;
if not, exiting;
step S6, the cloud platform analyzes the security event according to the characteristic dimension model to obtain the potential risk of the security event to the service stability, and generates and outputs a corresponding early warning event according to the potential risk;
the method for generating the feature dimension model specifically comprises the following steps:
step A1, the cloud platform acquires a plurality of historical flow data, and performs feature extraction on each historical flow data to obtain a corresponding historical flow feature group;
step a2, the cloud platform analyzes each historical traffic feature set:
if the traffic dimension characteristic which represents that the service is unstable exists in the historical traffic characteristic group, the cloud platform processes the historical traffic characteristic group to obtain the key characteristic data which causes that the service is unstable, and then the step A3 is performed;
if the safety dimension characteristics potentially influencing the service stability exist in the historical traffic characteristic group, the cloud platform processes the historical traffic characteristic group to obtain the potential risk of the safety dimension characteristics on the service stability, and then the process goes to step A3;
step A3, the cloud platform saves each historical flow characteristic group and the analysis result of the step A2 to generate a sample library;
and A4, the cloud platform trains according to the sample library to obtain the feature dimension model.
2. The method for monitoring the service stability of the converged media according to claim 1, wherein a switch is further disposed in the network link of the converged media, and a mirror image port of the switch is connected to the traffic collection probe;
the step S1 specifically includes:
step S11, the switch sends the real-time traffic data in the network link to the traffic collection probe through the mirror image port by the switch mirror image drainage;
and step S12, the flow acquisition probe processes the real-time flow data into the preset data format to generate and output a flow data log file.
3. The method for monitoring service stability of converged media according to claim 1, wherein in the step S2, the traffic dimension feature includes a real-time delay, and the method for extracting the real-time delay specifically includes:
step S211, the cloud platform extracts the real-time traffic data to obtain TCP protocol traffic;
step S212, the cloud platform records the time stamp of each data packet of each connection of the TCP protocol flow;
the time stamp comprises a data request time stamp and a data return time stamp;
step S213, the cloud platform calculates a time difference between the data request timestamp and the data return timestamp to obtain the real-time delay corresponding to each connection;
step S214, the cloud platform takes the real-time delay as the flow dimension characteristic of the real-time flow data and stores the flow dimension characteristic.
4. The method for monitoring service stability of converged media according to claim 1, wherein in the step S2, the method for extracting the security dimension feature specifically includes:
step S221, the cloud platform matches the real-time flow data with a preset safety product rule base according to the real-time flow data:
if the matching is successful, generating and outputting a corresponding safety event, and then turning to the step S6;
if the matching is not successful, turning to step S222;
step S222, the cloud platform extracts a corresponding characteristic value from the real-time traffic data, and compares the characteristic value with a preset safety threshold:
if the characteristic value is not smaller than the safety threshold value, generating a corresponding safety event, and storing the safety event as the safety dimension characteristic;
and if the characteristic value is smaller than the safety threshold value, generating a normal behavior record, and storing the normal behavior record as the safety dimension characteristic.
5. The traffic stability monitoring method for converged media, according to claim 4, wherein the security threshold comprises a threshold of the number of packets of a ddos attack, and/or a threshold of the number of sessions.
6. The method for monitoring the service stability of the converged media according to claim 1, wherein the characteristic dimension model is generated by a K-neighborhood classification algorithm.
7. The method for monitoring service stability of converged media according to claim 1, further comprising a process of updating the feature dimension model, specifically comprising:
and storing the real-time feature data set and the analysis result of the cloud platform on the real-time feature data set according to the feature dimension model into the sample library, and updating the feature dimension model according to the updated sample library.
8. A system for monitoring service stability of converged media, applying the method for monitoring service stability of converged media according to any one of claims 1 to 7, specifically comprising:
the flow acquisition probe is used for carrying out data preprocessing on the acquired real-time flow data to obtain and output a flow data log file;
the flow data log file comprises the real-time flow data with a preset data format;
the cloud platform is connected the flow acquisition probe, the cloud platform specifically includes:
the characteristic extraction module is used for extracting the characteristics of the real-time flow data in the flow data log file according to a preset rule to obtain a real-time characteristic data group corresponding to the real-time flow data;
the real-time characteristic data group comprises a flow dimension characteristic, a service dimension characteristic and a safety dimension characteristic which correspond to the real-time flow data;
a comparison module connected with the feature extraction module and used for extracting the real-time delay in the flow dimension feature from the real-time feature data set, generating and outputting a corresponding first comparison result when the real-time delay is not less than a preset delay threshold, and
when the real-time delay is smaller than the delay threshold, generating and outputting a corresponding second comparison result;
the first processing module is connected with the comparison module and used for analyzing the real-time characteristic data set according to the first comparison result and a pre-generated characteristic dimension model to obtain key characteristic data causing the real-time delay in the real-time characteristic data set, and generating and outputting a corresponding alarm event according to the key characteristic;
the key feature data comprises the business dimension features, and/or the security dimension features;
the query module is connected with the comparison module and used for querying in the real-time feature set according to the second comparison result, and generating and outputting a corresponding query result when a security event exists in the security dimension data;
and the second processing module is connected with the query module and used for analyzing the safety event according to the query result and the characteristic dimension model to obtain the potential risk of the safety event on the service stability, and generating a corresponding early warning event according to the potential risk.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910667901.9A CN110535716B (en) | 2019-07-23 | 2019-07-23 | Service stability monitoring method and system for converged media |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910667901.9A CN110535716B (en) | 2019-07-23 | 2019-07-23 | Service stability monitoring method and system for converged media |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110535716A CN110535716A (en) | 2019-12-03 |
| CN110535716B true CN110535716B (en) | 2022-09-30 |
Family
ID=68661898
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910667901.9A Active CN110535716B (en) | 2019-07-23 | 2019-07-23 | Service stability monitoring method and system for converged media |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110535716B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111191636B (en) * | 2020-02-17 | 2023-04-18 | 北京同方凌讯科技有限公司 | Fused media broadcasting consistency detection method based on image color quantity distribution and moment |
| CN112235312B (en) * | 2020-10-22 | 2022-04-26 | 新华三信息安全技术有限公司 | Method and device for determining credibility of security event and electronic equipment |
| CN115865759A (en) * | 2023-02-27 | 2023-03-28 | 科来网络技术股份有限公司 | Network equipment time delay obtaining method and system based on flow mirror protocol |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101977368A (en) * | 2010-10-22 | 2011-02-16 | 中国电信股份有限公司 | Method and system for eliminating traffic of IMS service on content charging gateway |
| CN106789184A (en) * | 2016-12-03 | 2017-05-31 | 浙江广播电视集团 | A kind of method that user independently sets up realization automation operation flow in operation system |
| CN107196910A (en) * | 2017-04-18 | 2017-09-22 | 国网山东省电力公司电力科学研究院 | Threat early warning monitoring system, method and the deployment framework analyzed based on big data |
| CN108777643A (en) * | 2018-06-08 | 2018-11-09 | 武汉思普崚技术有限公司 | A kind of traffic visualization plateform system |
| CN109391800A (en) * | 2018-11-28 | 2019-02-26 | 华中科技大学 | A kind of intelligence community video monitoring method and system based on broadcasting and TV TVOS Intelligent set top box |
| CN109639516A (en) * | 2018-10-17 | 2019-04-16 | 平安科技（深圳）有限公司 | Monitoring method, device, equipment and the storage medium of distributed network system (DNS) |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3042517A4 (en) * | 2013-09-05 | 2017-05-03 | Mitel Mobility Inc. | Converged media packet gateway for a novel lte data and voice core network architecture |
| US9419876B2 (en) * | 2014-03-18 | 2016-08-16 | Airmagnet, Inc. | Methods and apparatus to determine network delay with location independence from retransmission delay and application response time |
-
2019
- 2019-07-23 CN CN201910667901.9A patent/CN110535716B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101977368A (en) * | 2010-10-22 | 2011-02-16 | 中国电信股份有限公司 | Method and system for eliminating traffic of IMS service on content charging gateway |
| CN106789184A (en) * | 2016-12-03 | 2017-05-31 | 浙江广播电视集团 | A kind of method that user independently sets up realization automation operation flow in operation system |
| CN107196910A (en) * | 2017-04-18 | 2017-09-22 | 国网山东省电力公司电力科学研究院 | Threat early warning monitoring system, method and the deployment framework analyzed based on big data |
| CN108777643A (en) * | 2018-06-08 | 2018-11-09 | 武汉思普崚技术有限公司 | A kind of traffic visualization plateform system |
| CN109639516A (en) * | 2018-10-17 | 2019-04-16 | 平安科技（深圳）有限公司 | Monitoring method, device, equipment and the storage medium of distributed network system (DNS) |
| CN109391800A (en) * | 2018-11-28 | 2019-02-26 | 华中科技大学 | A kind of intelligence community video monitoring method and system based on broadcasting and TV TVOS Intelligent set top box |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110535716A (en) | 2019-12-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110535716B (en) | Service stability monitoring method and system for converged media | |
| CN110460594B (en) | Threat information data acquisition processing method, device and storage medium | |
| CN110781930A (en) | User portrait grouping and behavior analysis method and system based on log data of network security equipment | |
| CN112114995A (en) | Process-based terminal anomaly analysis method, device, equipment and storage medium | |
| CN111274218A (en) | Multi-source log data processing method for power information system | |
| CN112365265B (en) | Internet financial intelligent wind control system | |
| CN115134099B (en) | Network attack behavior analysis method and device based on full flow | |
| KR100628329B1 (en) | Generation apparatus and method of detection rules for attack behavior based on information of network session | |
| CN111782484B (en) | Anomaly detection method and device | |
| CN109150869A (en) | A kind of exchanger information acquisition analysis system and method | |
| CN106375295B (en) | Data store monitoring method | |
| CN112671767B (en) | Security event early warning method and device based on alarm data analysis | |
| CN108737193A (en) | A kind of failure prediction method and device | |
| CN113704328A (en) | User behavior big data mining method and system based on artificial intelligence | |
| CN106372171B (en) | Monitor supervision platform real-time data processing method | |
| CN113904829B (en) | Application firewall system based on machine learning | |
| CN115277113A (en) | Power grid network intrusion event detection and identification method based on ensemble learning | |
| CN115378619A (en) | Sensitive data access method, electronic equipment and computer readable storage medium | |
| CN110909380B (en) | Abnormal file access behavior monitoring method and device | |
| CN112291213A (en) | Abnormal flow analysis method and device based on intelligent terminal | |
| CN113750538A (en) | Big data-based hand-game security platform construction method and system | |
| CN116155581A (en) | Network intrusion detection method and device based on graph neural network | |
| CN115174205B (en) | Network space safety real-time monitoring method, system and computer storage medium | |
| CN107911232B (en) | Method and device for determining business operation rule | |
| CN116248393A (en) | Intranet data transmission loophole scanning device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200218 Address after: 200003 No. 298, Weihai Road, Jing'an District, Shanghai Applicant after: SHANGHAI MEDIA &ENTERTAINMENT GROUP Co.,Ltd. Applicant after: SHANGHAI MEDIA TECH CO.,LTD. Address before: 200041 No. 298, Weihai Road, Shanghai, Jingan District Applicant before: SHANGHAI MEDIA &ENTERTAINMENT GROUP Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |