CN112165400A

CN112165400A - System for troubleshooting data network based on network delay

Info

Publication number: CN112165400A
Application number: CN202011026910.9A
Authority: CN
Inventors: 赵来平; 任浩实; 聂力海; 高红运
Original assignee: Tianjin University
Current assignee: Tianjin University
Priority date: 2020-09-25
Filing date: 2020-09-25
Publication date: 2021-01-01

Abstract

The invention discloses a system for troubleshooting a data network based on network delay, which comprises a control module, a monitoring module and a data analysis module, wherein the control module is used for controlling the data network to be in a fault state; the control module is used for outputting detection data information to the monitoring module after detecting the topological structure network; the monitoring module is used for executing operation on the received detection data information to extract the characteristic information of the delay data; the data analysis module performs three-dimensional calculation and training on the characteristic information of the delay data through the delay data model to output network fault data information, and the system can rapidly and accurately solve network faults only by monitoring the delay data measurement of the data center network, so that the network data processing capacity is improved.

Description

System for troubleshooting data network based on network delay

Technical Field

The invention belongs to the technical field of switch fault positioning of a data center network in cloud computing, and particularly relates to a system for troubleshooting data network faults based on network delay.

Background

In modern data center networks, most data center networks consist of thousands of commercial ethernet switches and routers. Therefore, the localization of switch and router failures is an important guarantee that reliable communications and services are provided by data center networks. But identification of failed switches and routers is difficult due to the incompleteness of routing information in conventional networks. In a network topology in a data center, an ospf (open short Path first) protocol is often used as a routing protocol for communication; in the network topology between data centers, bgp (border Gateway protocol) protocol is often used as a routing protocol. In addition, both protocols support the ECMP (equal Cost Multi path) protocol, and the use of the ECMP protocol determines the transmission path of the data packet by calculating the TCP/UDP quintuple, thereby resulting in a more complex network environment. Thus, network failure location is difficult due to the routing path's incompleteness and constant switching of paths over time.

At present, many methods have been proposed for performing fault location of a data center switch or router, such as Prefix, Pingmesh, NetBouncer, etc., but these methods have more or less drawbacks. Prefix attempts to collect log files for each switch to locate the location of the failure. However, this approach causes competition for switch resources and increases additional network load of the data center; although the Pingmesh adopts a non-invasive mode, Pingmesh focuses more on the network in the data center, and the consideration of the network between the data centers is not sufficient. Furthermore, Pingmesh does not give the accuracy of the positioning. NetBounce uses IP-in-IP tunnel technology, and modifies the transmitted data message in the intrusion mode, thereby determining a routing path and finding out a failed switch to realize positioning.

Disclosure of Invention

Compared with the prior art, the invention discovers the influence of the fault of the switch or the router on the network delay, collects the communication delay of the data center in a non-invasive mode, extracts the characteristics of the delay data and positions the fault switch by machine learning. Meanwhile, the system can be operated in an inter-data center network and a data center network, and is a system for troubleshooting the data network based on network delay.

In order to overcome the defects of the prior art, the system for positioning the network fault of the data center based on the network delay utilizes the data delay collected by the host end and designs a special data structure, thereby realizing a machine learning model for positioning the network fault. The system can improve the practicability of the model only by monitoring the delay data measurement of the data center network and predicting the failed switch. The monitoring system of the invention consists of a controller module, an agent module and a data analysis module.

The invention is implemented by adopting the following technical scheme:

a system for troubleshooting data network based on network delay comprises a control module, a monitoring module and a data analysis module; wherein:

the control module is used for outputting detection data information to the monitoring module after detecting the topological structure network;

the monitoring module is used for executing operation on the received detection data information to extract the characteristic information of the delay data;

and the data analysis module performs three-dimensional calculation and training on the characteristic information of the delay data through the delay data model to output network fault data information.

Further, the control module comprises a topology network, a probe plan generator and an output interface;

the monitoring module comprises a data receiving port and an execution service unit consisting of all hosts; all the hosts communicate through an ICMP protocol, and the service unit is deployed on each host of the data center network through a proxy operated by Ping;

the data analysis module comprises a data receiving port, a delay data model, a machine learning model, a storage unit and a data output port; wherein:

the host computer receives the detection plan data information output by the detection plan generator through respective data receiving ports;

carrying out delay data characteristic extraction on the detection plan data information through respective proxy service units;

and transmitting the extracted delay data characteristics to the data analysis module through respective timing data transmission units.

Further, the delay data model constructs a data cube of a three-dimensional data structure for delay data characteristics, and the data cube enables the delay data characteristics to be a space dimension, a characteristic dimension and a time dimension respectively; wherein:

the spatial dimension is the spatial relationship communication data information between the hosts;

the time dimension is the continuous communication between the hosts, selects the delay data in the designated time interval as a group, and takes each time interval as the information of the time dimension;

the characteristic dimension is a designated time interval, delay data in the interval range is used as a designated group, and the average value, the variance, the standard deviation, the two-norm and the number of data packets for sending detection of the group of data are extracted at the same time.

Advantageous effects

Compared with the prior art, the method comprehensively considers the difference of the structures of the network in the data center and the network between the data centers, can position and integrate the faults of the network in the data center and the network between the data centers by adjusting the tags of the delay data, thereby realizing a universal system for positioning the faults of the data center network, and establishes a fault positioning machine learning model based on an extreme random tree algorithm by analyzing and extracting the characteristics of the delay data.

Compared with the prior art, the method comprehensively considers the difference between the network in the data center and the network between the data centers, makes up the defects of the related technology, and can be used for fault positioning of the switch or the router in the data center scene.

Drawings

FIG. 1 is a system architecture diagram of the present invention.

Fig. 2 is a flow chart of the operation of the extreme random tree algorithm.

FIG. 3 is a diagram showing the accuracy of four machine learning algorithms under the intra-data center topology and the inter-data center topology.

FIG. 4 is a diagram of the prediction accuracy, recall rate and F-Score curve obtained by model training of the extreme random tree algorithm under different time windows with 80% training data and size under different topological scenarios.

The specific implementation mode is as follows:

the techniques and methods of the present invention are described in detail below with reference to examples and figures, which are provided to illustrate the components of the present invention and are not intended to limit the scope of the invention.

1. System architecture design

As shown in fig. 1, which is a schematic diagram of a system architecture of the present invention, the present invention provides a system for troubleshooting a data network based on network delay, where the control module is configured to output detection data information to the monitoring module after detecting a topology network; the monitoring module is used for executing operation on the received detection data information to extract the characteristic information of the delay data;

and the data analysis module performs three-dimensional calculation and training on the characteristic information of the delay data through the delay data model to output network fault data information. The control module is mainly used for generating a detection plan and configuring various parameters, and then controlling and distributing the detection plan to agents on various servers through interfaces. After each agent receives the own detection plan, the communication operation is started and continuously operated. And storing the generated data locally, processing the data, and uploading the data to a data analysis system at a designated time by a local timer. And the data analysis system integrates the delay data uploaded by each agent in the period into a delay data cube, inputs the delay data cube into the machine learning model for classification and outputs a result.

Wherein:

1. control module

The controller module will generate a probe plan for each host. The probe plan is generated based on the topology of the current network. Since the data center network topology is generally stable, the probe plan does not change often. The probe plan is distributed by the controller to the various hosts, thereby specifying the current host to communicate with hosts within the network topology. Meanwhile, the detection plan includes some configuration files, such as the time for sending the message and the time window. The method comprises the following specific steps:

1) and generating a detection plan according to the data center topology. The probe plans are generated according to the corresponding data center topology. The probe plan is mainly used for determining host information used between communications, and the core idea is that the communication between hosts covers most routing paths as much as possible. Typically, a brute force algorithm is used by default to generate a probing plan, e.g., S (S-1) probes are performed knowing the number S of servers in the current topology. The administrator can adjust the detection plan and the detection times according to the specific topological situation.

2) And configuring information. The probe plan contains default configuration information of the system, such as the frequency of sending probe messages and the time window for controlling the delay data packets.

2. Monitoring module

The monitoring module comprises a first host and a second host, data transmission is carried out between the first host and the second host through an ICMP protocol, and the first host and the second host respectively comprise a data receiving port, an agent service unit and a timing data transmission unit; wherein:

each host runs an agent. The agent will first receive a probe plan from the controller. Then, the Ping operation based on the ICMP is applied in the proxy, and when two hosts are set to communicate, one host waits for the other host to return the result of the Ping operation, instead of discarding the communication message. The agent generated delay data is first stored locally. In the proxy, a timer is also maintained internally for periodically submitting processed data. Therefore, the processed data can be uploaded to a data analysis system for analysis in time. In addition, the agent may also receive a preset time window and a probing frequency from the probing plan. The time window may determine a series of delay data collected over a specified time interval, and the probing frequency may determine the rate at which data messages are sent. The method comprises the following specific steps:

1) design and development of Ping operation. In the present system, delay data may be used to characterize the state of the data center network, but different delays may be generated due to different types of applications, such as delay sensitive applications versus CPU intensive applications. Therefore, the system actively performs Ping operation through the service unit, and is deployed on each host of the data center network. And performing Ping operation, and storing the generated delay data locally and processing the delay data. In addition, the conventional Ping operation selects to discard the data packet when the time is out, but in the system, the Ping operation between the hosts does not discard the data packet, and the host waits until another host responds and then sends the next message. If the waiting time exceeds the specified time window, the operation is terminated, the delay is recorded as INF, and the next group of probing operations is executed.

2) Feature extraction of the delayed data. Delay data in a specified time interval is integrated according to parameters of a time window in a detection plan, and the delay data are set to be the same array. Since the time-series delay sequence is a series of fluctuating values, the data within the group is typically not independent of each other but rather dependent on each other. Therefore, four statistical values of the delay data divided into the same group, i.e., a mean, a variance, a standard deviation, and a two-norm, are calculated to reflect the difference between the delay data within a specific time interval. After the characteristics of each set of delay data are obtained, the delay data are sorted according to destination within a specific time window of each host. This step helps to better understand the delay data. After extracting these features for a particular time interval, the information content of the feature space is greater than the information content in the original delay data, which helps to improve the accuracy of fault location. Meanwhile, in order to better reflect the delay information, the system also calculates the number of Ping operations in the time period to be used as the information representing the flow. In addition, the system records the IP addresses of the target server and the source server, and the IP addresses are used for representing the space information of communication between the hosts. In addition, in order to align the delay data between different hosts in the time dimension, it is also necessary to record the time stamp of the first delay data in the group of data as the information of the time dimension of the group of data.

3) And (6) uploading the data. After the delay data are stored and processed, the uploading operation is triggered by a timer, and the processed delay data stored in the local area are uploaded to a data analysis module for fault location operation.

3. Data analysis module

And the data analysis module performs three-dimensional calculation and training on the characteristic information of the delay data through the delay data model to output network fault data information. The data analysis module comprises a data receiving port, a delay data model, a machine learning model, a storage unit and a data output port; wherein:

the delay data generated by the agent is uploaded to the data analysis module. The data are sequenced according to time to form a data cube recognized by machine learning input, and finally, a machine learning model is used for positioning the abnormal switchboard. The method comprises the following specific steps:

1) and constructing a data cube. The data cube is a three-dimensional data structure composed of delay data. In the data cube, these three dimensions are the spatial dimension, the feature dimension, and the temporal dimension, respectively. Since communication is performed between different servers, the spatial relationship between the servers serves as the spatial dimension information of the data cube, and represents the server pair performing communication. Because of the continuous communication between the servers, we select the delay data within a specified time interval as a group and take each time interval as information in the time dimension. And finally, the information of the characteristic dimension. According to the appointed time interval, the delay data in the interval range is used as an appointed group, and the average value, the variance, the standard deviation, the two norms and the number of the data packets for sending detection of the group are extracted at the same time. With these fine-grained records, the fault can be accurately located. Finally, the data in the same time interval at the same moment can be aggregated together to form a three-dimensional data cube.

2) And constructing a machine learning model. And solving the problem of data center network fault location by using a multi-classification machine learning model. And developing a multi-classification machine learning model based on an extreme random tree algorithm. The work flow of the extreme random tree algorithm is to analyze the input three-dimensional delay data cube and classify it according to various decision trees. Then, the extreme random tree algorithm records the classification results and the frequency of occurrence of the labels after each classification, and sorts the classification results according to the occurrence times of the labels. The algorithm will then examine and determine the most frequently occurring tag result. Finally, it checks the tag of this selected result and marks the instance as normal or pinpointing the switch that encountered the failure and outputs the tag. Because the generated data is positioned quickly and efficiently when the data center network fault is positioned, a computational random tree machine learning model based on stream processing is designed, and the positioning efficiency can be improved.

The invention is applied to the computer:

1. the invention has the following parameter configuration:

1) the current data center network topology is first entered, as well as the number of servers.

2) According to the topology and the number of servers, a default algorithm is used for calculating the detection times: f ═ S (S-1).

3) And configuring the size of a time window for data integration. The default time window is 2000 ms.

4) And configuring the detection frequency for controlling the rate of sending the data message.

2. Controller distribution probe plan

After the configuration information is written into the controller, the controller generates a corresponding probe plan for each server deploying the agent. And distributes the probe plan to the agent on the corresponding server through the network interface. Meanwhile, the detection plan can be modified and adjusted on the controller.

3. Collecting latency data

Delay data is generated and processed.

1) After receiving the probe plan, each agent starts communication and generates delay data. While holding the delay data locally.

2) Grouping delay data generated by all the hosts for communication according to the time interval of the time window parameter, and calculating statistical values such as variance, standard deviation, mean, two-norm and communication times of the group of data. At the same time, after processing, information of the pair of communicating hosts and which time interval they currently belong to are added. The information is stored locally as a data item.

3) And uploading all locally stored processed delay data to a data analysis system by a timer.

4. Training machine learning model

1) Tagged delay data is obtained. The agent of the data center is allowed to operate normally first, and then the delayed data generated during this period of time is marked as normal data. And then, closing the appointed switch, enabling the agents to communicate with each other, and marking the data in the time period as the closed switch number. In this way, all tagged data can be obtained.

2) The tagged data is processed into the format of a deferred data cube and input into the four machine learning algorithm models of the design. As shown in fig. 2, the accuracy of the four machine learning algorithms in the time window of 2000ms is compared between the inter-data center network topology and the intra-data center network topology. From the analysis of accuracy, recall and stability points of view, we finally choose the algorithm using the extreme random tree algorithm as the machine learning model.

3) In addition, as shown in fig. 3, the present invention compares the effect of different time windows on accuracy under different topologies. From the analysis of the accuracy, recall rate and stability, the 2000ms time window has better performance.

5. Data analysis module analyzes data

1) And the delay data sent by the timer to the data analysis module are arranged according to the sequence of time intervals and are gathered into a delay data cube.

2) And inputting the delay data cube into a designed machine learning algorithm model, and outputting a corresponding prediction result. The present invention is not limited to the above-described embodiments.

Claims

1. A system for troubleshooting data network based on network delay is characterized by comprising a control module, a monitoring module and a data analysis module; wherein:

2. The system for troubleshooting data network failures based on network delays of claim 1 wherein said control module includes a topology network, a probe plan generator and an output interface;

3. The system for troubleshooting network delay based on data network of claim 1 or 2 wherein said delayed data model constructs a data cube of a three-dimensional data structure for delayed data features, said data cube characterizing delayed data features as a spatial dimension, a feature dimension, and a time dimension, respectively; wherein:

the spatial dimension is the spatial relationship communication data information among all the hosts;

the time dimension is the continuous communication among all the hosts, delay data in a specified time interval is selected as a group, and each time interval is used as the information of the time dimension;