CN114500310A - An accurate method for determining the baseline of multi-dimensional network situation data - Google Patents
An accurate method for determining the baseline of multi-dimensional network situation data Download PDFInfo
- Publication number
- CN114500310A CN114500310A CN202111592628.1A CN202111592628A CN114500310A CN 114500310 A CN114500310 A CN 114500310A CN 202111592628 A CN202111592628 A CN 202111592628A CN 114500310 A CN114500310 A CN 114500310A
- Authority
- CN
- China
- Prior art keywords
- data
- baseline
- time points
- time
- data set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 11
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 4
- 238000004140 cleaning Methods 0.000 claims abstract description 4
- 238000012216 screening Methods 0.000 claims abstract description 4
- 238000012545 processing Methods 0.000 claims abstract description 3
- 238000010606 normalization Methods 0.000 claims abstract 2
- 230000002159 abnormal effect Effects 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000001502 supplementing effect Effects 0.000 claims 1
- 238000012423 maintenance Methods 0.000 abstract description 5
- 238000007619 statistical method Methods 0.000 abstract description 3
- 238000012544 monitoring process Methods 0.000 description 11
- 238000012360 testing method Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 2
- 238000010219 correlation analysis Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 241000408659 Darpa Species 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/04—Processing captured monitoring data, e.g. for logfile generation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Environmental & Geological Engineering (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
Description
技术领域technical field
本发明涉及到网络运维和流量管控领域，尤其涉及一种多维网络态势数据基线的精准确定方法。The invention relates to the field of network operation and maintenance and traffic management and control, in particular to a method for accurately determining a multi-dimensional network situation data baseline.
背景技术Background technique
目前网络中部署的流量监测和探测设备还只能针对单个节点和局部信息，从网络中获取的信息也相对孤立，对链路沿途多点信息的关联分析和综合呈现还很欠缺，无法实现流量的全程全域监测，在网络故障的快速定位、网络运行状态的综合评定等方面还都无法提供全面实时的数据基础；同时，各类网络监测手段独立建设，未形成合力。路由与流量的融合监测和协同控制尚未形成，上下联动多系统协同能力尚未形成。At present, the traffic monitoring and detection equipment deployed in the network can only target single node and local information, the information obtained from the network is relatively isolated, and the correlation analysis and comprehensive presentation of multi-point information along the link is still lacking, and the traffic flow cannot be realized. The whole-domain monitoring of the network cannot provide a comprehensive and real-time data foundation in terms of rapid location of network faults and comprehensive assessment of network operation status. The integrated monitoring and collaborative control of routing and traffic has not yet been formed, and the collaborative capability of up and down linkage of multiple systems has not yet been formed.
目前对网络流量的监测能力只限于千兆以下，对大带宽传输环节的流量监测还处于空白；此外，目前网络中部署的监测设备还只能针对单个节点和局部信息，从网络中获取的信息也相对孤立，对链路沿途多点信息的关联分析和综合呈现还很欠缺，无法实现流量的全程全域监测，在网络故障的快速定位、网络运行状态的综合评定等方面无法提供全面实时的数据基础；同时，流量监测方面还没有形成统一的、适合航天业务网应用特点的规范，各类网络监测手段独立建设，未形成合力。因此对大带宽全域流量感知监测技术的研究和网络流量监测规范的制定迫在眉睫。At present, the monitoring capability of network traffic is limited to below gigabit, and the traffic monitoring of large-bandwidth transmission links is still blank; in addition, the monitoring equipment currently deployed in the network can only obtain information from the network for a single node and local information. It is also relatively isolated, and the correlation analysis and comprehensive presentation of multi-point information along the link is still lacking. It is impossible to realize the whole-process and global monitoring of traffic, and it cannot provide comprehensive real-time data in terms of rapid location of network faults and comprehensive evaluation of network operation status. At the same time, in terms of traffic monitoring, a unified specification suitable for the application characteristics of the aerospace business network has not been formed, and various network monitoring methods have been independently constructed without forming a joint force. Therefore, it is urgent to study the large-bandwidth global traffic-aware monitoring technology and formulate network traffic monitoring specifications.
国际上一些组织提出了一些测试的基础架构。例如：Surveyor是AdvancedNetwork&Services公司联合其他组织提出的基于IPPMWG标准的网络测试基础架构，能够测量参与该项目的组织之间的互联网的路径性能；在该项目中还提出了分析性能数据的方法和工具。MMI是NSF发起、DARPA资助的一个项目，提出了一个基于探针(probe)的分布式可扩展动态的网络测试基础架构。另外还有一些项目如Ripe、AMP、PingER等都是跟网络测试相关的。Several international organizations have proposed some infrastructure for testing. For example, Surveyor is a network testing infrastructure based on the IPPMWG standard proposed by AdvancedNetwork&Services and other organizations, which can measure the path performance of the Internet between organizations participating in the project; methods and tools for analyzing performance data are also proposed in the project. MMI is a project initiated by NSF and funded by DARPA, which proposes a distributed, scalable and dynamic network testing infrastructure based on probes. In addition, some projects such as Ripe, AMP, PingER, etc. are related to network testing.
发明内容SUMMARY OF THE INVENTION
本发明所要解决的技术问题是网络运维管理中，人为主观判定与实际网络流量数据统计分析的阈值差距问题，提出一种多维数据基线精准确定方法。The technical problem to be solved by the present invention is the problem of threshold gap between human subjective judgment and actual network traffic data statistical analysis in network operation and maintenance management, and a method for accurately determining multi-dimensional data baseline is proposed.
本发明所采用的技术方案为：The technical scheme adopted in the present invention is:
一种多维网络态势数据基线的精准确定方法，包括以下步骤：A method for accurately determining a multi-dimensional network situation data baseline, comprising the following steps:
S1：确定数据基线的时间范围；S1: Determine the time range of the data baseline;
S2：进行历史数据清洗和筛选，剔除异常数据和空白数据，并记录被清洗和筛选数据对应的时间点；S2: Perform historical data cleaning and screening, remove abnormal data and blank data, and record the time points corresponding to the cleaned and screened data;
S3：进行对应时间点数据的补齐操作，形成数据基线时间范围内的完整数据集；S3: Perform the complementing operation of the data at the corresponding time point to form a complete data set within the data baseline time range;
S4：进行完整数据集的归一化处理，形成归一化数据集；S4: normalize the complete data set to form a normalized data set;
S5：将归一化数据集中在同一时间点的所有数据求解算术平均值，作为该时间点的数据基线值；S5: Calculate the arithmetic mean of all the data in the normalized data set at the same time point as the data baseline value at this time point;
S6：将所有时间点的数据基线值和时间点形成一一对应关系矩阵，并在平面坐标系中绘制一条曲线，横坐标代表时间点，纵坐标代表对应时间点的维度值；S6: form a one-to-one correspondence matrix between the data baseline values and time points of all time points, and draw a curve in the plane coordinate system, the abscissa represents the time point, and the ordinate represents the dimension value of the corresponding time point;
S7：将数据基线存入基线算法库中；S7: save the data baseline into the baseline algorithm library;
S8：重复步骤S1至S7，进一步精准数据基线值；S8: Repeat steps S1 to S7 to further refine the data baseline value;
完成多维数据基线的精准确定。Accurate determination of multidimensional data baselines is accomplished.
本发明相对于现有技术的优点和创新点如下：The advantages and innovations of the present invention relative to the prior art are as follows:
1、本发明首次提出网络态势数据基线的概念，通过将数据挖掘分析和深度学习训练等新技术应用于网络健康度评估，将采集的大量孤立原始数据，经过多维度建模计算，形成持续迭代动态更新的基线模型库，为网络全生命周期智能运维提供精准数据支撑和判定标准；1. The present invention proposes the concept of network situation data baseline for the first time. By applying new technologies such as data mining analysis and deep learning training to network health assessment, a large amount of isolated raw data collected is subjected to multi-dimensional modeling and calculation to form continuous iterations. The dynamically updated baseline model library provides accurate data support and judgment criteria for intelligent network operation and maintenance throughout the life cycle;
2、本发明提出的多维数据基线集，解决了人为主观判定与实际网络流量数据统计分析的阈值存在的误差问题，提升了网络效能评估指标体系的客观性和科学性，为网络态势预测和健康管理等前瞻性管控提供决策支撑。2. The multi-dimensional data baseline set proposed by the present invention solves the error problem existing in the threshold value of human subjective judgment and statistical analysis of actual network traffic data, improves the objectivity and scientificity of the network performance evaluation index system, and improves the network situation prediction and health. Management and other forward-looking controls provide decision support.
附图说明Description of drawings
图1是本发明的流程图。Figure 1 is a flow chart of the present invention.
具体实施方式Detailed ways
下面结合图1，对本发明的具体实施方式作进一步详细描述。以下实施例用于说明本发明，但不用来限制本发明的范围。The specific embodiments of the present invention will be described in further detail below with reference to FIG. 1 . The following examples are intended to illustrate the present invention, but not to limit the scope of the present invention.
所述的多维网络态势数据主要包括：网络中可采集可量化的状态数据、性能数据、告警数据以及承载的流量特性数据(时延、抖动、丢包、带宽、乱序)；通过对多维度态势数据的分析处理，精准确定并动态更新多类数据的数据基线集，形成数据基线模型库；在应用是，以时间为基准，综合运用多类数据基线，为各类智能运维任务提供网络健康度评估基准。The multi-dimensional network situation data mainly includes: quantifiable status data, performance data, alarm data and carried traffic characteristic data (delay, jitter, packet loss, bandwidth, disorder) that can be collected in the network; Analysis and processing of situational data, accurately determine and dynamically update data baseline sets of multiple types of data, and form a data baseline model library; in application, based on time, multiple types of data baselines are comprehensively used to provide network for various intelligent operation and maintenance tasks. Health Assessment Benchmarks.
一种多维网络态势数据基线的精准确定方法，包括以下步骤：A method for accurately determining a multi-dimensional network situation data baseline, comprising the following steps:
S1：确定数据基线的时间范围；S1: Determine the time range of the data baseline;
S2：进行历史数据清洗和筛选，剔除异常数据和空白数据，并记录被清洗和筛选数据对应的时间点；S2: Perform historical data cleaning and screening, remove abnormal data and blank data, and record the time points corresponding to the cleaned and screened data;
S3：进行对应时间点数据的补齐操作，形成数据基线时间范围内的完整数据集；S3: Perform the complementing operation of the data at the corresponding time point to form a complete data set within the data baseline time range;
S4：进行完整数据集的归一化处理，形成归一化数据集；S4: normalize the complete data set to form a normalized data set;
S5：将归一化数据集中在同一时间点的所有数据求解算术平均值，作为该时间点的数据基线值；S5: Calculate the arithmetic mean of all the data in the normalized data set at the same time point as the data baseline value at this time point;
S6：将所有时间点的数据基线值和时间点形成一一对应关系矩阵，并在平面坐标系中绘制一条曲线，横坐标代表时间点，纵坐标代表对应时间点的维度值；S6: form a one-to-one correspondence matrix between the data baseline values and time points of all time points, and draw a curve in the plane coordinate system, the abscissa represents the time point, and the ordinate represents the dimension value of the corresponding time point;
S7：将数据基线存入基线算法库中；S7: save the data baseline into the baseline algorithm library;
S8：重复步骤S1至S7，进一步精准数据基线值；S8: Repeat steps S1 to S7 to further refine the data baseline value;
完成多维数据基线的精准确定。Accurate determination of multidimensional data baselines is accomplished.
Claims (1)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111592628.1A CN114500310A (en) | 2021-12-23 | 2021-12-23 | An accurate method for determining the baseline of multi-dimensional network situation data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111592628.1A CN114500310A (en) | 2021-12-23 | 2021-12-23 | An accurate method for determining the baseline of multi-dimensional network situation data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114500310A true CN114500310A (en) | 2022-05-13 |
Family
ID=81494799
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111592628.1A Pending CN114500310A (en) | 2021-12-23 | 2021-12-23 | An accurate method for determining the baseline of multi-dimensional network situation data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114500310A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118641124A (en) * | 2024-08-15 | 2024-09-13 | 山东雅拓集团有限公司 | Abnormal flow measurement method and system for safe gas operation |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1677934A (en) * | 2004-03-31 | 2005-10-05 | 华为技术有限公司 | Method and system for monitoring network service performance |
| US7962607B1 (en) * | 2006-09-08 | 2011-06-14 | Network General Technology | Generating an operational definition of baseline for monitoring network traffic data |
| CN108111463A (en) * | 2016-11-24 | 2018-06-01 | 蓝盾信息安全技术有限公司 | The self study of various dimensions baseline and abnormal behaviour analysis based on average value and standard deviation |
| CN109167794A (en) * | 2018-09-25 | 2019-01-08 | 北京计算机技术及应用研究所 | A kind of attack detection method of network-oriented system security measure |
| CN109684310A (en) * | 2018-11-22 | 2019-04-26 | 安徽继远软件有限公司 | A kind of information system performance Situation Awareness method based on big data analysis |
| CN110134566A (en) * | 2019-04-29 | 2019-08-16 | 国网上海市电力公司 | A method for monitoring information system performance in cloud environment based on tag technology |
| CN113568954A (en) * | 2021-08-02 | 2021-10-29 | 湖北工业大学 | Parameter optimization method and system for network traffic prediction data preprocessing stage |
-
2021
- 2021-12-23 CN CN202111592628.1A patent/CN114500310A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1677934A (en) * | 2004-03-31 | 2005-10-05 | 华为技术有限公司 | Method and system for monitoring network service performance |
| US7962607B1 (en) * | 2006-09-08 | 2011-06-14 | Network General Technology | Generating an operational definition of baseline for monitoring network traffic data |
| CN108111463A (en) * | 2016-11-24 | 2018-06-01 | 蓝盾信息安全技术有限公司 | The self study of various dimensions baseline and abnormal behaviour analysis based on average value and standard deviation |
| CN109167794A (en) * | 2018-09-25 | 2019-01-08 | 北京计算机技术及应用研究所 | A kind of attack detection method of network-oriented system security measure |
| CN109684310A (en) * | 2018-11-22 | 2019-04-26 | 安徽继远软件有限公司 | A kind of information system performance Situation Awareness method based on big data analysis |
| CN110134566A (en) * | 2019-04-29 | 2019-08-16 | 国网上海市电力公司 | A method for monitoring information system performance in cloud environment based on tag technology |
| CN113568954A (en) * | 2021-08-02 | 2021-10-29 | 湖北工业大学 | Parameter optimization method and system for network traffic prediction data preprocessing stage |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118641124A (en) * | 2024-08-15 | 2024-09-13 | 山东雅拓集团有限公司 | Abnormal flow measurement method and system for safe gas operation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108008332B (en) | New energy remote testing equipment fault diagnosis method based on data mining | |
| US20220057367A1 (en) | Method for evaluating pipe condition | |
| CN108446711B (en) | A software defect prediction method based on transfer learning | |
| JP2021057042A (en) | Product classification system and product classification method | |
| CN113065702B (en) | Multi-linear prediction method of landslide displacement based on ST-SEEP segmentation method and space-time ARMA model | |
| CN111130890A (en) | Network flow dynamic prediction system | |
| CN111832432A (en) | A real-time prediction method of tool wear based on wavelet packet decomposition and deep learning | |
| WO2017071369A1 (en) | Method and device for predicting user unsubscription | |
| CN117667585B (en) | A method and system for evaluating operation and maintenance efficiency based on operation and maintenance quality management database | |
| CN106125713B (en) | The assessment of reliability growth and prediction technique in the case of a kind of Interval censoring | |
| CN114500310A (en) | An accurate method for determining the baseline of multi-dimensional network situation data | |
| Xiang et al. | Fault classification for high‐dimensional data streams: A directional diagnostic framework based on multiple hypothesis testing | |
| JP5889759B2 (en) | Missing value prediction device, missing value prediction method, missing value prediction program | |
| CN111090585B (en) | An automatic prediction method for the closing time of public testing tasks based on the public testing process | |
| CN110083933B (en) | Bayesian degradation analysis method for corrosion pipeline considering random effect | |
| CN106330544B (en) | A real-time alarm response method based on self-feedback model | |
| CN105117606A (en) | Method for determining element fault probability change tendency | |
| CN117135037A (en) | Method and device for defining network traffic performance abnormality | |
| CN114818460A (en) | Laboratory equipment residual service life prediction method based on automatic machine learning | |
| Smit et al. | Autonomic configuration adaptation based on simulation-generated state-transition models | |
| Kaur et al. | Comparative analysis of the software effort estimation models | |
| CN116773238B (en) | A fault monitoring method and system based on industrial data | |
| CN117370775A (en) | Method, equipment and storage medium for selecting optimal parameters of transverse wave speed of oil and gas well | |
| CN118883875A (en) | A water pollution monitoring system and method based on machine learning | |
| RU2571470C2 (en) | Method of neural network analysis of telemetry data for well stock |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220513 |
|
| RJ01 | Rejection of invention patent application after publication |