CN111654327A - Service feature extraction method for optical cable fiber core remote management control - Google Patents

Service feature extraction method for optical cable fiber core remote management control Download PDF

Info

Publication number
CN111654327A
CN111654327A CN201911090290.2A CN201911090290A CN111654327A CN 111654327 A CN111654327 A CN 111654327A CN 201911090290 A CN201911090290 A CN 201911090290A CN 111654327 A CN111654327 A CN 111654327A
Authority
CN
China
Prior art keywords
wavelet packet
network
time
representing
mid
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911090290.2A
Other languages
Chinese (zh)
Inventor
刘扬
李欢
孟凡博
王刚
蒋定德
陈得丰
杨智斌
耿洪碧
任帅
李桐
佟昊松
南洋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
State Grid Corp of China SGCC
Electric Power Research Institute of State Grid Liaoning Electric Power Co Ltd
Original Assignee
State Grid Corp of China SGCC
Electric Power Research Institute of State Grid Liaoning Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by State Grid Corp of China SGCC, Electric Power Research Institute of State Grid Liaoning Electric Power Co Ltd filed Critical State Grid Corp of China SGCC
Priority to CN201911090290.2A priority Critical patent/CN111654327A/en
Publication of CN111654327A publication Critical patent/CN111654327A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B10/00—Transmission systems employing electromagnetic waves other than radio-waves, e.g. infrared, visible or ultraviolet light, or employing corpuscular radiation, e.g. quantum communication
    • H04B10/25—Arrangements specific to fibre transmission
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0876—Network utilisation, e.g. volume of load or congestion level

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Environmental & Geological Engineering (AREA)
  • Physics & Mathematics (AREA)
  • Electromagnetism (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention belongs to network characteristic analysis in a large-scale network environment, and particularly relates to a service characteristic extraction method for optical cable fiber core remote management control. The invention comprises the following steps: obtaining an initial network flow matrix of the optical fiber network, and determining the wavelet packet decomposition times; performing wavelet packet transformation on the initial network flow to obtain a wavelet packet coefficient; dividing the obtained wavelet packet coefficients into a plurality of components; respectively carrying out reverse transformation on the plurality of components to obtain corresponding network flow components; respectively mining the obtained network flow components by using a cluster analysis method, and constructing the network flow components with the required characteristics; reconstructing the total network flow according to the constructed network flow component with the required characteristics; further mining hidden features of the reconstructed network total traffic using principal component analysis; and storing the hidden characteristic result of the reconstructed network total flow, and exiting. The invention improves the accuracy of characteristic analysis and has higher timeliness and accuracy.

Description

Service feature extraction method for optical cable fiber core remote management control
Technical Field
The invention belongs to network characteristic analysis in a large-scale network environment, and particularly relates to a service characteristic extraction method for optical cable fiber core remote management control.
Background
With the rapid emergence of various emerging network technologies and applications, optical fiber networks are commonly used in modern network construction, and optical fiber communication technologies use optical fibers as transmission media and utilize light to transmit information. Network traffic in fiber optic network activities exhibits increasingly new, unknown characteristics. In addition, some new applications such as crowdsourcing, online payment, online networking, etc. have created new traffic circulation patterns and features. These changes inevitably affect the performance and network operation of the current network, and the new characteristics of the network traffic also cause problems of path delay, packet loss, network failure, etc., and in addition, abnormal network traffic will directly affect the user experience and interfere with normal network activities. Therefore, it is important for operators and users to effectively capture network traffic characteristics. The feature analysis of network traffic has become a hot problem in academia and industry.
The optical fiber communication network generally has a plurality of traffics from an inlet node to an outlet node, and the network traffics show the characteristics of correlation, self-similarity, time-varying property and the like, so that the network traffics are difficult to model and describe by a general modeling method, and a new analysis method is necessary.
The problem of traffic characteristic analysis is mainly reflected in that the characteristics of network traffic can be accurately mined only by adopting a proper method for analysis on the traffic.
The existing research detects the attack behavior of denial of service distributed at a low rate by measuring the difference value between legal flow and attack flow and utilizing generalized entropy measurement and information distance measurement; searching and detecting abnormal phenomena in the network by using the spatial-temporal correlation and empirical measurement; identifying network anomalies using global traffic statistics and distributed spatial detection methods; identifying network abnormality by using characteristic analysis, and detecting abnormal network flow by using signal conversion from the perspective of global flow; establishing a model to detect a network event by analyzing network traffic characteristics; the variation index description network is introduced, and the problem of TCP flow abnormity of a router with a small buffer area is researched; periodically based network traffic anomalies are studied and used to identify traffic anomalies; detecting network abnormal behavior by using total flow statistics; anomaly detection of cloud computing network traffic is studied. The method provides various ideas for analyzing the network traffic characteristics, and can capture the network traffic characteristics to a certain extent, but all have larger errors. Therefore, new methods are still needed to analyze network traffic characteristics at this stage.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a service characteristic extraction method for optical cable fiber core remote management control, so as to achieve the purpose of accurately capturing network flow characteristics in real time.
Based on the above purpose, the invention is realized by the following technical scheme:
a service feature extraction method facing optical cable fiber core remote management control comprises the following steps:
step 1: obtaining an initial network flow matrix of the optical fiber network, and determining the wavelet packet decomposition times;
step 2: performing wavelet packet transformation on the initial network flow to obtain a wavelet packet coefficient;
and step 3: dividing the obtained wavelet packet coefficients into a plurality of components;
and 4, step 4: respectively carrying out reverse transformation on the plurality of components to obtain corresponding network flow components;
and 5: respectively mining the obtained network flow components by using a cluster analysis method, and constructing the network flow components with the required characteristics;
step 6: reconstructing the total network flow according to the constructed network flow component with the required characteristics;
and 7: further mining hidden features of the reconstructed network total traffic using principal component analysis;
and 8: and storing the hidden characteristic result of the reconstructed network total flow, and exiting.
The obtained network flow components are respectively mined by using a cluster analysis method, and the distance from each object or data point to a cluster center is used as a similarity and a clustering standard, so that the sum of squares of the total distances of each cluster is minimum.
The method for determining the wavelet packet coefficient comprises the following steps: determining a wavelet packet decomposition frequency n _ scale, obtaining an initial flow matrix, and obtaining the network flow x from a source node i to a target node j from the matrixij＝{xij(1),xij(2) ,., the formula is as follows:
Figure BDA0002266646220000022
wherein the content of the first and second substances,
Figure BDA0002266646220000023
the wavelet packet transform coefficient with the k time of 2n is represented under the l-th layer decomposition;
Figure BDA0002266646220000024
a wavelet packet transform coefficient with k time of 2n +1 is represented under the l-th layer decomposition;
Figure BDA0002266646220000025
the wavelet packet transformation coefficient with the scale of k +1 and the time of n under the s-th layer decomposition is represented; a iss-2lRepresents the s-2l scale converter; bs-2lRepresents the s-2l scale transformer; s is a variable;
Figure BDA0002266646220000026
represents wavelet packet transform coefficients with a scale k-time of n at the l-th layer decomposition (where n is 2n or 2n +1),
Figure BDA0002266646220000027
satisfy the requirement of
Figure BDA0002266646220000031
In the above-mentioned formula (2),
Figure BDA0002266646220000032
wavelet packet transform coefficients with a k-time n (where n is 2n or 2n +1) representing the scale at the l-th layer decomposition; x is the number ofij(t) represents network traffic from source node i to destination node j at time t; f represents the network traffic xij(t) a function for performing wavelet packet transformation;
Figure BDA0002266646220000033
pair from source node i to destination node j with a representation scale kNetwork flow with time of n corresponding to wavelet packet transform domain;
Figure BDA0002266646220000034
representing the network flow of a corresponding continuous time domain moment t from a source node i to a destination node j, wherein the corresponding packet transformation scale is k, and the time is n;
and is
Figure BDA0002266646220000035
Figure BDA0002266646220000036
Representing a subspace and a wavelet space, formed by
Figure BDA0002266646220000037
To obtain
Figure BDA0002266646220000038
And
Figure BDA0002266646220000039
wherein
Figure BDA00022666462200000310
The wavelet packet transform coefficient with the k time of 2n is represented under the l-th layer decomposition;
Figure BDA00022666462200000311
a wavelet packet transform coefficient with k time of 2n +1 is represented under the l-th layer decomposition;
Figure BDA00022666462200000312
wavelet packet transform coefficients with a k-time n (where n is 2n or 2n +1) representing the scale at the l-th layer decomposition;
the wavelet packet is reconstructed, resulting in the following transformation [ x ]:
Figure BDA00022666462200000313
in the above-mentioned formula (3),
Figure BDA00022666462200000314
representing wavelet packet transform coefficients with a k +1 time n under the l-th layer decomposition;
Figure BDA00022666462200000315
wavelet packet transform coefficients with the k time of 2n representing the dimension under the s-th layer decomposition;
Figure BDA00022666462200000316
the wavelet packet transformation coefficient with the k time of 2n +1 under the s-th layer decomposition is represented; h isl-2sRepresents the l-2s high-pass filter; gl-2sRepresents the l-2s low-pass filter;
network traffic xij(t) different scale characteristics, using wavelet packet coefficients
Figure BDA00022666462200000317
Represents:
Figure BDA00022666462200000318
in the above formula (4), xij(t) represents network traffic from source node i to destination node j at time t; f-1Representing network traffic xij(t) an inverse function of the function for performing the wavelet packet transform;
Figure BDA0002266646220000041
represents the wavelet packet transform coefficients at the l-th layer decomposition with a scale of k +1 and time of n.
The wavelet packet coefficients include: low frequency component, medium frequency component and high frequency component, respectively
Figure BDA0002266646220000042
And
Figure BDA0002266646220000043
representing the wavelet packet coefficients of low, intermediate and high frequencies, respectively.
The performing inverse transformation on the plurality of components respectively to obtain corresponding network traffic components includes:
obtaining a low frequency component xij,low(t), intermediate frequency component xij,mid(t) and a high frequency component xij,high(t), the formula is as follows:
Figure BDA0002266646220000044
Figure BDA0002266646220000045
Figure BDA0002266646220000046
in the above-mentioned formulas (5) to (7),
Figure BDA0002266646220000047
and
Figure BDA0002266646220000048
wavelet packet coefficients representing low, intermediate and high frequencies, respectively; f-1Representation to network traffic xij(t) an inverse function of the function for performing the wavelet packet transform; x is the number ofij,low(t)、 xij,mid(t) and xij,high(t) respectively represents the network traffic x from the source node i to the destination node j at time tijLow, mid, and high frequency time domain components in (t).
The cluster analysis method comprises the following steps:
performing clustering analysis by using a k-means clustering analysis method, introducing Euclidean distance from a vector X to a vector Y in a multidimensional space as similarity measure, and calculating the Euclidean distance as follows:
Figure BDA0002266646220000049
wherein X and Y are n-dimensional vectors in accordance with
Figure BDA00022666462200000410
Get h data points
Figure BDA00022666462200000411
The same reason is according to
Figure BDA00022666462200000412
Obtaining N-h data points; dividing the data object into k clusters by using a k-mean algorithm, selecting Euclidean distance as similarity and clustering standard, completing reconstruction of network flow, and calculating the distance u from each object or data point to the clustering centeriThe following equation is obtained:
Figure BDA0002266646220000051
wherein, ckRepresents a cluster, each cluster ckAll have a center uiUsing a clustering algorithm to make the total distance of each cluster
Figure BDA0002266646220000052
The sum of the squares of (a) is minimal, the following equation holds:
Figure BDA0002266646220000053
Figure BDA0002266646220000054
Figure BDA0002266646220000055
in the above formulae (11) and (12), Jmid(Cmid) Representing a set of clusters CmidA total distance; j. the design is a squaremid(ck,mid) Representing a set of clusters ck,midA total distance; | d |i,mid-uk||2Represents a point di,midTo the clustering center ukSquare of the distance of (d); j. the design is a squarehigh(Chigh) Representing a set of clusters ChighA total distance; j. the design is a squarehigh(ck,high) Representing a set of clusters ck,highA total distance; | d |i,high-uk||2Represents a point di,highTo the clustering center ukSquare of the distance of (d);
wherein the content of the first and second substances,
Figure BDA0002266646220000056
thus, obtained by clustering algorithm
Figure BDA0002266646220000057
And
Figure BDA0002266646220000058
and inverse transformation is carried out to obtain the reconstructed flow component xij,low(t)、xij,mid(t) and xij,high(t)。
The total flow method of the reconstruction network is as follows:
Figure BDA0002266646220000059
in the above formula (13), xij,low(t)、xij,mid(t) and xij,high(t) respectively represents the network traffic x from the source node i to the destination node j at time tijLow, medium, and high frequency time domain components in (t);
Figure BDA00022666462200000510
representing the network traffic x from the source node i to the destination node j at time tij(t) reconstructed value.
The invention has the following advantages and beneficial effects:
the invention relates to a service characteristic extraction method facing to optical cable fiber core remote management control, which adopts a time-frequency analysis theory based on wavelet packet transformation to convert network flow into a time-frequency domain, divides the network flow into low-frequency, medium-frequency and high-frequency characteristics in the time-frequency domain, and divides the network flow into three components, thereby providing convenience for mining; mining is carried out through a clustering analysis theory, network flow is reconstructed, and accuracy of characteristic analysis is improved; and then, the abnormal part in the network flow is further found by utilizing a principal component analysis method, the flow characteristic is further excavated, and the analysis accuracy is improved. The method analyzes the network traffic data in the time-frequency domain, and has higher timeliness and accuracy.
Drawings
The technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiment of the present invention. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. Based on the embodiments of the present invention, other embodiments obtained by persons of ordinary skill in the art without any creative effort belong to the protection scope of the present invention.
FIG. 1 is a general flowchart of a method for extracting service features for remote management control of fiber cores of optical cables according to the present invention;
FIG. 2a is a graph illustrating network traffic over time in a normal context according to the present invention;
FIG. 2b is a graph illustrating network traffic over time under abnormal conditions in accordance with the present invention;
FIG. 3a is a schematic diagram of the present invention after applying wavelet packet transformation with a 4-dimension to the flow;
FIG. 3b is a schematic diagram of the present invention after applying a wavelet packet transform with a scale of 8 to the traffic;
FIG. 3c is a schematic diagram of the present invention after applying a wavelet packet transform with a dimension of 12 to the traffic;
FIG. 3d is a schematic diagram of the present invention after performing wavelet packet transformation with a 16-dimension on the traffic;
FIG. 3e is a schematic diagram of the present invention after applying wavelet packet transformation with a dimension of 20 to the flow;
FIG. 3f is a schematic diagram of the present invention after applying a wavelet packet transform with a scale of 24 to the traffic;
FIG. 3g is a schematic diagram of the present invention after performing a wavelet packet transform with a scale of 28 on the traffic;
FIG. 3h is a schematic diagram of the present invention after applying a wavelet packet transform with a dimension of 32 to the traffic;
FIG. 4 is a schematic diagram of the main components of the network traffic mined by the present invention;
FIG. 5 is a diagram of the results of the anomaly detection simulation of the present invention.
Detailed Description
The invention relates to a service characteristic extraction method for optical cable fiber core remote management control, which is used for excavating the characteristics of network flow.
Example 1:
as shown in fig. 1, fig. 1 is a general flowchart of an embodiment of the method for extracting service features for remote management and control of a fiber core of an optical cable according to the present invention.
A service feature extraction method facing optical cable fiber core remote management control comprises the following steps:
step 1: obtaining an initial network flow matrix of the optical fiber network, and determining the wavelet packet decomposition times;
step 2: performing wavelet packet transformation on the initial network flow to obtain a wavelet packet coefficient;
and step 3: dividing the wavelet packet coefficients obtained in the step 2 into a plurality of components; the plurality of components includes: a low frequency component, a medium frequency component, and a high frequency component;
and 4, step 4: separately for the plurality of components obtained in step 3: performing inverse transformation on the low-frequency component, the medium-frequency component and the high-frequency component to obtain three corresponding network flow components;
and 5: respectively mining the three network flow components obtained in the step 4 by using a cluster analysis method, and taking the distance from each object or data point to a cluster center as similarity and a cluster standard to minimize the sum of squares of the total distance of each cluster and construct three network flow components with required characteristics;
step 6: reconstructing the total network flow according to the result obtained by mining in the step 5;
and 7: further mining hidden characteristics of the total network flow reconstructed in the step 6 by using principal component analysis;
and 8: and storing the result of the hidden characteristics of the reconstructed network total flow and exiting.
The step 2: performing wavelet packet transformation on the initial flow to obtain a wavelet packet coefficient, comprising:
giving an initial flow matrix as an analysis object, determining a wavelet packet decomposition time n _ scale, and obtaining the network flow x from a source node i to a target node j from the matrix after obtaining the initial flow matrixij＝{xij(1),xij(2) ,., the network traffic is time-domain complex, its characteristics are generally difficult to directly observe, requiring further processing.
First, a network flow x is constructedij＝{xij(1),xij(2) ,., the formula is as follows:
Figure BDA0002266646220000072
wherein the content of the first and second substances,
Figure BDA0002266646220000073
the wavelet packet transform coefficient with the k time of 2n is represented under the l-th layer decomposition;
Figure BDA0002266646220000074
a wavelet packet transform coefficient with k time of 2n +1 is represented under the l-th layer decomposition;
Figure BDA0002266646220000075
the wavelet packet transformation coefficient with the scale of k +1 and the time of n under the s-th layer decomposition is represented; a iss-2lRepresents the s-2l scale converter; bs-2lRepresents the s-2l scale transformer; s is a variable;
Figure BDA0002266646220000081
represents wavelet packet transform coefficients with a scale k-time of n at the l-th layer decomposition (where n is 2n or 2n +1),
Figure BDA0002266646220000082
satisfy the requirement of
Figure BDA0002266646220000083
In the above-mentioned formula (2),
Figure BDA0002266646220000084
wavelet packet transform coefficients with a k-time n (where n is 2n or 2n +1) representing the scale at the l-th layer decomposition; x is the number ofij(t) represents network traffic from source node i to destination node j at time t; f represents the network traffic xij(t) a function for performing wavelet packet transformation;
Figure BDA0002266646220000085
representing the network flow with the scale k, from the source node i to the destination node j and the corresponding wavelet packet transform domain time being n;
Figure BDA0002266646220000086
and the network traffic of the corresponding continuous time domain time t from the source node i to the destination node j with the corresponding packet transformation scale of k time of n is represented.
And is
Figure BDA0002266646220000087
Figure BDA0002266646220000088
Representing a subspace and a wavelet space. Then, according to equation (1), can be prepared
Figure BDA0002266646220000089
To obtain
Figure BDA00022666462200000810
And
Figure BDA00022666462200000811
wherein
Figure BDA00022666462200000812
The wavelet packet transform coefficient with the k time of 2n is represented under the l-th layer decomposition;
Figure BDA00022666462200000813
the wavelet packet transform coefficient with the k time of 2n +1 is represented under the l-th layer decomposition;
Figure BDA00022666462200000814
the wavelet packet transform coefficient with the k time n (where n is 2n or 2n +1) at the l-th layer decomposition is represented.
Finally, reconstructing the wavelet packet to obtain the following transformation:
Figure BDA00022666462200000815
in the above-mentioned formula (3),
Figure BDA00022666462200000816
representing wavelet packet transform coefficients with a k +1 time n under the l-th layer decomposition;
Figure BDA00022666462200000817
wavelet packet transform coefficients with the k time of 2n representing the dimension under the s-th layer decomposition;
Figure BDA00022666462200000818
the wavelet packet transformation coefficient with the k time of 2n +1 under the s-th layer decomposition is represented; h isl-2sRepresents the l-2s high-pass filter; gl-2sThe l-2s low pass filter is shown.
According to
Figure BDA00022666462200000819
And
Figure BDA00022666462200000820
computing
Figure BDA00022666462200000821
Network traffic xij(t) is noted as:
Figure BDA00022666462200000822
in the above formula (4), xij(t) represents network traffic from source node i to destination node j at time t; f-1Representing network traffic xij(t) an inverse function of the function for performing the wavelet packet transform;
Figure BDA0002266646220000091
represents the wavelet packet transform coefficients at the l-th layer decomposition with a scale of k +1 and time of n.
Network traffic xij(t) shows different scale characteristics, using wavelet packet coefficients
Figure BDA0002266646220000092
And (4) showing. In order to perform the clustering process efficiently, we will use the network traffic x, which has different frequency characteristics in the low, medium and high frequency componentsij(t) separation into low, medium and high frequency components, using
Figure BDA0002266646220000093
And
Figure BDA0002266646220000094
representing the wavelet packet coefficients of low, intermediate and high frequencies, respectively.
Wavelet packet coefficient obtained according to the step 4
Figure BDA0002266646220000095
And equation (4) performs an inverse transform to obtain the low frequency component xij,low(t), intermediate frequency component xij,mid(t) and a high frequency component xij,high(t), the formula is as follows:
Figure BDA0002266646220000096
Figure BDA0002266646220000097
Figure BDA0002266646220000098
in the above-mentioned formulas (5) to (7),
Figure BDA0002266646220000099
and
Figure BDA00022666462200000910
wavelet packet coefficients representing low, intermediate and high frequencies, respectively; f-1Representation to network traffic xij(t) an inverse function of the function for performing the wavelet packet transform; x is the number ofij,low(t)、 xij,mid(t) and xij,high(t) respectively represents the network traffic x from the source node i to the destination node j at time tijLow, mid, and high frequency time domain components in (t).
According to step 5, xij,low(t)、xij,mid(t) and xij,high(t)。
Feature components of the three flows are mined using k-mean cluster analysis. According to the k-means analysis theory, Euclidean distance from a vector X to a vector Y in a multi-dimensional space is introduced as a similarity measure. The euclidean distance is calculated as follows:
Figure BDA00022666462200000911
wherein X and Y are n-dimensional vectors. According to
Figure BDA00022666462200000912
Get h data points
Figure BDA00022666462200000913
The same reason is according to
Figure BDA00022666462200000914
N-h data points were obtained. The data object is segmented into k clusters using the k-mean algorithm. Selecting Euclidean distance as similarity and clustering criterion to calculate the distance u from each object or data point to the clustering centeriThe following equation is obtained:
Figure BDA0002266646220000101
wherein, ckRepresenting a clusterEach cluster ckAll have a center uk，ukRepresenting a cluster ckCenter of (d)iRepresenting a cluster ckThe ith point of (1) is di∈ck。
Using a clustering algorithm to make each cluster total distance
Figure BDA0002266646220000102
The sum of squares of (a) is minimal. For low frequency signal xij,low(t), the following equation holds true:
Figure BDA0002266646220000103
in the above formula (10), Jlow(Clow) Representing a set of clusters ClowA total distance; j. the design is a squarelow(ck,low) Representing a set of clusters ck,lowA total distance; | d |i,low-uk||2Represents a point di,lowTo the clustering center ukSquare of the distance of (d); dkiAre variables and satisfy:
Figure BDA0002266646220000104
for x, the same principle appliesij,mid(t) and xij,high(t), the following equation holds true:
Figure BDA0002266646220000105
Figure BDA0002266646220000106
in the above formulae (11) and (12), Jmid(Cmid) Representing a set of clusters CmidA total distance; j. the design is a squaremid(ck,mid) Representing a set of clusters ck,midA total distance; | d |i,mid-uk||2Represents a point di,midTo the clustering center ukSquare of the distance of (d); j. the design is a squarehigh(Chigh) Representing a collection of clustersChighA total distance; j. the design is a squarehigh(ck,high) Representing a set of clusters ck,highA total distance; | d |i,high-uk||2Represents a point di,highTo the clustering center ukIs squared.
Thus, obtained by clustering algorithm
Figure BDA0002266646220000108
And
Figure BDA0002266646220000109
obtained according to step 6
Figure BDA00022666462200001010
And
Figure BDA00022666462200001011
performing inverse transformation to obtain reconstructed flow component xij,low(t)、xij,mid(t) and xij,high(t), these components have the desired characteristics, and the total network traffic is reconstructed in the time domain as follows:
Figure BDA0002266646220000111
in the above formula (13), xij,low(t)、xij,mid(t) and xij,high(t) respectively represents the network traffic x from the source node i to the destination node j at time tijLow, medium, and high frequency time domain components in (t);
Figure BDA0002266646220000112
representing the network traffic x from the source node i to the destination node j at time tij(t) reconstructed value.
Step 7 for reconstructed network traffic
Figure BDA0002266646220000113
Principal component analysis is performed, and the principal component analysis method is prior art and is not described herein again. Reconstructed network traffic has moreObvious characteristics, can be deeply mined through principal component analysis.
Example 2:
during the actual simulation, the simulation uses real network data in order to better verify the algorithm. The figure shows two cases of the presence and absence of an anomaly, namely fig. 2a shows the traffic in the normal background and fig. 2b shows the network traffic in the anomaly. It is shown that it is difficult to observe the difference between normal and abnormal network traffic. In the simulation, the network traffic anomalies in FIG. 2b were constructed by adding anomalous network traffic to the normal network traffic in FIG. 2 a. Fig. 2 shows that we cannot directly detect and diagnose the abnormal part of the network traffic. We used the invention in simulations to analyze the abnormal network traffic in fig. 2 b.
To illustrate the wavelet packet transformation at 32 different scales of network traffic without loss of generality, we show in fig. 3 the wavelet packet transformation at 8 different scales, including fig. 3 a-3 h. As can be seen from the figure, the network traffic has different time-frequency characteristics in the time-frequency domain for different transform scales. FIG. 3a shows the high frequency characteristics of the flow shown at scale 4; 3b-3d show that medium frequency characteristics of network traffic at scales 8, 12, and 16 can be effectively captured; figures 3e-3h show that at scales 20, 24, 28 and 32, the low frequency features of the network traffic can also be accurately extracted and mined.
Fig. 4 shows the main constituent features of the network traffic mined by the present invention, and it can be seen that the main components of the network traffic are correctly extracted.
Fig. 5 shows the abnormality detection result of the present invention. In the simulation, we add abnormal network traffic at four different times at time points 300, 500, 800 and 1200, respectively, with a duration of 50 unit slots. As can be seen from fig. 5, the present invention can detect the abnormal portions in the network traffic at the four different time points properly and accurately, compared to the network traffic at other time points which are far below the detection threshold line.
Although specific embodiments of the present invention have been described above, it will be appreciated by those skilled in the art that these are merely illustrative and that various changes or modifications may be made to these embodiments without departing from the principles and spirit of the invention. The scope of the invention is only limited by the appended claims.

Claims (7)

1. A service feature extraction method facing optical cable fiber core remote management control is characterized in that: the method comprises the following steps:
step 1: obtaining an initial network flow matrix of the optical fiber network, and determining the wavelet packet decomposition times;
step 2: performing wavelet packet transformation on the initial network flow to obtain a wavelet packet coefficient;
and step 3: dividing the obtained wavelet packet coefficients into a plurality of components;
and 4, step 4: respectively carrying out reverse transformation on the plurality of components to obtain corresponding network flow components;
and 5: respectively mining the obtained network flow components by using a cluster analysis method, and constructing the network flow components with the required characteristics;
step 6: reconstructing the total network flow according to the constructed network flow component with the required characteristics;
and 7: further mining hidden features of the reconstructed network total traffic using principal component analysis;
and 8: and storing the hidden characteristic result of the reconstructed network total flow, and exiting.
2. The method for extracting service features oriented to optical cable fiber core remote management control according to claim 1, wherein the method comprises the following steps: and respectively mining the obtained network flow components by using a cluster analysis method, and taking the distance from each object or data point to a cluster center as similarity and a clustering standard to minimize the sum of squares of the total distances of each cluster.
3. The method for extracting service features oriented to optical cable fiber core remote management control according to claim 1, wherein the method comprises the following steps: the method for determining the wavelet packet coefficient comprises the following steps: determining a wavelet packet decompositionThe number n _ scale of times, after the initial flow matrix is obtained, the network flow x from the source node i to the target node j is obtained from the matrixij＝{xij(1),xij(2) ,., the formula is as follows:
Figure FDA0002266646210000011
wherein the content of the first and second substances,
Figure FDA0002266646210000012
the wavelet packet transform coefficient with the k time of 2n is represented under the l-th layer decomposition;
Figure FDA0002266646210000013
a wavelet packet transform coefficient with k time of 2n +1 is represented under the l-th layer decomposition;
Figure FDA0002266646210000014
the wavelet packet transformation coefficient with the scale of k +1 and the time of n under the s-th layer decomposition is represented; a iss-2lRepresents the s-2l scale transformer; bs-2lRepresents the s-2l scale transformer; s is a variable;
Figure FDA0002266646210000015
represents wavelet packet transform coefficients with a scale k-time of n at the l-th layer decomposition (where n is 2n or 2n +1),
Figure FDA0002266646210000016
satisfy the requirement of
Figure FDA0002266646210000021
In the above-mentioned formula (2),
Figure FDA0002266646210000022
wavelet packet transform coefficients with a k-time n (where n is 2n or 2n +1) representing the scale at the l-th layer decomposition; x is the number ofij(t) represents network traffic from source node i to destination node j at time t; f represents the network traffic xij(t) a function for performing wavelet packet transformation;
Figure FDA0002266646210000023
representing the network flow with the scale k, from the source node i to the destination node j and the corresponding wavelet packet transform domain time being n;
Figure FDA0002266646210000024
representing the network flow of the corresponding continuous time domain time t from the source node i to the destination node j with the corresponding small packet transformation scale of k time being n;
and is
Figure FDA0002266646210000025
Figure FDA0002266646210000026
Representing a subspace and a wavelet space, formed by
Figure FDA0002266646210000027
To obtain
Figure FDA0002266646210000028
And
Figure FDA0002266646210000029
wherein
Figure FDA00022666462100000210
The wavelet packet transform coefficient with the k time of 2n is represented under the l-th layer decomposition;
Figure FDA00022666462100000211
a wavelet packet transform coefficient with k time of 2n +1 is represented under the l-th layer decomposition;
Figure FDA00022666462100000212
wavelet packet transform coefficients with a k-time n (where n is 2n or 2n +1) representing the scale at the l-th layer decomposition;
the wavelet packet is reconstructed, resulting in the following transformation [ x ]:
Figure FDA00022666462100000213
in the above-mentioned formula (3),
Figure FDA00022666462100000214
representing wavelet packet transform coefficients with a k +1 time n under the l-th layer decomposition;
Figure FDA00022666462100000215
wavelet packet transform coefficients with the k time of 2n representing the dimension under the s-th layer decomposition;
Figure FDA00022666462100000216
wavelet packet transform coefficients with a k time of 2n +1 representing the level under the s-th decomposition; h isl-2sRepresents the l-2s high-pass filter; gl-2sRepresents the l-2s low-pass filter;
network traffic xij(t) different scale characteristics, using wavelet packet coefficients
Figure FDA00022666462100000217
Represents:
Figure FDA00022666462100000218
in the above formula (4), xij(t) represents network traffic from source node i to destination node j at time t; f-1Representation to network traffic xij(t) an inverse function of the function for performing the wavelet packet transform;
Figure FDA0002266646210000031
represents the wavelet packet transform coefficients at the l-th layer decomposition with a scale of k +1 and time of n.
4. The method for extracting service features oriented to optical cable fiber core remote management control according to claim 1, wherein the method comprises the following steps: the wavelet packet coefficients include: low frequency component, medium frequency component and high frequency component, respectively
Figure FDA0002266646210000032
Figure FDA0002266646210000033
And
Figure FDA0002266646210000034
representing the wavelet packet coefficients of low, intermediate and high frequencies, respectively.
5. The method for extracting service features oriented to optical cable fiber core remote management control according to claim 1, wherein the method comprises the following steps: the performing inverse transformation on the plurality of components respectively to obtain corresponding network traffic components includes:
obtaining a low frequency component xij,low(t), intermediate frequency component xij,mid(t) and a high frequency component xij,high(t), the formula is as follows:
Figure FDA0002266646210000035
Figure FDA0002266646210000036
Figure FDA0002266646210000037
in the above-mentioned formulas (5) to (7),
Figure FDA0002266646210000038
and
Figure FDA0002266646210000039
wavelet packet coefficients representing low, intermediate and high frequencies, respectively; f-1Representation to network traffic xij(t) an inverse function of the function for performing the wavelet packet transform; x is the number ofij,low(t)、xij,mid(t) and xij,high(t) respectively represents the network traffic x from the source node i to the destination node j at time tijLow, mid, and high frequency time domain components in (t).
6. The method for extracting service features oriented to optical cable fiber core remote management control according to claim 1, wherein the method comprises the following steps: the cluster analysis method comprises the following steps:
performing clustering analysis by using a k-means clustering analysis method, introducing Euclidean distance from a vector X to a vector Y in a multidimensional space as similarity measure, and calculating the Euclidean distance as follows:
Figure FDA00022666462100000310
wherein X and Y are n-dimensional vectors in accordance with
Figure FDA00022666462100000311
Get h data points
Figure FDA00022666462100000312
The same reason is according to
Figure FDA00022666462100000313
Obtaining N-h data points; dividing the data object into k clusters by using a k-mean algorithm, selecting Euclidean distance as similarity and clustering standard, completing reconstruction of network flow, and calculating the distance u from each object or data point to the clustering centeriThe following equation is obtained:
Figure FDA0002266646210000041
wherein, ckRepresents a cluster, each cluster ckAll have a center uiUsing a clustering algorithm to make the total distance of each cluster
Figure FDA0002266646210000042
The sum of the squares of (a) is minimal, the following equation holds:
Figure FDA0002266646210000043
Figure FDA0002266646210000044
Figure FDA0002266646210000045
in the above formulae (11) and (12), Jmid(Cmid) Representing a set of clusters CmidA total distance; j. the design is a squaremid(ck,mid) Representing a set of clusters ck,midA total distance; | d |i,mid-uk||2Represents a point di,midTo the clustering center ukSquare of the distance of (d); j. the design is a squarehigh(Chigh) Representing a set of clusters ChighA total distance; j. the design is a squarehigh(ck,high) Representing a set of clusters ck,highA total distance; | d |i,high-uk||2Represents a point di,highTo the clustering center ukSquare of the distance of (d);
wherein the content of the first and second substances,
Figure FDA0002266646210000046
thus, obtained by clustering algorithm
Figure FDA0002266646210000047
And
Figure FDA0002266646210000048
and performing an inverse transformation to obtain a reconstructed flow component xij,low(t)、xij,mid(t) and xij,high(t)。
7. The method for extracting service features oriented to optical cable fiber core remote management control according to claim 1, wherein the method comprises the following steps: the total flow method of the reconstruction network is as follows:
Figure FDA0002266646210000049
in the above formula (13), xij,low(t)、xij,mid(t) and xij,high(t) respectively represents the network traffic x from the source node i to the destination node j at time tijLow, medium, and high frequency time domain components in (t);
Figure FDA0002266646210000051
representing the network traffic x from the source node i to the destination node j at time tij(t) reconstructed value.
CN201911090290.2A 2019-11-08 2019-11-08 Service feature extraction method for optical cable fiber core remote management control Pending CN111654327A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911090290.2A CN111654327A (en) 2019-11-08 2019-11-08 Service feature extraction method for optical cable fiber core remote management control

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911090290.2A CN111654327A (en) 2019-11-08 2019-11-08 Service feature extraction method for optical cable fiber core remote management control

Publications (1)

Publication Number Publication Date
CN111654327A true CN111654327A (en) 2020-09-11

Family

ID=72348015

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911090290.2A Pending CN111654327A (en) 2019-11-08 2019-11-08 Service feature extraction method for optical cable fiber core remote management control

Country Status (1)

Country Link
CN (1) CN111654327A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113919694A (en) * 2021-10-11 2022-01-11 国网山西省电力公司信息通信分公司 Method and system for analyzing key service bearing state of power communication optical cable

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101666746A (en) * 2009-09-30 2010-03-10 太原市海通自动化技术有限公司 Laser induced spectrum data processing method based on wavelet analysis
CN107454073A (en) * 2017-07-31 2017-12-08 国网辽宁省电力有限公司 A kind of Network Traffic Analysis method and system
US20180041573A1 (en) * 2015-07-16 2018-02-08 Tsinghua University Method for detecting abnormal load in cloud computing oriented online service

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101666746A (en) * 2009-09-30 2010-03-10 太原市海通自动化技术有限公司 Laser induced spectrum data processing method based on wavelet analysis
US20180041573A1 (en) * 2015-07-16 2018-02-08 Tsinghua University Method for detecting abnormal load in cloud computing oriented online service
CN107454073A (en) * 2017-07-31 2017-12-08 国网辽宁省电力有限公司 A kind of Network Traffic Analysis method and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
夏会: ""基于用户行为模式特征的时间序列异常检测"", 《中国博士学位论文全文数据库》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113919694A (en) * 2021-10-11 2022-01-11 国网山西省电力公司信息通信分公司 Method and system for analyzing key service bearing state of power communication optical cable

Similar Documents

Publication Publication Date Title
Zhang et al. Intelligent acoustic-based fault diagnosis of roller bearings using a deep graph convolutional network
CN108809948B (en) Abnormal network connection detection method based on deep learning
CN106375339B (en) Attack mode detection method based on event sliding window
CN111107072B (en) Authentication graph embedding-based abnormal login behavior detection method and system
CN107493277B (en) Large data platform online anomaly detection method based on maximum information coefficient
CN110852584B (en) Risk identification method based on complex network analysis
CN110430224B (en) Communication network abnormal behavior detection method based on random block model
CN113225359A (en) Safety flow analysis system based on brain-like calculation
CN113344134A (en) Data acquisition abnormity detection method and system for low-voltage power distribution monitoring terminal
CN108282460B (en) Evidence chain generation method and device for network security event
Krishnaswamy et al. Data-driven learning to predict wan network traffic
CN117113262B (en) Network traffic identification method and system
He et al. Intelligent Fault Analysis With AIOps Technology
CN115567269A (en) Internet of things anomaly detection method and system based on federal learning and deep learning
CN111600878A (en) Low-rate denial of service attack detection method based on MAF-ADM
CN110458071B (en) DWT-DFPA-GBDT-based optical fiber vibration signal feature extraction and classification method
CN115358306A (en) Non-supervision BGP abnormity detection method and system based on graph structure
CN111654327A (en) Service feature extraction method for optical cable fiber core remote management control
CN117118849B (en) Gateway system of Internet of things and implementation method
CN112383488B (en) Content identification method suitable for encrypted and non-encrypted data streams
CN115473748A (en) DDoS attack classification detection method, device and equipment based on BiLSTM-ELM
Pan et al. Study on intelligent anti–electricity stealing early-warning technology based on convolutional neural networks
Hu et al. Classification of Abnormal Traffic in Smart Grids Based on GACNN and Data Statistical Analysis
CN116108376A (en) Monitoring system and method for preventing electricity stealing, electronic equipment and medium
CN115356599A (en) Multi-mode urban power grid fault diagnosis method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200911

RJ01 Rejection of invention patent application after publication