CN117591854A - Self-adaptive flow analysis method, system, electronic equipment and medium - Google Patents
Self-adaptive flow analysis method, system, electronic equipment and medium Download PDFInfo
- Publication number
- CN117591854A CN117591854A CN202311499229.XA CN202311499229A CN117591854A CN 117591854 A CN117591854 A CN 117591854A CN 202311499229 A CN202311499229 A CN 202311499229A CN 117591854 A CN117591854 A CN 117591854A
- Authority
- CN
- China
- Prior art keywords
- flow
- characteristic
- information table
- basic
- feature information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000005206 flow analysis Methods 0.000 title claims abstract description 46
- 238000000034 method Methods 0.000 title claims abstract description 44
- 239000011159 matrix material Substances 0.000 claims abstract description 92
- 238000004458 analytical method Methods 0.000 claims abstract description 29
- 238000007781 pre-processing Methods 0.000 claims abstract description 19
- 238000011156 evaluation Methods 0.000 claims abstract description 13
- 238000004891 communication Methods 0.000 claims description 29
- 230000003044 adaptive effect Effects 0.000 claims description 25
- 238000004140 cleaning Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 8
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000013507 mapping Methods 0.000 abstract description 8
- 238000004422 calculation algorithm Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000013461 design Methods 0.000 description 6
- 238000005259 measurement Methods 0.000 description 5
- 238000010606 normalization Methods 0.000 description 5
- 230000002093 peripheral effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000000513 principal component analysis Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000002759 z-score normalization Methods 0.000 description 2
- 206010000117 Abnormal behaviour Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- BULVZWIRKLYCBC-UHFFFAOYSA-N phorate Chemical compound CCOP(=S)(OCC)SCSCC BULVZWIRKLYCBC-UHFFFAOYSA-N 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/04—Processing captured monitoring data, e.g. for logfile generation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1416—Event detection, e.g. attack signature detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
- H04L65/1101—Session protocols
- H04L65/1104—Session initiation protocol [SIP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/40—Network security protocols
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/02—Preprocessing
- G06F2218/04—Denoising
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Environmental & Geological Engineering (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Multimedia (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention belongs to the technical field of flow mapping, and aims to provide a self-adaptive flow analysis method, a self-adaptive flow analysis system, electronic equipment and a self-adaptive flow analysis medium. The invention discloses a self-adaptive flow analysis method, which comprises the following steps: acquiring a flow record file, and carrying out feature analysis on the flow record file to obtain a flow basic feature information table; preprocessing the flow basic feature information in the flow basic feature information table to obtain a preprocessed flow feature information table; obtaining a flow characteristic association matrix according to the preprocessed flow characteristic information table; dynamically and adaptively updating a pre-stored historical flow characteristic association matrix according to the flow characteristic association matrix to obtain an updated flow characteristic association matrix, and obtaining an optimal characteristic association set according to the updated flow characteristic association matrix; and obtaining a flow evaluation result according to the optimal characteristic association set. The invention can improve the accuracy and the effectiveness of flow analysis.
Description
Technical Field
The invention belongs to the technical field of flow mapping, and particularly relates to a self-adaptive flow analysis method, a self-adaptive flow analysis system, electronic equipment and a self-adaptive flow analysis medium.
Background
VoIP (Voice over Internet Protocol, voice over IP) is a technology for voice communication using the IP protocol. VoIP can convert voice signals into digital data packets, so that the voice signals can be transmitted through the Internet or other IP networks, and the VoIP communication technology is widely applied to families, enterprises, operators and other places and provides technical support for high-quality, economical and multifunctional communication services due to the characteristics of low cost, flexible communication, high expandability, convenience in integration and management and the like.
Traffic mapping in a VoIP system environment network refers to measuring and analyzing traffic characteristics in VoIP communications, such as delay, jitter, packet loss, etc., in order to extract various performance indicators and characteristic information about call quality, network performance, and user experience. By measuring, analyzing and evaluating actual communication traffic, such as security monitoring, quality of service (QoS) evaluation, data compression, traffic classification, anomaly detection and traffic analysis, voIP network security can be effectively enhanced and VoIP quality of service optimized. In the application process, firstly, a network administrator can find out bottlenecks and problems existing in a network through flow measurement and analysis, and then take corresponding optimization measures, such as adjusting network parameters, increasing bandwidth, improving network topology and the like, so as to improve the quality and stability of VoIP call; secondly, through measurement and analysis of the call flow of the user, the service side can know the experience conditions of the user in different network environments, so that the communication experience of the user is optimized, and the user satisfaction is improved; in addition, the periodic VoIP network traffic analysis can identify and monitor abnormal activities and potential security threats in the network, prevent network abnormal behaviors and intrusion attempts, protect the network from potential attacks and abuse, and is beneficial to improving the security reliability of the VoIP network.
However, in using the prior art, the inventors found that there are at least the following problems in the prior art:
in the prior art, when traffic mapping is performed in a VoIP system environment network, the problem of low accuracy and low effectiveness exists. Specifically, in the prior art, most of flow measurement and drawing work is usually implemented through a network flow monitoring tool, a flow analysis system is constructed or an active measurement technology, and meanwhile, flow statistics and analysis software are used for implementation, so that when flow mapping is performed in a VoIP soft switch system, mapping is performed in the prior art due to lack of targeted flow characteristics, and when a high throughput call flow scene is performed, the problems of weak mapping capability, low mapping accuracy and low effectiveness exist in the prior art.
Disclosure of Invention
The invention aims to solve the technical problems at least to a certain extent, and provides a self-adaptive flow analysis method, a self-adaptive flow analysis system, electronic equipment and a medium.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
in a first aspect, the present invention provides a method for adaptive traffic analysis, comprising:
acquiring a flow record file, and carrying out feature analysis on the flow record file to obtain a flow basic feature information table; the flow basic characteristic information table comprises flow basic characteristic information in each time period;
preprocessing the flow basic feature information in the flow basic feature information table to obtain a preprocessed flow feature information table;
obtaining a flow characteristic association matrix according to the preprocessed flow characteristic information table;
dynamically and adaptively updating a pre-stored historical flow characteristic association matrix according to the flow characteristic association matrix to obtain an updated flow characteristic association matrix, and obtaining an optimal characteristic association set according to the updated flow characteristic association matrix;
and obtaining a flow evaluation result according to the optimal characteristic association set.
The invention can improve the accuracy and the effectiveness of flow analysis. Specifically, in the flow analysis and flow characteristic extraction process, the flow characteristic association matrix is obtained according to the preprocessed flow characteristic information table, mutual information is introduced as an index for measuring the correlation between the preprocessed flow characteristics, and mutual information analysis can help to identify and quantify the interaction relationship between the characteristics, so that key information and modes hidden in flow data can be found conveniently. Meanwhile, in the process of obtaining the optimal feature association set, the method and the device also dynamically and adaptively update the pre-stored historical flow feature association matrix according to the flow feature association matrix, so as to obtain the optimal feature association set, and further obtain a flow evaluation result according to the optimal feature association set. In the process, the invention establishes a self-adaptive feature extraction process, can automatically adjust the weights of the features according to actual conditions, improves the attention degree of key features, reduces the influence on irrelevant features, reduces the feature range to be evaluated, further can improve the accuracy and the effectiveness of flow analysis, and can automatically adapt to different flow scenes and extract the most representative features. Based on the above, the present invention can optimize the network performance index of the VoIP system environment by adaptively measuring and analyzing the flow characteristics in the VoIP system environment network in the terminal soft switch network, and explore and analyze the network problems possibly occurring in the communication of the VoIP system environment, so as to improve the security and reliability of the network, and provide a new solution for further improvement of network security and network services.
In one possible design, the feature analysis is performed on the flow record file to obtain a flow basic feature information table, which includes:
performing segmentation processing on the flow record file to obtain a plurality of flow basic fields;
analyzing the flow basic fields to obtain a full flow protocol field;
analyzing the full-flow protocol field to obtain a SIP protocol header field;
and obtaining a flow basic characteristic information table according to the full flow protocol field and the SIP protocol header field.
In one possible design, preprocessing the flow basic feature information in the flow basic feature information table to obtain a preprocessed flow feature information table, including:
data cleaning is carried out on the flow basic feature information in the flow basic feature information table, and flow basic feature information after cleaning is obtained;
denoising the cleaned flow basic feature information in the flow basic feature information table to obtain denoised flow basic feature information;
and normalizing the denoised flow basic characteristic information in the flow basic characteristic information table to obtain a preprocessed flow characteristic information table.
In one possible design, any flow characteristic association information in the flow characteristic association matrix is:
wherein p (x) is an edge probability density function corresponding to any one of the preprocessed flow basic feature information x in the preprocessed flow feature information table, p (y) is an edge probability density function corresponding to preprocessed flow basic feature information y located in a time period adjacent to the preprocessed flow basic feature information x in the preprocessed flow feature information table, and p (x, y) is a joint probability density function corresponding to the preprocessed flow basic feature information x and the preprocessed flow basic feature information y.
In one possible design, the updated traffic feature correlation matrix is:
wherein, C is the flow characteristic incidence matrix,and w is a preset updating coefficient for a prestored historical flow characteristic association matrix.
In one possible design, obtaining the optimal feature association set according to the updated flow feature association matrix includes:
obtaining a covariance matrix of the updated flow characteristic association matrix by a PCA method;
selecting the first k principal components in the covariance matrix according to the magnitude of the eigenvalue in the covariance matrix; wherein k is a natural number greater than 1;
and projecting the first k principal components into the new feature space to obtain an optimal feature association set.
In one possible design, the covariance matrix is:
wherein X is i For updated flow characteristic association matrixThe i-th updated flow characteristic association information in (1), n is the total data amount in the updated flow characteristic association matrix, and T is the transposed symbol.
In a second aspect, the present invention provides an adaptive flow analysis system for implementing an adaptive flow analysis method as described in any one of the preceding claims; the adaptive flow analysis system includes:
the flow characteristic analysis module is used for acquiring a flow record file, and carrying out characteristic analysis on the flow record file to obtain a flow basic characteristic information table; the flow basic characteristic information table comprises flow basic characteristic information in each time period;
the data preprocessing module is in communication connection with the flow characteristic analysis module and is used for preprocessing the flow basic characteristic information in the flow basic characteristic information table to obtain a preprocessed flow characteristic information table;
the relevance calculating module is in communication connection with the data preprocessing module and is used for obtaining a flow characteristic relevance matrix according to the preprocessed flow characteristic information table;
the self-adaptive feature extraction module is in communication connection with the relevance calculation module and is used for carrying out dynamic self-adaptive update on a pre-stored historical flow feature relevance matrix according to the flow feature relevance matrix to obtain an updated flow feature relevance matrix, and obtaining an optimal feature relevance set according to the updated flow feature relevance matrix;
and the characteristic analysis module is in communication connection with the self-adaptive characteristic extraction module and is used for obtaining a flow evaluation result according to the optimal characteristic association set.
In a third aspect, the present invention provides an electronic device, comprising:
a memory for storing computer program instructions; the method comprises the steps of,
a processor for executing the computer program instructions to perform the operations of the adaptive flow analysis method as claimed in any one of the preceding claims.
In a fourth aspect, the present invention provides a computer readable storage medium storing computer program instructions readable by a computer, the computer program instructions being configured to perform the operations of the adaptive flow analysis method as claimed in any one of the preceding claims when run.
Drawings
FIG. 1 is a flow chart of an adaptive flow analysis method in an embodiment;
FIG. 2 is a block diagram of an adaptive flow analysis system in an embodiment;
fig. 3 is a block diagram of an electronic device in an embodiment.
Detailed Description
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be briefly described below with reference to the accompanying drawings and the description of the embodiments or the prior art, and it is obvious that the following description of the structure of the drawings is only some embodiments of the present invention, and other drawings can be obtained according to these drawings without inventive effort to a person skilled in the art. It should be noted that the description of these examples is for aiding in understanding the present invention, but is not intended to limit the present invention.
Example 1:
the embodiment discloses an adaptive flow analysis method, which can be executed by a computer device or a virtual machine with a certain computing resource, for example, an electronic device such as a personal computer, a smart phone, a personal digital assistant or a wearable device, or a virtual machine.
As shown in fig. 1, an adaptive flow analysis method may include, but is not limited to, the following steps:
s1, acquiring a flow record file by carrying out full flow capture on flow in a VoIP network, and carrying out feature analysis on the flow record file to obtain a flow basic feature information table; the flow basic characteristic information table comprises flow basic characteristic information in each time period.
Specifically, the flow record file is subjected to feature analysis to obtain a flow basic feature information table, which comprises the following steps:
s101, carrying out segmentation processing on the flow record file to obtain a plurality of flow basic fields;
s102, analyzing the flow basic fields to obtain a full flow protocol field;
s103, analyzing the full-flow protocol field to obtain a SIP (Session initialization Protocol, session initiation protocol) protocol header field;
s104, obtaining a flow basic characteristic information table according to the full flow protocol field and the SIP protocol header field.
In this embodiment, the time stamp of the traffic data is used to obtain the parameter T at the preset time interval default Segmenting the flow record file to obtain basic characteristics of flow information in each time period, namely a plurality of flow basic fields; the embodiment can also perform segmented collection and preliminary analysis on the captured multiple basic flow fields according to the configured time interval parameters so as to obtain a final basic flow characteristic information table.
S2, preprocessing the flow basic feature information in the flow basic feature information table to obtain a preprocessed flow feature information table; when the flow basic feature information table is preprocessed, operations such as data cleaning, noise removal, standardization and the like are performed on the flow basic feature information in the flow basic feature information table, so that accuracy, reliability and consistency of analysis data are ensured.
In this embodiment, preprocessing is performed on the flow basic feature information in the flow basic feature information table to obtain a preprocessed flow feature information table, which includes:
s201, data cleaning is carried out on the flow basic feature information in the flow basic feature information table, and flow basic feature information after cleaning is obtained; specifically, the data cleansing may remove invalid or erroneous data in the traffic base characteristic information to improve data quality.
S202, denoising the cleaned flow basic feature information in the flow basic feature information table to obtain denoised flow basic feature information;
specifically, in this embodiment, the denoising process is used to filter offset information that occasionally appears in the flow basic feature information table, so as to reduce the number of noise points in the flow basic feature information, and in this embodiment, a Z-Score normalization algorithm is adopted, and the flow basic feature information after cleaning in the flow basic feature information table is normalized by adopting the following formula:
wherein x is i ' is the flow basic characteristic information after cleaning in the flow basic characteristic information table, x i For the flow basic characteristic information x after cleaning i ' normalized data after matching, μ is an average value of the flow basic feature information after cleaning in the flow basic feature information table, σ is a standard deviation of the flow basic feature information after cleaning in the flow basic feature information table;
and then adopting a median filtering algorithm in the following formula to further remove noise in the information:
Y i ＝median(x i-n+1 ,x i-n+2 ,…,x i )；
wherein Y is i The flow basic characteristic information after denoising in the flow basic characteristic information table is x i-n+1 ,x i-n+2 ,…,x i For normalized data, n is the filter window size, which is typically a positive integer greater than 3.
S203, normalizing the denoised flow basic characteristic information in the flow basic characteristic information table to obtain a preprocessed flow characteristic information table.
Specifically, in this embodiment, the normalization process may normalize the flow basic feature information in the flow basic feature information table, scale all the information features to the same range, and in this embodiment, the normalization algorithm is usually [0,1] or [ -1,1] according to the normalization algorithm adopted, and usually the Min-Max normalization algorithm or the Z-score normalization algorithm is adopted. The calculation formula of the Min-Max normalization algorithm is as follows:
wherein Y is norm Is Y with i Preprocessing flow characteristic information in a matched preprocessing flow characteristic information table, Y min For the minimum value in the de-noised flow basic feature information in the flow basic feature information table, Y max And the maximum value in the de-noised flow basic characteristic information in the flow basic characteristic information table.
S3, obtaining a flow characteristic association matrix according to the preprocessed flow characteristic information table; obtaining a flow characteristic association matrix according to the preprocessed flow characteristic information table, wherein the flow characteristic association matrix comprises the following steps: sequentially obtaining the association degree between the preprocessed flow basic feature information in the adjacent time period in the preprocessed flow feature information table to obtain a plurality of flow feature association information; obtaining a flow characteristic association matrix according to the plurality of flow characteristic association information; in this embodiment, by introducing a measurement standard of mutual information, the correlation degree between the preprocessed flow basic feature information is quantized two by two, and after the correlation calculation, a flow feature correlation matrix can be obtained for evaluating the key degree of the feature in the next step.
Specifically, in this embodiment, any one of the flow characteristic association information in the flow characteristic association matrix is:
wherein p (x) is an edge probability density function corresponding to any one of the preprocessed flow basic feature information x in the preprocessed flow feature information table, p (y) is an edge probability density function corresponding to preprocessed flow basic feature information y located in a time period adjacent to the preprocessed flow basic feature information x in the preprocessed flow feature information table, and p (x, y) is a joint probability density function corresponding to the preprocessed flow basic feature information x and the preprocessed flow basic feature information y.
S4, carrying out dynamic self-adaptive updating on a pre-stored historical flow characteristic incidence matrix (namely an updated flow characteristic incidence matrix obtained in the previous period) according to the flow characteristic incidence matrix to obtain an updated flow characteristic incidence matrix, and obtaining an optimal characteristic incidence set according to the updated flow characteristic incidence matrix.
In this embodiment, the updated flow characteristic association matrix is:
wherein, C is the flow characteristic incidence matrix,and w is a preset updating coefficient for a prestored historical flow characteristic association matrix.
In this embodiment, a pre-stored historical flow characteristic association matrix is dynamically adaptively updated in an exponentially weighted moving average manner, where an update coefficient w may be obtained by user-defined configuration and used to control the update speed of the matrix.
Specifically, in the present embodiment, an example calculation formula of the update coefficient w is as follows:
w＝α×CosSim+β×EucilideanDist；
wherein CosSim is a pre-stored historical flow characteristic association matrixCosine similarity between the flow characteristic correlation matrix C and the Euciliideanent dist is a pre-stored historical flow characteristic correlation matrix +.>And Euclidean distance between the flow characteristic incidence matrixes C, wherein alpha is a preset first weight, and beta is a preset second weight. In the present embodiment, α and β are set in consideration of the simultaneous use of the cosine similarity value and the euclidean distance, and are divided into three cases: if only cosine similarity is used, α is 1 and β is 0; if only Euclidean distance is used, then alpha is 0 and beta is 1; if both are used, the user is required to customize the weights of the two distance methods and then assign alpha and beta.
In this embodiment, the update coefficient w formula input by the user is parsed during initialization, and the corresponding update coefficient w can be finally obtained by verifying the validity of the formula and parsing the mathematical expression. At each timeT is set up default And when the period is over, updating the flow characteristic association matrix according to a calculation formula of the update coefficient w.
In this embodiment, obtaining the optimal feature association set according to the updated traffic feature association matrix includes:
s401, obtaining a covariance matrix of the updated flow characteristic association matrix by a PCA (Principal Component Analysis ) method;
in this embodiment, the covariance matrix is:
wherein X is i For updated flow characteristic association matrixThe i-th updated flow characteristic association information in (1), n is the total data amount in the updated flow characteristic association matrix, and T is the transposed symbol.
S402, selecting the first k main components in the covariance matrix according to the magnitude of the eigenvalue in the covariance matrix; wherein k is a natural number greater than 1;
s403, projecting the first k principal components into a new feature space to obtain an optimal feature association set.
Specifically, for a dataset having n features, the covariance matrix is an n×n matrix representing the covariance between the features. In this embodiment, the eigenvalues and eigenvectors of the covariance matrix can be solved by a linear algebra method. The feature vector represents a direction in the feature space, and the feature value represents a degree of importance of the feature vector in the direction. After the feature value calculation is completed, k can be ordered and customized according to the size of the feature value, and the first k principal components of the feature value are selected to obtain an optimal feature association set according to the k principal components, so that the reduced features are represented.
In this embodiment, the importance of the feature is adjusted by dynamically updating the weight, and the pre-stored historical flow feature association matrix is dynamically and adaptively updated by adaptively giving data weights with different importance degrees through an analysis method of exponentially weighted moving average; and selecting the first k principal components (specified by a user) according to the feature sizes in the latest feature association set to obtain an optimal feature association set, thereby being beneficial to narrowing the feature research range and further more efficiently carrying out subsequent network data analysis work.
S5, obtaining a flow evaluation result according to the optimal feature association set. In this embodiment, the SVM classifier obtained through the manual experience training of the network administrator performs classification evaluation on the optimal feature association set, so as to obtain a flow evaluation result. In this embodiment, the flow evaluation results include indexes of QoS (Quality of Service ) of the traditional Chinese medicine network such as delay, packet loss rate, jitter and broadband, and the indexes can be sequentially classified into five-level classification results for evaluating the QoS of the network from good to bad.
The embodiment can improve the accuracy and the effectiveness of flow analysis. Specifically, in the flow analysis and flow feature extraction process, the flow feature correlation matrix is obtained according to the preprocessed flow feature information table, mutual information is introduced as an index for measuring the correlation between the preprocessed flow features, and mutual information analysis can help to identify and quantify the interaction relationship between the features, so that key information and modes hidden in flow data can be found conveniently. Meanwhile, in the process of obtaining the optimal feature association set, the embodiment also carries out dynamic self-adaptive update on the pre-stored historical flow feature association matrix according to the flow feature association matrix, so as to obtain the optimal feature association set, and further obtain a flow evaluation result according to the optimal feature association set. In this process, the embodiment establishes a self-adaptive feature extraction process, which can automatically adjust the weights of the features according to actual conditions, improve the attention to key features, reduce the influence on irrelevant features, reduce the feature range to be evaluated, further improve the accuracy and effectiveness of flow analysis, and automatically adapt to different flow scenes and extract the most representative features. Based on the above, the present embodiment searches and analyzes the network problems possibly occurring in the communication of the VoIP system environment by adaptively measuring and analyzing the flow characteristics in the VoIP system environment network in the terminal soft switch network, so as to optimize the network performance index of the VoIP system environment, so as to improve the security and reliability of the network, and provide a new solution for further improvement of network security and network service.
Example 2:
the embodiment discloses a self-adaptive flow analysis system, which is used for realizing the self-adaptive flow analysis method in the embodiment 1; as shown in fig. 2, the adaptive flow analysis system includes:
the flow characteristic analysis module is used for acquiring a flow record file, and carrying out characteristic analysis on the flow record file to obtain a flow basic characteristic information table; the flow basic characteristic information table comprises flow basic characteristic information in each time period;
the data preprocessing module is in communication connection with the flow characteristic analysis module and is used for preprocessing the flow basic characteristic information in the flow basic characteristic information table to obtain a preprocessed flow characteristic information table;
the relevance calculating module is in communication connection with the data preprocessing module and is used for obtaining a flow characteristic relevance matrix according to the preprocessed flow characteristic information table;
the self-adaptive feature extraction module is in communication connection with the relevance calculation module and is used for carrying out dynamic self-adaptive update on a pre-stored historical flow feature relevance matrix according to the flow feature relevance matrix to obtain an updated flow feature relevance matrix, and obtaining an optimal feature relevance set according to the updated flow feature relevance matrix;
and the characteristic analysis module is in communication connection with the self-adaptive characteristic extraction module and is used for obtaining a flow evaluation result according to the optimal characteristic association set.
Example 3:
on the basis of embodiment 1 or 2, this embodiment discloses an electronic device, which may be a smart phone, a tablet computer, a notebook computer, a desktop computer, or the like. The electronic device may be referred to as a user terminal, a portable terminal, a desktop terminal, etc., as shown in fig. 3, the electronic device includes:
a memory for storing computer program instructions; the method comprises the steps of,
a processor configured to execute the computer program instructions to perform the operations of the adaptive flow analysis method according to any one of embodiment 1.
In particular, processor 301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 301 may be implemented in at least one hardware form of DSP (Digital Signal Processing ), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array ). The processor 301 may also include a main processor, which is a processor for processing data in an awake state, also called a CPU (Central Processing Unit ), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 301 may be integrated with a GPU (Graphics Processing Unit, image processor) for taking care of rendering and drawing of content that the display screen is required to display.
Memory 302 may include one or more computer-readable storage media, which may be non-transitory. Memory 302 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 302 is used to store at least one instruction for execution by processor 301 to implement the adaptive flow analysis method provided by embodiment 1 herein.
In some embodiments, the terminal may further optionally include: a communication interface 303, and at least one peripheral device. The processor 301, the memory 302 and the communication interface 303 may be connected by a bus or signal lines. The respective peripheral devices may be connected to the communication interface 303 through a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 304, a display screen 305, and a power supply 306.
The communication interface 303 may be used to connect at least one peripheral device associated with an I/O (Input/Output) to the processor 301 and the memory 302. In some embodiments, processor 301, memory 302, and communication interface 303 are integrated on the same chip or circuit board; in some other embodiments, either or both of the processor 301, the memory 302, and the communication interface 303 may be implemented on separate chips or circuit boards, which is not limited in this embodiment.
The Radio Frequency circuit 304 is configured to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuitry 304 communicates with a communication network and other communication devices via electromagnetic signals.
The display screen 305 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof.
The power supply 306 is used to power the various components in the electronic device.
Example 4:
on the basis of any one of embodiments 1 to 3, this embodiment discloses a computer-readable storage medium for storing computer-readable computer program instructions configured to perform the operations of the adaptive flow analysis method as described in embodiment 1 when run.
It will be apparent to those skilled in the art that the modules or steps of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, or they may alternatively be implemented in program code executable by computing devices, such that they may be stored in a memory device for execution by the computing devices, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps within them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
Finally, it should be noted that the above embodiments are merely illustrative of the technical solution of the present invention, and not limiting thereof; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some of the technical features thereof can be replaced by equivalents. Such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. The self-adaptive flow analysis method is characterized in that: comprising the following steps:
acquiring a flow record file, and carrying out feature analysis on the flow record file to obtain a flow basic feature information table; the flow basic characteristic information table comprises flow basic characteristic information in each time period;
preprocessing the flow basic feature information in the flow basic feature information table to obtain a preprocessed flow feature information table;
obtaining a flow characteristic association matrix according to the preprocessed flow characteristic information table;
dynamically and adaptively updating a pre-stored historical flow characteristic association matrix according to the flow characteristic association matrix to obtain an updated flow characteristic association matrix, and obtaining an optimal characteristic association set according to the updated flow characteristic association matrix;
and obtaining a flow evaluation result according to the optimal characteristic association set.
2. An adaptive flow analysis method according to claim 1, wherein: performing feature analysis on the flow record file to obtain a flow basic feature information table, wherein the method comprises the following steps:
performing segmentation processing on the flow record file to obtain a plurality of flow basic fields;
analyzing the flow basic fields to obtain a full flow protocol field;
analyzing the full-flow protocol field to obtain a SIP protocol header field;
and obtaining a flow basic characteristic information table according to the full flow protocol field and the SIP protocol header field.
3. An adaptive flow analysis method according to claim 1, wherein: preprocessing the flow basic feature information in the flow basic feature information table to obtain a preprocessed flow feature information table, wherein the preprocessing comprises the following steps:
data cleaning is carried out on the flow basic feature information in the flow basic feature information table, and flow basic feature information after cleaning is obtained;
denoising the cleaned flow basic feature information in the flow basic feature information table to obtain denoised flow basic feature information;
and normalizing the denoised flow basic characteristic information in the flow basic characteristic information table to obtain a preprocessed flow characteristic information table.
4. An adaptive flow analysis method according to claim 1, wherein: any flow characteristic association information in the flow characteristic association matrix is as follows:
wherein p (x) is an edge probability density function corresponding to any one of the preprocessed flow basic feature information x in the preprocessed flow feature information table, p (y) is an edge probability density function corresponding to preprocessed flow basic feature information y located in a time period adjacent to the preprocessed flow basic feature information x in the preprocessed flow feature information table, and p (x, y) is a joint probability density function corresponding to the preprocessed flow basic feature information x and the preprocessed flow basic feature information y.
5. An adaptive flow analysis method according to claim 1, wherein: the updated flow characteristic association matrix is as follows:
wherein, C is the flow characteristic incidence matrix,and w is a preset updating coefficient for a prestored historical flow characteristic association matrix.
6. An adaptive flow analysis method according to claim 1, wherein: obtaining an optimal feature association set according to the updated flow feature association matrix, including:
obtaining a covariance matrix of the updated flow characteristic association matrix by a PCA method;
selecting the first k principal components in the covariance matrix according to the magnitude of the eigenvalue in the covariance matrix; wherein k is a natural number greater than 1;
and projecting the first k principal components into the new feature space to obtain an optimal feature association set.
7. The adaptive flow analysis method of claim 6, wherein: the covariance matrix is:
wherein X is i For updated flow characteristic association matrixI-th updated flow feature in (b)And the association information, n is the total data quantity in the updated flow characteristic association matrix, and T is a transposed symbol.
8. An adaptive flow analysis system, characterized by: for implementing the adaptive flow analysis method according to any one of claims 1 to 7; the adaptive flow analysis system includes:
the flow characteristic analysis module is used for acquiring a flow record file, and carrying out characteristic analysis on the flow record file to obtain a flow basic characteristic information table; the flow basic characteristic information table comprises flow basic characteristic information in each time period;
the data preprocessing module is in communication connection with the flow characteristic analysis module and is used for preprocessing the flow basic characteristic information in the flow basic characteristic information table to obtain a preprocessed flow characteristic information table;
the relevance calculating module is in communication connection with the data preprocessing module and is used for obtaining a flow characteristic relevance matrix according to the preprocessed flow characteristic information table;
the self-adaptive feature extraction module is in communication connection with the relevance calculation module and is used for carrying out dynamic self-adaptive update on a pre-stored historical flow feature relevance matrix according to the flow feature relevance matrix to obtain an updated flow feature relevance matrix, and obtaining an optimal feature relevance set according to the updated flow feature relevance matrix;
and the characteristic analysis module is in communication connection with the self-adaptive characteristic extraction module and is used for obtaining a flow evaluation result according to the optimal characteristic association set.
9. An electronic device, characterized in that: comprising the following steps:
a memory for storing computer program instructions; the method comprises the steps of,
a processor for executing the computer program instructions to perform the operations of the adaptive flow analysis method of any one of claims 1 to 7.
10. A computer readable storage medium storing computer program instructions readable by a computer, characterized by: the computer program instructions are configured to perform the operations of the adaptive flow analysis method of any one of claims 1 to 7 when run.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311499229.XA CN117591854A (en) | 2023-11-10 | 2023-11-10 | Self-adaptive flow analysis method, system, electronic equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311499229.XA CN117591854A (en) | 2023-11-10 | 2023-11-10 | Self-adaptive flow analysis method, system, electronic equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117591854A true CN117591854A (en) | 2024-02-23 |
Family
ID=89914278
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311499229.XA Withdrawn CN117591854A (en) | 2023-11-10 | 2023-11-10 | Self-adaptive flow analysis method, system, electronic equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117591854A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20230275921A1 (en) * | 2020-12-30 | 2023-08-31 | T-Mobile Usa, Inc. | Cybersecurity system for services of interworking wireless telecommunications networks |
-
2023
- 2023-11-10 CN CN202311499229.XA patent/CN117591854A/en not_active Withdrawn
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20230275921A1 (en) * | 2020-12-30 | 2023-08-31 | T-Mobile Usa, Inc. | Cybersecurity system for services of interworking wireless telecommunications networks |
| US12113825B2 (en) * | 2020-12-30 | 2024-10-08 | T-Mobile Usa, Inc. | Cybersecurity system for services of interworking wireless telecommunications networks |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2019169928A1 (en) | Traffic detection method and traffic detection device | |
| US9961157B2 (en) | Adaptive compression management for web services | |
| CN117591854A (en) | Self-adaptive flow analysis method, system, electronic equipment and medium | |
| WO2019062405A1 (en) | Application program processing method and apparatus, storage medium, and electronic device | |
| CN114448830B (en) | Equipment detection system and method | |
| US20230124389A1 (en) | Model Determination Method and Electronic Device | |
| WO2024098699A1 (en) | Entity object thread detection method and apparatus, device, and storage medium | |
| WO2019062404A1 (en) | Application program processing method and apparatus, storage medium, and electronic device | |
| WO2024208075A1 (en) | Updating method, apparatus and device for behavior data prediction model, and storage medium | |
| US11003513B2 (en) | Adaptive event aggregation | |
| CN117201340A (en) | Message feature recognition method, device, equipment and storage medium | |
| CN114724144B (en) | Text recognition method, training device, training equipment and training medium for model | |
| CN115296917A (en) | Asset exposure surface information acquisition method, device, equipment and storage medium | |
| CN115312042A (en) | Method, apparatus, device and storage medium for processing audio | |
| CN112863548A (en) | Method for training audio detection model, audio detection method and device thereof | |
| CN109272005A (en) | A kind of generation method of recognition rule, device and deep packet inspection device | |
| CN112380406B (en) | Real-time network traffic classification method based on crawler technology | |
| CN116405330B (en) | Network abnormal traffic identification method, device and equipment based on transfer learning | |
| CN118378304B (en) | Data security management method, system, equipment and product based on deep learning | |
| CN115378746B (en) | Network intrusion detection rule generation method, device, equipment and storage medium | |
| CN115102728B (en) | Scanner identification method, device, equipment and medium for information security | |
| CN115019837B (en) | Voice data processing method and device, storage medium and electronic equipment | |
| US20240078189A1 (en) | Multi-tenant distributed cache architecture for object access and expiration and systems and methods for customized computer vision-oriented convolutional neural networks | |
| CN113408664B (en) | Training method, classification method, device, electronic equipment and storage medium | |
| CN117544417A (en) | Multi-model-based Cobalt Strike attack identification method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20240223 |
|
| WW01 | Invention patent application withdrawn after publication |