CN116996393A - Method, device, equipment and medium for determining correlation between KQI and KPI in network communication - Google Patents
Method, device, equipment and medium for determining correlation between KQI and KPI in network communication Download PDFInfo
- Publication number
- CN116996393A CN116996393A CN202310821505.3A CN202310821505A CN116996393A CN 116996393 A CN116996393 A CN 116996393A CN 202310821505 A CN202310821505 A CN 202310821505A CN 116996393 A CN116996393 A CN 116996393A
- Authority
- CN
- China
- Prior art keywords
- data
- kpi
- kqi
- network
- standardized
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000004891 communication Methods 0.000 title claims abstract description 28
- 238000012549 training Methods 0.000 claims abstract description 91
- 238000013507 mapping Methods 0.000 claims abstract description 44
- 238000003860 storage Methods 0.000 claims abstract description 10
- 238000013499 data model Methods 0.000 claims description 43
- 238000012545 processing Methods 0.000 claims description 43
- 230000006870 function Effects 0.000 claims description 32
- 238000012360 testing method Methods 0.000 claims description 19
- 230000005540 biological transmission Effects 0.000 claims description 12
- 230000003213 activating effect Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 7
- 230000015654 memory Effects 0.000 claims description 7
- 239000000523 sample Substances 0.000 description 57
- 230000000875 corresponding effect Effects 0.000 description 26
- 239000010410 layer Substances 0.000 description 26
- 241000282326 Felis catus Species 0.000 description 22
- 238000010586 diagram Methods 0.000 description 16
- 238000013528 artificial neural network Methods 0.000 description 14
- 238000004422 calculation algorithm Methods 0.000 description 14
- 238000010606 normalization Methods 0.000 description 12
- 230000003287 optical effect Effects 0.000 description 11
- 230000008569 process Effects 0.000 description 9
- 238000001994 activation Methods 0.000 description 8
- 230000004913 activation Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 230000008859 change Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 230000007774 longterm Effects 0.000 description 3
- 238000004220 aggregation Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 2
- 238000004140 cleaning Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000009472 formulation Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000011144 upstream manufacturing Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000012792 core layer Substances 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000003064 k means clustering Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000002759 z-score normalization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/27—Regression, e.g. linear or logistic regression
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0894—Policy-based network configuration management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/142—Network analysis or design using statistical or mathematical methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/147—Network analysis or design for predicting network behaviour
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Mathematical Physics (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Probability & Statistics with Applications (AREA)
- Algebra (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the application discloses a method, a device, electronic equipment and a storage medium for determining the correlation between KQI and KPI in network communication, wherein the method comprises the following steps: acquiring KPI data, and acquiring KQI data, wherein the KPI data and the KQI data respectively comprise associated fields; establishing an association relation between KPI data and KQI data according to the association field; selecting KPI index fields with data values meeting fluctuation conditions from KPI data with association relations, and taking the data values of the selected KPI index fields and the mutually associated KQI data as initial samples; carrying out standardization treatment on the initial sample to obtain a training sample; training the EQL network according to the training sample, and determining the mapping relation between the standardized KQI data and the standardized KPI data according to the network weights of the EQL network and the EQL network after the training is completed. The embodiment of the application can improve the accuracy of the mapping relation between the KQI and the KPI.
Description
Technical Field
The embodiment of the application relates to the technical field of communication, in particular to a method and a device for determining the correlation between KQI and KPI in network communication, electronic equipment and a storage medium.
Background
With the rapid development of internet services carried in a communication network, the requirements of users on network service quality are higher and higher, while the problem that KPIs (Key Performance Indicators, key performance indexes) of communication equipment cannot reflect real service perception of users is increasingly prominent, operators gradually shift from traditional KPIs to KQIs (Key Quality Indicators, key quality indexes) close to users, so that the change trend of the user network is perceived in advance, preferential network resources are reasonably allocated, planned optimization of network expansion is made, and the method is important for solving network congestion and providing higher-quality user services.
In the prior art, a curve fitting method and a correlation coefficient method are generally adopted to determine the correlation between KQI and KPI. Curve fitting (fit theory), commonly known as a pull curve, is a method of substituting existing data into a numerical expression, and obtaining a plurality of discrete data by methods such as sampling and experiment, according to which a continuous function (i.e., curve) or a more dense discrete equation is expected to be matched with the known data, which is called fitting. And (3) finding out the relation between the KPI and the KQI index through the fitted mapping model, and realizing the mapping of the KPI and the KQI. The method needs to obtain the KQI in a mode of deploying the probe close to the user side equipment, the deployment cost is high, the real user perception of the user cannot be accurately obtained, the conditions of 'good KPI, poor KQI' and 'good KQI' are generated, but the field network condition of 'poor KPI' is generated, namely the accurate mapping relation of the two conditions cannot be provided.
Disclosure of Invention
The embodiment of the application provides a method, a device, electronic equipment and a storage medium for determining the correlation between KQI and KPI in network communication, which are beneficial to improving the accuracy of the mapping relationship between KQI and KPI.
In order to solve the above problems, in a first aspect, an embodiment of the present application provides a method for determining correlation between KQI and KPI in network communications, including:
acquiring Key Performance Index (KPI) data, and acquiring Key Quality Index (KQI) data, wherein the KPI data and the KQI data respectively comprise associated fields, and the KPI data also comprise data values of a plurality of KPI index fields;
establishing an association relationship between the KPI data and the KQI data according to the association field;
selecting KPI index fields with data values meeting fluctuation conditions from KPI data with the association relation, and taking the data values of the selected KPI index fields and KQI data with the association relation as initial samples;
carrying out standardization processing on the initial sample to obtain a training sample, wherein the training sample comprises standardized KPI data and standardized KQI data;
and training an equation learner (EQL) network according to the training sample, and determining the mapping relation between the standardized KQI data and the standardized KPI data according to the EQL network and the network weight of the EQL network after training.
In a second aspect, an embodiment of the present application provides a device for determining correlation between KQI and KPI in network communications, including:
the system comprises an index data acquisition module, a Key Performance Index (KPI) data acquisition module and a Key Quality Index (KQI) data acquisition module, wherein the KPI data and the KQI data respectively comprise associated fields, and the KPI data also comprise data values of a plurality of KPI index fields;
the association relation establishing module is used for establishing association relation between the KPI data and the KQI data according to the association field;
the initial sample determining module is used for selecting KPI index fields with data values meeting fluctuation conditions from KPI data with the association relation, and taking the data values of the selected KPI index fields and KQI data with the association relation as initial samples;
the standardized processing module is used for carrying out standardized processing on the initial sample to obtain a training sample, wherein the training sample comprises standardized KPI data and standardized KQI data;
and the mapping relation determining module is used for training the equation learner (EQL) network according to the training sample, and determining the mapping relation between the standardized KQI data and the standardized KPI data according to the EQL network and the network weight of the EQL network after the training is completed.
In a third aspect, an embodiment of the present application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements a method for determining KQI and KPI correlation in network communication according to the embodiment of the present application when the processor executes the computer program.
In a fourth aspect, an embodiment of the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs a method for determining correlation between KQI and KPI in network communications as disclosed in the embodiment of the present application.
According to the method, the device, the electronic equipment and the storage medium for determining the correlation between the KQI and the KPI in the network communication, the correlation between the KPI data and the KQI data is established according to the correlation field, the KPI index field with the data value meeting the fluctuation condition is selected from the KPI data with the correlation, the data value of the selected KPI index field and the associated KQI data are used as initial samples, the initial samples are subjected to standardized processing to obtain training samples, the EQL network is trained according to the training samples, the mapping relation between the standardized KQI data and the standardized KPI data is determined according to the network weights of the EQL network and the EQL network after the training is completed, the KPI data and the KQI data are obtained through long-term transverse modeling, the KPI data and the network layer KPI data of a user are subjected to transverse modeling according to time sequences, the characteristic analysis training is carried out on the KQI time and the KPI time by adopting a KQI neural network algorithm, and finally the relationship between the KQI data and the KPI data is obtained, namely, the mapping relation between the KQI data and the KPI is well-defined, the effect is improved, and the accuracy of the perceived relation between the KPI and the KPI is improved, and the user is reasonably perceived, and the accuracy is improved, and the accuracy is achieved, and the accuracy is improved, the accuracy is achieved, and the accuracy is realized, and the user is improved, and the accuracy is improved, and the a user is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings used in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a method for determining correlation between KQI and KPI in network communication according to an embodiment of the present application;
FIG. 2 is an exemplary diagram of a trained EQL network in an embodiment of the application;
FIG. 3 is an exemplary diagram of an EQL network employing neural networks for symbolic regression in an embodiment of the application;
fig. 4a is an effect diagram of visual comparison between a predicted value and a true value of a time delay KQI index in the embodiment of the present application;
FIG. 4b is a graph showing the effect of visual comparison of the predicted and actual values of the rate KQI index in the example of the present application;
FIG. 5 is a flow chart of performing KQI and KPI correlation determination in accordance with an embodiment of the present application;
fig. 6 is a block diagram of a device for determining correlation between KQI and KPI in network communication according to an embodiment of the present application;
Fig. 7 is a block diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Fig. 1 is a flowchart of a method for determining correlation between KQI and KPI in network communication according to an embodiment of the present application, as shown in fig. 1, including the following steps.
Step 110, key performance index KPI data is obtained, key quality index KQI data is obtained, the KPI data and the KQI data respectively comprise associated fields, and the KPI data further comprise data values of a plurality of KPI index fields.
Wherein, the associated fields comprise fields such as broadband access number, data batch number, date and the like.
And collecting KPI data of each layer in the metropolitan area network, and cleaning and arranging the collected KPI data to obtain KPI data comprising associated fields and KPI index fields.
The method comprises the steps of deploying a video client through simulating a user scene, continuously grabbing packets through a packet grabbing tool in the process of playing videos by the video client, obtaining packet grabbing data such as packet grabbing time, packet length and time delay, and obtaining KQI data based on packet grabbing data statistics. The KQI data includes at least one of a network transmission rate, jitter, and delay, etc.
In one embodiment of the present application, the acquiring key performance indicator KPI data includes: collecting KPI basic data about a plurality of data models in a network transmission layer, wherein the KPI basic data comprises collection time and broadband access numbers; counting the KPI basic data with acquisition time within the same preset time length according to the broadband access number aiming at each data model to obtain KPI statistical data of each data model, wherein the KPI statistical data comprises the associated fields; and according to the associated fields in the KPI statistical data of each data model, associating the KPI statistical data of a plurality of data models to obtain the KPI data.
The network transmission layer may be three layers in the metropolitan area network, namely a core layer, a convergence layer and an access layer. The data models can comprise four data models of user light attenuation, access network flow, total router, light cat performance and the like, and are obtained by performing KPI data acquisition on equipment of different layers in the metropolitan area network.
In the process of playing videos of a video client deployed in a simulated user scene, performance data of network transmission layer equipment are collected at regular time or in real time, and the performance data collected at different time are tidied based on a broadband access number, so that KPI basic data of a plurality of data models of the network transmission layer are obtained, wherein the KPI basic data comprise data values of fields such as equipment identification, the broadband access number, collection time, transmission light power, reception light power, transmission light attenuation, inflow rate, outflow rate, downlink light attenuation and the like. And storing the acquired KPI basic data into a Hadoop cluster of the data center station.
And preprocessing the KPI basic data such as data cleaning, data association and the like to form KPI data expressed in a data broad table. Under each data model, statistics is carried out on KPI basic data corresponding to the same broadband access number according to a preset time length (for example, 5 minutes), each preset time length corresponds to each field to obtain a data value, the data value is used as KPI statistical data of each data model, and associated fields in the KPI statistical data comprise fields such as broadband access numbers, dates, data batch numbers corresponding to each preset time length and the like.
The KPI statistical data of the plurality of data models comprise the same associated field, so that the KPI statistical data of the same associated field in different data models can be associated based on the associated field in the KPI statistical data of each data model, and uplink network element KPI data in the user test period in the same dimension represented by the data broad table can be obtained.
The KPI data of the same associated fields of a plurality of data models are obtained through data acquisition and arrangement, and accuracy of the KPI data obtained in a real test scene is guaranteed.
In one embodiment of the present application, the statistics of KPI basic data with acquisition time within the same preset time length is performed for each data model according to the broadband access number, so as to obtain KPI statistical data of each data model, including: dividing each day into a plurality of time periods according to the preset time length, and determining the data batch number corresponding to each time period; determining an average value of KPI basic data with acquisition time in each time period according to the broadband access number aiming at each data model; and taking the broadband access number and the data batch number as the association field, and determining the association field and the average value corresponding to the association field as KPI statistical data of the data model.
According to a preset time length (for example, 5 minutes), dividing each day into a plurality of time periods, wherein each time period corresponds to a data batch number, so that data statistics can be carried out on each time period respectively. For example, when the preset time period is 5 minutes, that is, the data are summarized according to the granularity of 5 minutes, a day is divided into 288 time periods, that is, the data of a day are formed into 288 batches.
When data are summarized by taking the preset time length as granularity, respectively summarizing the data for each data model, under each data model, calculating the average value of all KPI basic data in the same time period in the same KPI index field of the same broadband access number and the same date, and obtaining the average value of KPI index fields corresponding to each KPI index field in the time period, namely the average value of KPI index fields corresponding to the broadband access number, the date and the data batch number under each data model, taking the broadband access number, the date and the data batch number as associated fields, taking the average value of each KPI index field as the data value of the KPI index field, and further determining the average value of the associated field and each KPI index field corresponding to the associated field under each data model as KPI statistical data of the data model.
The daily is divided into a plurality of same time periods through the preset time length, and data statistics is carried out on each time period respectively, so that statistics on data of different data models according to uniform granularity is realized, and KPI statistical data of each data model and association with KQI data are conveniently associated.
In one embodiment of the present application, the acquiring key quality indicator KQI data includes: acquiring KQI basic data when a user side plays a network video; and counting the KQI basic data according to the preset time length and the broadband access number to obtain the KQI data.
The method comprises the steps of deploying a video client through simulating a user scene, continuously grabbing packets through a packet grabbing tool in the process of playing videos of the video client, obtaining packet grabbing data such as packet grabbing time, packet length and time delay, calculating the packet grabbing data, and obtaining at least one KQI basic data such as network transmission rate, jitter and time delay. And counting the KQI basic data according to the preset time length aiming at each broadband access number, namely counting (e.g. averaging, summing and the like) the KQI basic data with the acquisition time within the same preset time length to obtain the KQI data. Wherein, the time delay refers to the time required for data to be transmitted from one end of the network to the other end, which is equal to the difference between the arrival times of two packets; the network transmission rate is generally the speed of uploading and downloading, and the higher the rate is, the faster the uploading and downloading is, and the unit of the rate in the experiment can be byte/s (bytes per second).
The acquired data are collated to obtain the KQI data by carrying out data acquisition when the network video is played on the user side, so that the accuracy of the acquired KQI data in a real test scene is ensured.
In one embodiment of the present application, the counting the KQI basic data according to a preset time length and a broadband access number to obtain KQI data includes:
dividing each day into a plurality of time periods according to the preset time length, determining a data batch number corresponding to each time period, and determining an average value of KQI basic data in each time period according to the broadband access number;
and taking the broadband access number and the data batch number as the association field, and determining the average value of the association field and KQI basic data corresponding to the association field as the KQI data.
According to the same preset time length (for example, 5 minutes) as the KPI data statistics, dividing each day into a plurality of time periods, wherein each time period corresponds to a data batch number, so that the data statistics can be carried out on each time period later. For example, when the preset time period is 5 minutes, that is, the data are summarized according to the granularity of 5 minutes, a day is divided into 288 time periods, that is, the data of a day are formed into 288 batches. For the same broadband access number and the same date, calculating the average value of all KQI basic data in the same time period in the same KQI index field at the acquisition time to obtain the average value of each KQI index field corresponding to the time period, namely the average value of the KQI index fields corresponding to the broadband access number, the date and the data batch number, taking the broadband access number, the date and the data batch number as the associated field, taking the average value of each KQI index field as the data value of the KQI index field, and further determining the average value of the associated field and each KQI index field corresponding to the associated field as the KQI data. After KQI data processing, converging the data center Hadoop clusters to form a final KQI data model, wherein the KQI data model comprises the following key information: broadband access number, batch number, date, time delay, jitter, rate and other KQI characteristic information.
Dividing each day into a plurality of same time periods through preset time length, respectively carrying out data statistics on each time period, adopting the same preset time length as the KPI data statistics, facilitating the association of the KQI data and the KPI data, and further searching the mapping relation between the KQI data and the KPI data.
And step 120, establishing an association relationship between the KPI data and the KQI data according to the association field.
And performing data integration on KPI data and KQI data through related fields such as broadband access numbers, dates, data batch numbers and the like, establishing a related relation between the KPI data and the KQI data, deleting the KPI data which are not related to the KQI data and the KQI data which are not related to the KPI data, reserving the KPI data and the KQI data which are simultaneously arranged in a time period, performing unsupervised clustering (for example, K-Means clustering) on all the whole data by taking the related KPI data and the KQI data as whole data, and removing single whole data (for example, most of the data are relatively close, and less data are relatively far away from most of the data, and removing the less data). And for the preserved associated KPI data and KQI data, uniformly storing the KPI data and the KQI data through a data table to form a convergence model containing the KQI data and the KPI data in a continuous time period, and providing a data basis for subsequent algorithm calculation.
After the association relationship between the KPI data and the KQI data is established, that is, after an aggregation model for protecting the KQI data and the KPI data is generated, the aggregation model includes data of a plurality of fields (for example, 122 fields can be obtained) such as an association field, a KPI index field, a KQI index field, and the like, where at least the key fields include: the key information of the user such as broadband access number, data batch number, date, time delay, speed, transmitting optical power, receiving optical power, transmitting optical attenuation, inflow speed, outflow speed, downlink optical attenuation and the like. The broadband access number, the data batch number and the date are associated fields, the time delay and the speed are KQI index fields, and the key information such as the transmitted light power, the received light power, the transmitted light attenuation, the inflow speed, the outflow speed, the downlink light attenuation and the like is KPI index fields.
And 130, selecting KPI index fields with data values meeting fluctuation conditions from the KPI data with the association relation, and taking the data values of the selected KPI index fields and the KQI data with the association relation as initial samples.
Through the data processing, the KQI data at the user side and the KPI data at the network layer of the user are successfully converged, and the original samples of a wide table of a convergence model in which the KPI data and the KQI data are converged, wherein the original samples have a plurality of (e.g. 122) fields in total, are obtained. Illustratively, of the 122 fields, 2 fields are user-side KQI indexes, 1 field is time information, 11 fields are associated fields, and the remaining 107 fields are KPI indexes.
In the embodiment of the application, the architecture of an EQL (Equation Learner) network is used for carrying out symbolic regression, and the EQL can learn a relatively simple association relation formula from the input and output of a numerical form, so that a KPI index field with a certain fluctuation value is selected from KPI data to be used as the input of the EQL, and the data value of the KPI index field meeting the fluctuation condition and KQI data correlated with the KPI data are used as initial samples. In an exemplary embodiment, graphs corresponding to data values of all the KPI index fields can be obtained by drawing all the KPI index fields in the KPI data according to time (i.e. data batch number), the graphs of all the KPI index fields are respectively input into a graph neural network, and the KPI index fields meeting the fluctuation condition are selected through the graph neural network. Illustratively, of the 107 KPI index fields described above, 38 KPI index fields are not numerical indicators after processing; the index contents of the 17 KPI index fields are all null values; the index values of the 11 KPI index fields are all consistent; the other 33 index fields have non-unique values, but have no change or only do simple linear change with time in the daily measurement time period; the remaining available KPI index fields (i.e., KPI index fields satisfying the fluctuation condition) are 8 in total, including inflow rate, outflow rate, photo-cat upstream traffic, photo-cat downstream traffic, photo-cat received light power, photo-cat working time, photo-cat CPU utilization, downstream light decay.
And 140, carrying out standardization processing on the initial sample to obtain a training sample, wherein the training sample comprises standardized KPI data and standardized KQI data.
The KPI index fields in the initial sample and the KQI index fields at the user side have different units, the index variable dimensions are different, the numerical distribution is quite different, and in order to be capable of uniformly processing, the data of each KPI index field and the data of each KQI index field in the initial sample are required to be standardized to obtain a training sample consisting of standardized KPI data and standardized KQI data, and the training sample is used for training an EQL network. The normalized KPI data is data normalized by a plurality of KPI indicator fields, and the normalized KQI data is data normalized by at least one KQI indicator field. The normalization may be performed by Min-max normalization, z-score normalization, or the like.
In one embodiment of the present application, the normalizing the initial sample to obtain a training sample includes:
determining a first mean value and a first standard deviation of a data value of each KPI index field in the initial sample, and carrying out standardization processing on the data value of the KPI index field according to the first mean value and the first standard deviation to obtain standardized KPI index data;
Determining a second average value and a second standard deviation of the data values of the KQI index fields aiming at each KQI index field in the initial sample, and carrying out standardization processing on the data values of the KQI index fields according to the second average value and the second standard deviation to obtain standardized KQI index data;
and generating the training sample according to the standardized KPI index data of each KPI index field and the standardized KQI index data of each KQI index field.
When the normalization processing is performed on the initial sample, considering that the neural network model of the EQL algorithm needs to control the input value and the output value within a limited value range, all the index fields (such as the 8 KPI index fields and the 2 KQI index fields) can be normalized into a distribution with a mean value of 0 and a standard deviation of 1, that is, the data value of each index field is completely subtracted by the mean value of the index field and divided by the standard deviation of the index field.
The normalization processing is performed for each index field. For each KPI index field in an initial sample, calculating the mean value and standard deviation of all data values in the KPI index field, respectively recording the mean value and the standard deviation as a first mean value and a first standard deviation, determining a first difference value between the data values of the KPI index field and the first mean value, dividing the first difference value by the first standard deviation of the KPI index field to obtain a standardized value corresponding to the data value, and forming standardized KPI index data of the KPI index field by the standardized value corresponding to all the data values in the KPI index field.
For each KQI index field in an initial sample, calculating the mean value and standard deviation of all data values in the KQI index field, respectively recording the mean value and standard deviation as a second mean value and a second standard deviation, determining a second difference value between each data value of the KQI index field and the second mean value, dividing the second difference value by the second standard deviation of the KQI index field to obtain a standardized value corresponding to each data value, wherein the standardized value corresponding to all data values in the KQI index field forms standardized KQI index data of the KQI index field.
The standardized KPI index data of all KPI index fields and the standardized KQI index data of all KQI index fields form a training sample for training the EQL network.
The standard deviation is the arithmetic square root of variance, reflects the degree of dispersion among individuals in the group, and the calculation process of the standard deviation is to calculate the difference between each data value and the average value:
wherein sigma represents standard deviation, n represents total number of data values in the index field, and X i Represents the i-th data value in the indicator field and μ represents the mean value of the indicator field.
Illustratively, the data normalization table shown in table 1 can be obtained after the above data is normalized by 8 KPI index fields and 2 KQI index fields (including rate and delay), and the raw average values and standard deviations of 10 index fields are shown in table 1.
Table 1 data normalization table
| Index field name | Raw average value | Original standard deviation |
| Inflow rate | 42921244.53 | 3535521.504 |
| Outflow Rate | 71645105.87 | 2290611.009 |
| Upstream flow of light cat | 165.8788187 | 223.1111438 |
| Light cat downstream flow | 573.617237 | 475.1562313 |
| Light cat receiving light power | -22.39197634 | 0.219376252 |
| Working time of light cat | 4128.804368 | 1664.691144 |
| Light cat cpu utilization rate | 0.610947328 | 0.456328026 |
| Descending light attenuation | -24.94691527 | 0.217089688 |
| Time delay (KQI) | 0.076246756 | 0.099990651 |
| Rate (KQI) | 88869.72763 | 32770.02743 |
The training samples are obtained by respectively carrying out standardization processing on each index field, so that the training samples are standardized data, the training of the EQL network is facilitated, the standardization processing is carried out through the mean value and the standard deviation, the method is more suitable for the input and the output of the EQL network, and the accuracy of the mapping relation between the determined KQI data and the KPI data can be improved.
And step 150, training an equation learner (EQL) network according to the training sample, and determining the mapping relation between the standardized KQI data and the standardized KPI data according to the EQL network and the network weight of the EQL network after the training is completed.
And inputting the standardized KPI data in the training sample into an EQL network, processing the standardized KPI data through the EQL network to obtain output data, substituting the output data and the standardized KQI data in the training sample into a formula of a loss function to obtain a loss function value, performing back propagation based on the loss function value, adjusting the network weight of the EQL network, and iteratively executing the training process until the training ending condition is met, wherein the EQL network training is completed. After the EQL network training is completed, a formula of a mapping relation between output and input is determined according to the structure of the EQL network and the network weight of the EQL network, and a mapping relation between standardized KQI data and standardized KPI data is obtained.
FIG. 2 is an exemplary diagram of a trained EQL network in accordance with the embodiments of the present application, as shown in FIG. 2, the mapping relationship between output and input according to the network weights shown in FIG. 2 may be expressed as:
y＝-0.33sin(-3.13x 1 )+0.33sin(6.28x 2 +0.39)-0.35[-0.94(x 2 -0.17)]+0.42(0.83x 3 *-0.95x 4 )
＝-0.33sin(-3.13x 1 )+0.33sin(6.28x 2 +0.39)+0.33x 2 -0.056-0.33x 3 x 4
where y is the output, x 1 、x 2 、x 3 、x 4 Representing 4 inputs.
In one embodiment of the present application, the training the equation learner EQL network according to the training samples includes:
inputting standardized KPI data in the training sample into the EQL network, sequentially processing the input standardized KPI data through a plurality of hidden layers in the EQL network, and activating the processing result through a plurality of nonlinear activating functions to obtain output data; and determining a loss function value according to the output data and the standardized KQI data in the training sample, and adjusting the network weight of the EQL network based on the loss function value, wherein the loss function is a mixed regularized loss function.
FIG. 3 is an exemplary diagram of an EQL network employing neural networks for symbolic regression in an embodiment of the application. EQL (Equation Learner) is a special neural network algorithm, and the unified activation function in the traditional neural network is replaced by a plurality of nonlinear activation functions with each layer, as shown in fig. 3, only three activation functions (the activation functions may be id, square, sine, multiplication, etc.) and two hidden layers are shown in the figure, and in fact, the network may include more activation functions and more hidden layers to adapt to a wider class of functions.
Inputting standardized KPI data in a training sample into an EQL network, sequentially processing the input standardized KPI data through a plurality of hidden layers in the EQL network, activating processing results through a plurality of nonlinear activating functions corresponding to the hidden layers, inputting the processing results of each hidden layer into the next hidden layer to continue such processing, and outputting through an output layer after processing of all hidden layers is completed to obtain output data; substituting the output data and the standardized KQI data in the training sample into a formula of a loss function to obtain a loss function value, performing directional propagation based on the loss function value, adjusting the network weight of the EQL network, and iteratively executing the training process until the training ending condition is met to obtain the trained EQL network.
The hybrid regularization loss function is expressed as: l=reg lambda *L1+(1-reg lambda ) L2, wherein L1 represents L1 regularization and L2 represents L2 regularization. One key parameter of the EQL algorithm is the regularization term coefficient (reg) in the hybrid regularized loss function lambda ). Scaling up the regular term coefficients may make the algorithm more prone to preferentially simplify the formulation during the training process, and scaling down the regular term coefficients may make the algorithm more prone to preferentially reduce the prediction error of the formulation during the training process. The regular term parameters need to be adjusted to moderate values to enable the EQL algorithm to converge into a formula with reasonable length and generalization capability.
By training the EQL network based on the training samples and performing the activation process through a plurality of nonlinear activation functions in the EQL network, the mapping relationship between the output and the input can be effectively learned.
According to the method for determining the correlation between the KQI and the KPI in the network communication, provided by the embodiment of the application, the association relation between the KPI data and the KQI data is established according to the association field, the KPI index field with the data value meeting the fluctuation condition is selected from the KPI data with the association relation, the data value of the selected KPI index field and the associated KQI data are used as initial samples, the initial samples are subjected to standardized processing, a training sample is obtained, the EQL network is trained according to the training sample, after the training is completed, the mapping relation between the standardized KQI data and the standardized KPI data is determined according to the network weights of the EQL network and the EQL network, the KPI data and the KQI data are obtained through long-term acquisition, the KPI data and the network layer KPI data of a user are subjected to transverse modeling according to a time sequence, the characteristic analysis training is carried out on the KQI time and the KPI data by adopting a QI neural network algorithm, and finally the mapping relation between the KQI data and the KPI data is obtained, namely, the mapping relation between the QI and the KPI can be reflected by a QI is obtained, the correlation between the KPI and the KPI is improved, the accuracy and the user is improved, the sense of the correlation between the QI and the user is reasonably provided, and the sense of the user is improved, and the sense of the relation is reasonable.
On the basis of the above technical solution, after determining the mapping relationship between the standardized KQI data and the standardized KPI data according to the network weights of the EQL network and the EQL network, the method further includes: determining standardized KQI prediction data corresponding to standardized KPI data in a test sample according to the mapping relation; a mean square error between the normalized KQI predicted data and normalized KQI data in the test sample is determined.
The test sample acquiring mode is the same as the training sample acquiring mode, and includes standardized KQI data and standardized KPI data, and the training samples obtained after all data standardization processing can be used for training the EQL network in part and used as the test samples in the other part to test the mapping relationship between the trained KQI and KPI.
The mean square error in mathematical statistics refers to the expected value of the square of the difference between the parameter estimation value and the parameter true value, denoted MSE. MSE is a convenient method for measuring average error, the MSE can evaluate the change degree of data, and the smaller the MSE value is, the better accuracy of the prediction model description experimental data is shown.
Calculating the mean value of the sum of squares of errors of corresponding points of the predicted data and the original data according to a calculation formula of the MSE:
where MSE represents the mean square error, n represents the number of test samples, y i Represents normalized KQI data in the ith test sample,and the normalized KQI prediction data of the ith test sample determined by the mapping relation is represented.
Substituting the standardized KPI data in the test sample into a formula of the mapping relation to obtain standardized KQI prediction data, and determining the mean square error between the standardized prediction data and the standardized KQI data in the test sample based on the formula of the mean square error. The accuracy of the mapping relation description test sample can be embodied through the mean square error. If the mean square error is too large, training of the EQL network can be conducted again, and finally a relatively simple mapping formula between KQI and KPI with higher accuracy can be obtained.
When the KQI index field includes a plurality of fields, training of the EQL network may be performed for each field to obtain a mapping relationship between the KQI index and the KPI index.
Illustratively, for the 2 KQI indices and 8 KPI indices of the above table, a relatively compact formula may be obtained after training. For KQI index delay, when the regularization term coefficient reg lambda When=1/128, the mean square error mse= 0.3478, a highly readable, relatively compact formula can be obtained as follows:
y 1 ＝-0.49x 1 -1.14x 2 2 +0.41x 2 +1.15
wherein y is 1 Indicating time delay (KQI index), x 1 Representing the received light power of the light cat, x 2 Indicating the working time of the light cat.
For KQI index rate, when the regularization term coefficient reg lambda When=1/128, the mean square error mse= 0.5956, a highly readable, relatively compact formula can be obtained as follows:
y 2 ＝0.47x 1 -0.02x 2 x 3 +0.82x 2 2 -0.86
wherein y is 2 Indicating the rate (KQI index), x 1 Representing the received light power of the light cat, x 2 Indicating the working time length of the light cat, x 3 Indicating the inflow rate.
In the process of training the EQL network, all KQI indexes are standardized so that the mean value is 0 and the variance is 1, so that the MSE error obtained by directly predicting 0 is equal to 1, and correspondingly, the predicted MSE error is obviously smaller than 1, which means that the prediction effect of the formula is obviously better than that of the mean value of the index. From the results of the formulas obtained above, it can be seen that most formulas learned by EQL achieve MSE errors significantly less than 1 with a number of parameters much less than that of conventional neural networks.
For two KQI indexes of time delay and speed, a relatively simple mapping formula learned by EQL can be respectively selected, and based on experimental data (test samples), the comparison of the KQI predicted by using KPI data and the real KQI is plotted and observed. Fig. 4a is an effect diagram of visual comparison between a predicted value and a real value of a time delay KQI index in the embodiment of the present application, as shown in fig. 4a, the horizontal axis is time, the vertical axis is a normalized real value and a normalized predicted value of the time delay KQI index during experimental data acquisition, the scattered point 1 represents the normalized real value of the time delay, and the scattered point 2 represents the normalized predicted value of the time delay. Fig. 4b is an effect diagram of visual comparison between a predicted value and a real value of a velocity KQI index in the embodiment of the present application, as shown in fig. 4b, the horizontal axis is time, the vertical axis is a normalized real value and a normalized predicted value of the velocity KQI index during experimental data acquisition, the scattered point 3 represents the normalized real value of the velocity, and the scattered point 4 represents the normalized predicted value of the velocity.
From fig. 4a and fig. 4b, it can be seen that a certain fitting trend appears between the true KQI and the predicted KQI, and the formula obtained by the KQL algorithm can still predict each KQI index to a certain extent while keeping the formula simple and readable and the parameter number far less than that of the traditional neural network model.
The correlation between KQI indexes and KPIs under the two non-zero formulas of the time delay and the speed obtained by training is related to the received light power of the optical cat and the working time of the optical cat from a simple formula, the correlation between the related KPIs and the KQI can be observed through a scatter diagram, the received light power of the time delay KQI and the optical cat is drawn in a graph, the correlation between the time delay KQI and the received light power of the optical cat is visually represented, the working time of the time delay KQI and the optical cat is drawn in a graph, and the correlation between the time delay KQI and the working time of the optical cat is visually represented; the rate is related to the received light power of the photo cat, the working time of the photo cat and the inflow rate, the relation between the related KPI and KQI can be observed through a scatter diagram, the rate and the received light power of the photo cat can be drawn in a diagram, the association relation between the rate KQI and the received light power of the photo cat is visually represented, the rate and the working time of the photo cat are drawn in a diagram, the association relation between the rate KQI and the working time of the photo cat is visually represented, the rate and the inflow rate are drawn in a diagram, and the association relation between the rate KQI and the inflow rate is visually represented. When the association relationship between the KQI and the KPI is visually represented, the normalized value of each index may be plotted.
Fig. 5 is a schematic flow chart of performing KQI and KPI correlation determination in the embodiment of the application, and as shown in fig. 5, KPI data is collected by a network device; collecting KQI data through grabbing a packet; storing the acquired data into a hadoop cluster; processing the acquired data to obtain a convergence wide table, wherein the convergence wide table comprises KPI data (KPI model), KQI data (KQI model) and the association relation (convergence model) between KQI and KPI; and predicting an EQL algorithm based on the association relation between the KQI and the KPI to obtain a mapping relation between the KQI and the KPI, wherein the time delay KQI index is related to the received light power and the working time of the photo cat, and the rate KQI index is related to the received light power, the working time and the inflow rate of the photo cat.
The embodiment of the application adopts a big data technology to aggregate and simulate the user side KQI data and the network transmission layer equipment KPI data in a user scene to carry out modeling; and the symbol regression architecture EQL based on the neural network predicts the relationship between the KQI and the KPI, and verifies the result by adopting big data AI, thereby ensuring the accuracy of the mapping relationship between the KQI and the KPI.
Embodiments of the present application use a neural network-based symbolic regression architecture, known as an equation learner (EQL) network, for finding analytical equations describing data, which can explain the model and the ability to predict invisible data. In contrast, conventional neural network algorithms are generally considered to be difficult to interpret and often infer poor black box models. The neural network architecture based on symbolic regression in the embodiment of the application predicts the relationship between KQI and KPI, and can utilize a powerful deep learning technology, and meanwhile, the method still generates interpretable and generalizable results.
Fig. 6 is a block diagram of a device for determining correlation between KQI and KPI in network communication according to an embodiment of the present application, where, as shown in fig. 6, the device includes:
an index data obtaining module 610, configured to obtain key performance index KPI data and obtain key quality index KQI data, where the KPI data and the KQI data respectively include associated fields, and the KPI data further includes data values of multiple KPI index fields;
an association relationship establishing module 620, configured to establish an association relationship between the KPI data and the KQI data according to the association field;
an initial sample determining module 630, configured to select a KPI indicator field whose data value meets a fluctuation condition from KPI data having the association relationship, and use the data value of the selected KPI indicator field and KQI data having the association relationship as an initial sample;
the normalization processing module 640 is configured to perform normalization processing on the initial sample to obtain a training sample, where the training sample includes normalized KPI data and normalized KQI data;
the mapping relation determining module 650 is configured to train the EQL network of the algorithm learner according to the training samples, and determine a mapping relation between the standardized KQI data and the standardized KPI data according to the network weights of the EQL network and the EQL network after the training is completed.
Optionally, the mapping relation determining module includes:
the network processing unit is used for inputting the standardized KPI data in the training sample into the EQL network, sequentially processing the input standardized KPI data through a plurality of hidden layers in the EQL network, and activating the processing result through a plurality of nonlinear activation functions to obtain output data;
and the network weight adjusting unit is used for determining a loss function value according to the output data and the standardized KQI data in the training sample, adjusting the network weight of the EQL network based on the loss function value, and the loss function is a mixed regularized loss function.
Optionally, the standardized processing module includes:
the KPI normalization processing unit is used for determining a first mean value and a first standard deviation of the data value of each KPI index field in the initial sample, and performing normalization processing on the data value of the KPI index field according to the first mean value and the first standard deviation to obtain normalized KPI index data;
a KQI normalization processing unit, configured to determine, for each KQI indicator field in the initial sample, a second average value and a second standard deviation of data values of the KQI indicator field, and perform normalization processing on the data values of the KQI indicator field according to the second average value and the second standard deviation, so as to obtain normalized KQI indicator data;
And the training sample generation unit is used for generating the training sample according to the standardized KPI index data of each KPI index field and the standardized KQI index data of each KQI index field.
Optionally, the index data acquisition module includes:
the KPI basic data acquisition unit is used for acquiring KPI basic data about a plurality of data models in a network transmission layer, wherein the KPI basic data comprises acquisition time and broadband access numbers;
the KPI basic data statistics unit is used for counting KPI basic data with acquisition time within the same preset time length according to the broadband access number aiming at each data model to obtain KPI statistical data of each data model, wherein the KPI statistical data comprises the associated fields;
and the KPI data generating unit is used for correlating the KPI statistical data of the plurality of data models according to the correlation field in the KPI statistical data of each data model to obtain the KPI data.
Optionally, the KPI base data statistics unit is specifically configured to:
dividing each day into a plurality of time periods according to the preset time length, and determining the data batch number corresponding to each time period;
determining an average value of KPI basic data with acquisition time in each time period according to the broadband access number aiming at each data model;
And taking the broadband access number and the data batch number as the association field, and determining the association field and the average value corresponding to the association field as KPI statistical data of the data model.
Optionally, the index data acquisition module includes:
the KQI basic data acquisition unit is used for acquiring KQI basic data when the user side plays the network video;
and the KQI data generation unit is used for counting the KQI basic data according to the preset time length and the broadband access number to obtain the KQI data.
Optionally, the KQI data generating unit is specifically configured to:
dividing each day into a plurality of time periods according to the preset time length, determining a data batch number corresponding to each time period, and determining an average value of KQI basic data in each time period according to the broadband access number;
and taking the broadband access number and the data batch number as the association field, and determining the average value of the association field and KQI basic data corresponding to the association field as the KQI data.
Optionally, the apparatus further includes:
the prediction data determining module is used for determining standardized KQI prediction data corresponding to standardized KPI data in the test sample according to the mapping relation;
And the mean square error determining module is used for determining the mean square error between the standardized KQI predicted data and the standardized KQI data in the test sample.
The device for determining the correlation between the KQI and the KPI in the network communication provided by the embodiment of the application is used for implementing each step of the method for determining the correlation between the KQI and the KPI in the network communication described by the embodiment of the application, and specific embodiments of each module of the device refer to corresponding steps, which are not repeated herein.
According to the determining device for the correlation between the KQI and the KPI in the network communication, provided by the embodiment of the application, the correlation between the KPI data and the KQI data is established according to the acquired KPI data and the KQI data, the KPI index field with the data value meeting the fluctuation condition is selected from the KPI data with the correlation, the data value of the selected KPI index field and the associated KQI data are used as initial samples, the initial samples are standardized, the training samples are obtained, the EQL network is trained according to the training samples, after the training is completed, the mapping relation between the standardized KQI data and the standardized KPI data is determined according to the network weights of the EQL network and the EQL network, the KPI data and the KQI data are acquired in a long-term mode, the KPI data and the network layer KPI data of a user are transversely modeled according to a time sequence, the characteristic analysis training is carried out on the KQI time and the KPI data by adopting the EQL neural network algorithm, finally, the mapping relation between the KQI data and the KPI data is obtained, namely, the mapping relation between the QI and the KPI can be represented by the QI is well, the accuracy and the influence of the user is improved, the accuracy and the sensitivity are reasonably provided, the sensitivity of the user is improved, the correlation is improved, and the user is improved, and the accuracy is improved, and the user is better is realized.
Fig. 7 is a block diagram of an electronic device according to an embodiment of the present application, and as shown in fig. 7, the electronic device 700 may include one or more processors 710 and one or more memories 720 connected to the processors 710. Electronic device 700 may also include an input interface 730 and an output interface 740 for communicating with another apparatus or system. Program code executed by processor 710 may be stored in memory 720.
Processor 710 in electronic device 700 invokes program code stored in memory 720 to perform the method of determining the correlation of KQI with KPI in network communications in the above-described embodiments.
The embodiment of the application also provides a computer readable storage medium, and a computer program is stored on the computer readable storage medium, and when the program is executed by a processor, the method for determining the correlation between KQI and KPI in network communication according to the embodiment of the application is realized.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other. For the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points.
The above detailed description of the method, the device, the electronic equipment and the storage medium for determining the correlation between KQI and KPI in network communication provided by the embodiment of the application applies specific examples to illustrate the principle and implementation of the application, and the above description of the embodiment is only used for helping to understand the method and the core idea of the application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or may be implemented by hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Claims (11)
1. A method for determining correlation between KQI and KPI in network communication, comprising:
acquiring Key Performance Index (KPI) data, and acquiring Key Quality Index (KQI) data, wherein the KPI data and the KQI data respectively comprise associated fields, and the KPI data also comprise data values of a plurality of KPI index fields;
establishing an association relationship between the KPI data and the KQI data according to the association field;
selecting KPI index fields with data values meeting fluctuation conditions from KPI data with the association relation, and taking the data values of the selected KPI index fields and KQI data with the association relation as initial samples;
carrying out standardization processing on the initial sample to obtain a training sample, wherein the training sample comprises standardized KPI data and standardized KQI data;
and training an equation learner (EQL) network according to the training sample, and determining the mapping relation between the standardized KQI data and the standardized KPI data according to the EQL network and the network weight of the EQL network after training.
2. The method of claim 1, wherein training an equation learner EQL network based on the training samples comprises:
Inputting standardized KPI data in the training sample into the EQL network, sequentially processing the input standardized KPI data through a plurality of hidden layers in the EQL network, and activating the processing result through a plurality of nonlinear activating functions to obtain output data;
and determining a loss function value according to the output data and the standardized KQI data in the training sample, and adjusting the network weight of the EQL network based on the loss function value, wherein the loss function is a mixed regularized loss function.
3. The method of claim 1, wherein normalizing the initial sample to obtain a training sample comprises:
determining a first mean value and a first standard deviation of a data value of each KPI index field in the initial sample, and carrying out standardization processing on the data value of the KPI index field according to the first mean value and the first standard deviation to obtain standardized KPI index data;
determining a second average value and a second standard deviation of the data values of the KQI index fields aiming at each KQI index field in the initial sample, and carrying out standardization processing on the data values of the KQI index fields according to the second average value and the second standard deviation to obtain standardized KQI index data;
And generating the training sample according to the standardized KPI index data of each KPI index field and the standardized KQI index data of each KQI index field.
4. The method of claim 1, wherein the obtaining key performance indicator KPI data comprises:
collecting KPI basic data about a plurality of data models in a network transmission layer, wherein the KPI basic data comprises collection time and broadband access numbers;
counting the KPI basic data with acquisition time within the same preset time length according to the broadband access number aiming at each data model to obtain KPI statistical data of each data model, wherein the KPI statistical data comprises the associated fields;
and according to the associated fields in the KPI statistical data of each data model, associating the KPI statistical data of a plurality of data models to obtain the KPI data.
5. The method of claim 4, wherein the counting KPI base data with acquisition time within the same preset time length according to the broadband access number for each data model to obtain KPI statistical data of each data model includes:
dividing each day into a plurality of time periods according to the preset time length, and determining the data batch number corresponding to each time period;
Determining an average value of KPI basic data with acquisition time in each time period according to the broadband access number aiming at each data model;
and taking the broadband access number and the data batch number as the association field, and determining the association field and the average value corresponding to the association field as KPI statistical data of the data model.
6. The method according to claim 1, wherein the obtaining key quality indicator KQI data comprises:
acquiring KQI basic data when a user side plays a network video;
and counting the KQI basic data according to the preset time length and the broadband access number to obtain the KQI data.
7. The method of claim 6, wherein the counting the KQI basic data according to a preset time length and a broadband access number to obtain KQI data includes:
dividing each day into a plurality of time periods according to the preset time length, determining a data batch number corresponding to each time period, and determining an average value of KQI basic data in each time period according to the broadband access number;
and taking the broadband access number and the data batch number as the association field, and determining the average value of the association field and KQI basic data corresponding to the association field as the KQI data.
8. The method of claim 1, further comprising, after the determining of the mapping relationship between the normalized KQI data and normalized KPI data according to the network weights of the EQL network and the EQL network:
determining standardized KQI prediction data corresponding to standardized KPI data in a test sample according to the mapping relation;
a mean square error between the normalized KQI predicted data and normalized KQI data in the test sample is determined.
9. A device for determining correlation between KQI and KPI in network communication, comprising:
the system comprises an index data acquisition module, a Key Performance Index (KPI) data acquisition module and a Key Quality Index (KQI) data acquisition module, wherein the KPI data and the KQI data respectively comprise associated fields, and the KPI data also comprise data values of a plurality of KPI index fields;
the association relation establishing module is used for establishing association relation between the KPI data and the KQI data according to the association field;
the initial sample determining module is used for selecting KPI index fields with data values meeting fluctuation conditions from KPI data with the association relation, and taking the data values of the selected KPI index fields and KQI data with the association relation as initial samples;
The standardized processing module is used for carrying out standardized processing on the initial sample to obtain a training sample, wherein the training sample comprises standardized KPI data and standardized KQI data;
and the mapping relation determining module is used for training the equation learner (EQL) network according to the training sample, and determining the mapping relation between the standardized KQI data and the standardized KPI data according to the EQL network and the network weight of the EQL network after the training is completed.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method for determining the correlation of KQI and KPI in network communications according to any of the claims 1 to 8 when executing the computer program.
11. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements a method of determining a KQI and KPI correlation in a network communication according to any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310821505.3A CN116996393A (en) | 2023-07-05 | 2023-07-05 | Method, device, equipment and medium for determining correlation between KQI and KPI in network communication |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310821505.3A CN116996393A (en) | 2023-07-05 | 2023-07-05 | Method, device, equipment and medium for determining correlation between KQI and KPI in network communication |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116996393A true CN116996393A (en) | 2023-11-03 |
Family
ID=88533053
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310821505.3A Pending CN116996393A (en) | 2023-07-05 | 2023-07-05 | Method, device, equipment and medium for determining correlation between KQI and KPI in network communication |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116996393A (en) |
-
2023
- 2023-07-05 CN CN202310821505.3A patent/CN116996393A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109347668A (en) | A kind of training method and device of service quality assessment model | |
| CN111294812B (en) | Resource capacity-expansion planning method and system | |
| Poskitt | Autoregressive approximation in nonstandard situations: The fractionally integrated and non-invertible cases | |
| CN103987056A (en) | Wireless network telephone traffic prediction method based on big-data statistical model | |
| CN113780436B (en) | Complex network key node identification method based on comprehensive degree | |
| CN103840988A (en) | Network traffic measurement method based on RBF neural network | |
| CN104468257B (en) | Cloud application availability Forecasting Methodology and system based on mobile subscriber's time-space behavior | |
| CN112054943A (en) | Traffic prediction method for mobile network base station | |
| Shi et al. | Privacy-aware edge computing based on adaptive DNN partitioning | |
| CN108229731B (en) | System and method for predicting user behavior under hot topics under multi-message interaction | |
| CN104581749B (en) | A kind of prediction technique and device of mobile network data service traffic | |
| CN113988441A (en) | Power wireless network link quality prediction and model training method and device | |
| CN109242250A (en) | A kind of user's behavior confidence level detection method based on Based on Entropy method and cloud model | |
| CN114039870B (en) | Deep learning-based real-time bandwidth prediction method for video stream application in cellular network | |
| Mosavat-Jahromi et al. | Prediction and modeling of spectrum occupancy for dynamic spectrum access systems | |
| CN102629341A (en) | Web service QoS (Quality of Service) on-line prediction method based on geographic position information of user | |
| WO2022021420A1 (en) | Root cause localization method and apparatus, and electronic device | |
| CN111901134B (en) | Method and device for predicting network quality based on recurrent neural network model (RNN) | |
| CN114301935A (en) | Reputation-based method for selecting edge cloud collaborative federated learning nodes of Internet of things | |
| CN111785093A (en) | Air traffic flow short-term prediction method based on fractal interpolation | |
| Dong et al. | Multigranular wavelet decomposition-based support vector regression and moving average method for service-time prediction on web map service platforms | |
| CN110222098A (en) | Electric power high amount of traffic abnormality detection based on flow data clustering algorithm | |
| CN103281555A (en) | Half reference assessment-based quality of experience (QoE) objective assessment method for video streaming service | |
| US20230409008A1 (en) | Methods for smart gas data management, internet of things systems, and storage media | |
| CN106789349B (en) | Quality of experience modeling analysis and conversation flow classification based method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |