CN107026750A - A kind of user's online QoE evaluation methods and device - Google Patents
A kind of user's online QoE evaluation methods and device Download PDFInfo
- Publication number
- CN107026750A CN107026750A CN201610074975.8A CN201610074975A CN107026750A CN 107026750 A CN107026750 A CN 107026750A CN 201610074975 A CN201610074975 A CN 201610074975A CN 107026750 A CN107026750 A CN 107026750A
- Authority
- CN
- China
- Prior art keywords
- qoe
- index
- kpi
- major classifications
- influence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/50—Network service management, e.g. ensuring proper service fulfilment according to agreements
- H04L41/5003—Managing SLA; Interaction between SLA and QoS
- H04L41/5009—Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/50—Testing arrangements
- H04L43/55—Testing of service level quality, e.g. simulating service usage
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/10—Scheduling measurement reports ; Arrangements for measurement reports
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Environmental & Geological Engineering (AREA)
- Quality & Reliability (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a kind of user's online experience quality (QoE) evaluation method, methods described includes：Collect Key Performance Indicator (KPI) data of network element；KPI to QoE mapping is set up based on collected KPI data；QoE evaluation system models are set up based on mapping result.The present invention further simultaneously discloses a kind of user's online QoE evaluating apparatus.Using technical solution of the present invention, the use that can more accurately obtain user's online is perceived.
Description
Technical field
The present invention relates to user's perception analysis technology of communication technical field, more particularly to a kind of upper dictyosome of user
The amount of checking the quality (QoE, Quality of Experience) evaluation method and device.
Background technology
Client perception is the subjective feeling of the user using network, the side commonly used at present in real network O＆M
Method has three kinds, is client's sample survey method, existing network operational indicator testing method and traditional single net respectively
The method of network O＆M metrics evaluation client perception.
Client's sample survey fado collects the perception situation of client in the form of survey.But,
The evaluation method workload is big, and efficiency is low, is difficult to repeat to measure.When evaluating heterogeneous networks, due to region
With the difference in humanity, the evaluation method fluctuation it is larger, the comparativity between network is weaker.
Existing network operational indicator testing method, without NMS and other operational systems, can measure any
The index situation of network, and directly obtain the use sensing results of network, with independently of equipment manufacturers,
And it is largely unrelated with bearing bed, the advantages of test process is flexible.But, the evaluation method needs
Appointed place is reached, labor intensive material resources, test result only represents some areas index situation, without general
All over property.
The method of traditional single network O＆M metrics evaluation client perception, based on operation angle initialization network fortune
Row Key Performance Indicator (KPI, Key Performance Indicator), can be obtained by network management system.Should
The terminal online that evaluation method evaluates user with for example wireless percent of call completed of single index, success rate of attachment etc. makes
With perception.But, the evaluation method can be with the situation of trickle observation each single item index, and user makes to terminal
With the susceptibility of perception far away from the network equipment, so that cause O＆M to concentrate on the alarm for handling relevant device,
But can not accurate description go out influence degree of each index to user.With the continuous expansion of network size and logical
The continuous renewal of letter technology, the workload of equipment O＆M increases therewith, and human cost is stablized relatively.Therefore should
Evaluation method had both been unfavorable for the evaluation of user's perception or had been unfavorable for the lifting of operating efficiency.
The content of the invention
In view of this, can be more accurate present invention contemplates that providing a kind of user's online QoE evaluation methods and device
The use that ground obtains user's online is perceived.
To reach above-mentioned purpose, the technical proposal of the invention is realized in this way：
The invention discloses a kind of user online QoE evaluation methods, methods described includes：
Collect the KPI data of network element；
KPI to QoE mapping is set up according to collected KPI data；
QoE evaluation system models are set up based on mapping result.
In such scheme, it is preferable that the mapping that KPI to QoE is set up based on collected KPI data,
Including：
The KPI data being collected into is classified according to access property, integrality, retentivity；
The Key Quality Indicator (KQI, Key Quality Indicators) of the major classifications of setting influence；
KQI based on the major classifications of influence determines the thin item index of the major classifications of influence.
In such scheme, it is preferable that described to set up QoE evaluation system models based on mapping result, including：
Analyze the weight accounting of major classifications；
The weight accounting of the thin item index of the major classifications of analyzing influence；
Set external influence factor regulation coefficient；
According to the weight accounting of major classifications, the weight accounting of the thin item index of the major classifications of influence, with
And the external influence factor regulation coefficient determines the QoE value calculation formula of QoE evaluation system models.
In such scheme, it is preferable that methods described also includes：
When being scored according to the QoE values calculation formula QoE appraisement systems, made using nine points, often
Individual index obtains 9 points when reaching the scoring upper limit, and 0 point, when between bound is scored at when scoring lower limit
Linear Score.
In such scheme, it is preferable that described to be set up based on mapping result after QoE evaluation system models, institute
Stating method also includes：
Count the type of service delta data in preset time period；
The type of service delta data is analyzed；
The index included based on analysis result to the QoE evaluation system models and the weight system of each index
Number, external influence factor regulation coefficient carry out adaptivity amendment.
The invention also discloses a kind of user online QoE evaluating apparatus, described device includes：
Collector unit, the KPI data for collecting network element；
Map unit, the mapping for setting up KPI to QoE according to collected KPI data；
Unit is set up, for setting up QoE evaluation system models based on mapping result.
In such scheme, it is preferable that the map unit, it is additionally operable to：
The KPI data being collected into is classified according to access property, integrality, retentivity；
The KQI of the major classifications of setting influence；
KQI based on the major classifications of influence determines the thin item index of the major classifications of influence.
In such scheme, it is preferable that described to set up unit, it is additionally operable to：
Analyze the weight accounting of major classifications；
The weight accounting of the thin item index of the major classifications of analyzing influence；
Set external influence factor regulation coefficient；
According to the weight accounting of major classifications, the weight accounting of the thin item index of the major classifications of influence, with
And the external influence factor regulation coefficient determines the QoE value calculation formula of QoE evaluation system models.
In such scheme, it is preferable that described device also includes：Computing unit, is used for：
When being scored according to the QoE values calculation formula QoE appraisement systems, made using nine points, often
Individual index obtains 9 points when reaching the scoring upper limit, and 0 point, when between bound is scored at when scoring lower limit
Linear Score.
In such scheme, it is preferable that described device also includes：Amending unit, is used for：
Count the type of service delta data in preset time period；
The type of service delta data is analyzed；
The index included based on analysis result to the QoE evaluation system models and the weight system of each index
Number, external influence factor regulation coefficient carry out adaptivity amendment.
User's online QoE evaluation methods provided by the present invention and device, collect the Key Performance Indicator of network element
KPI data；KPI to QoE mapping is set up according to collected KPI data；Set up based on mapping result
QoE evaluation system models.In this way, technical scheme of the present invention can more comprehensively, more accurately obtain user
The use of online is perceived, and realizes efficient, the intelligent, standardization for being surfed the Net to user and perceiving and evaluating.
Brief description of the drawings
Fig. 1 is the implementation process figure of user provided in an embodiment of the present invention online QoE evaluation methods；
Fig. 2 is a kind of schematic diagram of KPI to QoE mapping provided in an embodiment of the present invention；
Fig. 3 implements flow for user's online QoE appraisement systems provided in an embodiment of the present invention of setting up
Figure；
The QoE appraisement systems that Fig. 4 obtains for the statistics provided in an embodiment of the present invention according to one month
The other weight accounting schematic diagram of three major types；
Fig. 5 provides QoE appraisement system appraisal result schematic diagrames for the embodiment of the present invention；
Fig. 6 is that QoE appraisement systems provided in an embodiment of the present invention scoring locally complains examination item to complain with LTE
The contrast schematic diagram of amount；
Fig. 7 is the composition structural representation of user provided in an embodiment of the present invention online QoE evaluating apparatus.
Embodiment
In order to more fully hereinafter understand the features of the present invention and technology contents, below in conjunction with the accompanying drawings to this hair
Bright realization is described in detail, appended accompanying drawing purposes of discussion only for reference, not for limiting the present invention.
Fig. 1 is the implementation process figure of user provided in an embodiment of the present invention online QoE evaluation methods, such as Fig. 1
Shown, methods described is mainly included the following steps that：
Step 101：Collect Key Performance Indicator (KPI) data of network element.
In an embodiment, the KPI numbers of network element can be collected by existing NMS
According to.
Specifically, the whole flow process surfed the Net according to user, (PDN, Public from attachment flow to public data network
Data Network) set up, then to transmission control protocol (TCP, Transmission Control Protocol)
Connection and final user's GET or POST behavior, by wireless side, core side, industry in data traffic flows
The KPI of business side is listed one by one.
Wherein, GET behaviors refer to the page info for asking to specify, and return to entity body.
Wherein, POST behaviors refer to that request server receives specified document as the unification to being identified
The new subordinate entity of resource identifier (URI, Uniform Resource Identifier).
Step 102：KPI to QoE mapping is set up according to collected KPI data.
Preferably, the mapping that KPI to QoE is set up based on collected KPI data, can include：
The KPI data being collected into is classified according to access property, integrality, retentivity；
The Key Quality Indicator (KQI, Key Quality Indicators) of the major classifications of setting influence；
KQI based on the major classifications of influence determines the thin item index of the major classifications of influence.
Specifically, when the KQI based on the major classifications of influence determines the thin item index of the major classifications of influence,
It is contemplated that following factor：
Pass through the quantity of request times, the size of time delay, the accounting for the number of times for going offline or switching.
Fig. 2 shows a kind of schematic diagram of KPI to QoE mapping, as shown in Fig. 2 QoE feels for user
Know, and KPI is network performance index, for that will set up mapping relations between the two, then introduces KQI.
That is, the conventional mapping that QoE appraisement systems are set up is KPI → KQI → QoE process.
From the point of view of network operation angle, most directly reaction client to the service condition of network be client throwing
Tell information；For user perceives angle, the access of network, integrality, retentivity are related generally to；Cause
This, can by calling information according to access property, that integrality, retentivity are divided into three major types is other.
By the analysis to customer complaint information, can be using the links of process with reference to user's business of networking
Each corresponding KQI of big category setting and each KQI indexs algorithm.It should be noted that KQI is one group
The KPI indexs that can be measured.
Specifically, the KQI indexs and index implication corresponding to each big classification are referred to table 1.
Table 1
Step 103：QoE evaluation system models are set up based on mapping result.
Preferably, it is described that QoE evaluation system models are set up based on mapping result, it can include：
Analyze the weight accounting of major classifications；
The weight accounting of the thin item index of the major classifications of analyzing influence；
Set external influence factor regulation coefficient；
According to the weight accounting of major classifications, the weight accounting of the thin item index of the major classifications of influence, with
And the external influence factor regulation coefficient determines the QoE value calculation formula of QoE evaluation system models.
Specifically, QoE values calculation formula is as follows：
Wherein, f and λ represents the influence coefficient of external influence factor respectively；I represents one kind in major classifications,
I=1,2,3 ..., n；K represents the thin item index under every big classification, and k is the positive integer more than or equal to 1；
δkThe weight corresponding to each thin item index under representing per major class；δiRepresent corresponding to per big classification index
Weight.
In such scheme, it is preferable that methods described also includes：
When being scored according to the QoE values calculation formula QoE appraisement systems, made using nine points, often
Individual index obtains 9 points when reaching the scoring upper limit, and 0 point, when between bound is scored at when scoring lower limit
Linear Score.
Constantly changed with custom with the usage behavior of user, it is type of service meeting to be mapped to network side
Constantly change；Therefore, appraisement system is perceived in order to constantly improve user's online QoE, it is described to be based on
Mapping result is set up after QoE evaluation system models, and methods described also includes：
Count the type of service delta data in preset time period；
The type of service delta data is analyzed；
The index included based on analysis result to the QoE evaluation system models and the weight system of each index
Number, external influence factor regulation coefficient carry out adaptivity amendment.
So, it is possible according to the use habit of user QoE is perceived appraisement system carry out it is constantly perfect.
The online QoE evaluation methods of user described in the present embodiment, at least have the following advantages that：
First, the KPI data of network element can be collected by NMS, be convenient for measuring based on existing network and
Monitoring, can timely feed back achievement data, be more suitable for the KPI indexs day based on network O＆M management system
Often monitoring optimization and the monitoring and lifting of client perception；
Second, the weight of index is obtained by the calculating to the equipment KPI achievement datas in network O＆M, is led to
The mapping model for crossing the KPI-KQI-QoE of setting is calculated, and show that user's online perceives QoE appraisal results；
Method with traditional single network O＆M metrics evaluation client perception is more complete, and QoE appraisal result is straight
Connect the perception situation for having reflected user；
Wireless, core, the full-range factor of business platform are combined in the foundation of 3rd, QoE appraisement system,
And can be more accurately positioned by the distribution of weight client perception is had a major impact in network O＆M because
Element, relative to the method for traditional single network O＆M metrics evaluation client perception, the present embodiment methods described
The influence factor of consideration is more thorough；
4th, can be cleverer compared with the method for traditional single network O＆M metrics evaluation client perception
Adjustment QoE appraisement systems living, to adapt to the development of network.
Embodiment two
Fig. 3 implements flow for user's online QoE appraisement systems provided in an embodiment of the present invention of setting up
Figure, as indicated at 3, the flow is mainly included the following steps that：
Step 301：KPI is refined from network performance, step 302 is then performed.
When refining KPI, it is contemplated that by the quantity of request times, the size of time delay, go offline or switch
The factors such as the accounting of number of times.
Preferably, the KPI, can include：Wireless side, core side, business side in data traffic flows
Index；
Wherein, for access property classification, the index of wireless side includes：RRC is created as power, E-RAB and built
Vertical success rate；The index of core side includes：Success rate of attachment, service request success rate；The index bag of SP sides
Include：TCP is created as power, dns resolution success rate, GET success rates, POST success rates；
Wherein, for classification of pile integrity, the index of wireless side includes：When big bag downloading rate, parcel are downloaded
Prolong；The index of core side includes：Adhere to time delay, TCP setup delays, dns resolution time delay；SP sides
Index includes：GET first packet time delays；
Wherein, for retentivity classification, the index of wireless side includes：Per the wireless number of dropped calls of GB flows,
(with frequency/alien frequencies) per the E-RAB number of dropped calls of GB flows, per GB flows cuts out the frequency of failure；Core
The index of heart side includes：Per the TAU frequency of failures of GB flows.
Wherein, SP is Service Provider abbreviation, and its Chinese is service provider；RRC is
Radio Resource Control abbreviation, its Chinese is wireless heterogeneous networks；E-RAB is Evolved
Radio Access Bearer abbreviation, its Chinese is the RAB of evolution；TCP is
Transmission Control Protocol abbreviation, its Chinese is transmission control protocol；DNS is
Domain Name System abbreviation, its Chinese is domain name system；TAU is Tracking Area
Update abbreviation, its Chinese is tracing section updating；GB is used to represent computer storage cell.
Step 302：KPI to QoE mapping is set up, step 303 is then performed.
Specifically, KPI to QoE mapping specifically how is set up, the step 102 in embodiment one is can refer to,
It will not be repeated here.
Step 303：Analysis draws the weight of each index, then performs step 304.
Preferably, step 303 includes：
Step 303a：Calculate the other weight accounting of QoE appraisement system three major types.
Specifically, for the other weight accounting of QoE appraisement system three major types, computational methods are：
The broad sense counted in a period of time complains data, complains data to classify the broad sense；
Count the complaint quantity of each classification；
Calculate the ratio that each class complains quantity and total complaint quantity.
For example, October, the shared broad sense from 3900 clients was complained, by access property, integrality, guarantor
Holding property is classified, and all kinds of complaint difference accountings are 61.85%, 25.74%, 12.41%.Fig. 4 is implemented for the present invention
The other weight accounting of QoE appraisement system three major types obtained according to the statistics of one month that example is provided is shown
It is intended to.
Through whole to weight coefficient progressization, it is believed that the weight accounting of the other index of QoE appraisement systems three major types
As shown in table 2：
| QoE three major types are other | Access property | Integrality | Retentivity |
| October weight coefficient | 62% | 26% | 12% |
Table 2
Step 303b：Calculate the weight accounting of the other each thin item index of QoE appraisement system three major types.
Because each thin item index is central particularly important in the scoring of QoE appraisement systems, QoE can be directly embodied
The reliability of appraisement system.
Data instance is still complained with the shared broad sense from 3900 clients in above-mentioned October, in 301
Each KPI, the weight accounting situation such as table of the other index of QoE appraisement systems three major types and each thin item index
Shown in 3：
Table 3
Specifically, the computational methods of the weight accounting of each thin item index are as follows：
(1) access property：Wireless side, core side, business side have 8 key indexs, count the whole network 8
The request number of times that individual index whole day occurs, each 8 total request number of times of index of index request number of times accounting are
The weight coefficient of the index.For example, RRC is created as power, 8 fingers of request number of times accounting that whole day occurs
Mark the request number of times that whole day occurs, i.e. 210581181/900759602=23.38% ≈ 23%.Similarly, remaining
7 indexs also can calculate weight coefficient according to the same manner.
(2) integrality：It is broadly divided into download class and time delay class, each accounting weight 50%.Download class main 2
The big bag of class and parcel are downloaded, each accounting weight 25%.When time delay class is mainly set up including attachment time delay, TCP
Prolong, DNS setup delays, the class of GET first packets time delay 4, each this 4 indexs of index overall delay accounting it is total
Time delay is the weight coefficient of the index.For example, this 4 index whole days of attachment time delay whole day overall delay accounting are total
Time delay, i.e. 557.31/5834.11=4.78% ≈ 5%, similarly, remaining 3 index also can be according to the same manner
Calculate weight coefficient.
(3) retentivity：Wireless side, core side have 4 key indexs, complete according to this 4 key indexs
It the per unit flow frequency of failure, the failure of each 4 indexs of index unit flow frequency of failure accounting always
Number of times is the weight coefficient of the index.For example, the wireless 4 index whole days of whole day number of dropped calls accounting that go offline are fallen
Line number of times, i.e. 271677/1313225=20.67% ≈ 21%, similarly, remaining 3 index also can be according to same
Sample loading mode calculates weight coefficient.
Step 303c：Determine external influence factor regulation coefficient f and λ.
Specifically, f is defined as festivals or holidays when ratio of the daily flow compared with first three day red-letter day non-festivals or holidays flow.
That is, f is for whether festivals or holidays are adjusted, festivals or holidays are as the term suggests be on every Saturdays all
Day and national defined the Ching Ming Festival, the festivals or holidays such as Dragon Boat Festival.
It was found that in Sunday Saturday and festive occasion among routine work, due to stream of people's Relatively centralized,
The zone of action of people produces change, causes local indexes to be deteriorated due to congestion or using concentrating, so as to draw
Play the slight degradation of the whole network index, but this part is perceived by can not be influenceed for factor.In order to more preferable
The user for assessing the whole network level perceives, it is believed that when the scoring same day is non-festivals or holidays, f=1；And it is false in section
During day, f=is when the average discharge of daily flow/first three non-festivals or holidays.
Specifically, lambda definition is the influence coefficient that same day cutover is operated.
Different departments such as core net, wireless network etc. will carry out related for network structure regulation in routine duties
Cutover operation, to optimize network performance.
(1) that night operates successfully, when not influenceing to perceive for second day, then λ=1；
(2) that night delivery failure, but success is refunded before 6 points of morning, then λ=0.9；
(3) if that night delivery failure, and refunded unsuccessfully before 6 points of morning, then λ=0.5.
The coefficient carries out the adjustment of index of correlation for that night cutover network element.If that night cutover is wireless side cutover
Operation, then all index λ=1 of each index of core side and business platform side, all indexs of wireless side are according to cutting
Depending on connecing the success or not of operation, λ=x.
Step 304：Calculate the QoE values of specific business.
The scoring of QoE quantification of targets systems is obtained using nine points of systems, each KPI indexs when reaching the scoring upper limit
It is full 9 points, 0 point is scored at when scoring lower limit, Linear Score when between bound.
Data instance is still complained with the shared broad sense from 3900 clients in above-mentioned October, with reference to step 301
In each KPI, the setting situation of the scoring upper limit of each thin item index of QoE appraisement systems and scoring lower limit is such as
Shown in table 4：
Table 4
Specifically, the scoring upper limit of each thin item index of QoE appraisement systems and the establishing method for the lower limit that scores are as follows：
(1) access property：The bound setting Primary References of 8 index success rates is daily to corresponding index
Monitor thresholding.
(2) integrality：It is divided into the index for downloading class and time delay class.Class is downloaded according to the whole network speed TOP80%
Cell downloading rate be the upper limit, the worst TOP10% of the whole network for lower limit.When parcel downloads time delay, attachment
Prolong and define upper-lower door limit value respectively 2s, 8s with reference to 258 principles.TCP setup delays, DNS setup delays,
Three indexs of GET first packets time delay are a totality, and total time delay 2 seconds is the upper limit, are within 8 seconds lower limit.It is every
Time delay is multiplied by 2s for the upper limit in the accounting of three index average delay summations, and every time delay is average in three indexs
The accounting of time delay summation is multiplied by 8s for lower limit.(this process is set equivalent to the normal distribution process of row index is entered
Determine thresholding.) as TCP set up average delay be 44.82ms, it is 23.33ms that DNS, which sets up average delay,
GET first packets response average delay 146.32ms；Then the TCP setup delays upper limit is：
44.82/ (44.82+23.33+146.32) * 2000=416ms=0.42s；TCP setup delay lower limits are：
44.82/ (44.82+23.33+146.32) * 8000=1671ms=1.67s.Similarly, it can be calculated according to corresponding mode
Go out DNS setup delays, GET first packet time delay bounds.
(3) retentivity：It is the upper limit, worst TOP10% to take every GB flow indicators number of times optimal T 0P20%
For lower limit.(being still the normal distribution according to data, index bound is set according to probability statistics) such as：Nothing
The every GB flows number of dropped calls optimal T OP20% of line number of dropped calls the whole network cell is worst in 4.53 times
TOP10% cells are more than 58.3 times.Therefore bound is respectively set to 4.53 times and 58.3 times.
Above-mentioned TOP refers to ranking.
The scoring event of each single item is calculated in aforementioned manners, weight coefficient is multiplied by by score calculates QoE and comment
Divide result, QoE value calculation formula are as follows：
Wherein, f and λ represents the influence coefficient of external influence factor respectively；I represents one kind in major classifications,
I=1,2,3 ..., n；K represents the thin item index under every big classification, and k is the positive integer more than or equal to 1；
δkThe weight corresponding to each thin item index under representing per major class；δiRepresent corresponding to per big classification index
Weight.
Step 305：Data verification.
Specifically, if the result meets preset standard, step 307 is performed；If the result is not
Meet preset standard, then perform step 306.
Preferably, step 305 includes：
Step 305a：The whole network level appraisal result to QoE appraisement systems is estimated.
In order to verify QoE appraisement systems to the whole network and the scoring accuracy of cell-level, using in step 304
QoE calculation formula October 10 to the whole network index on October 19 is estimated, specific score and comment
Estimate result as shown in table 5：
Table 5
According to the scoring event in table 5, more intuitive QoE appraisement systems appraisal result signal can be drawn out
Figure, Fig. 5 provides QoE appraisement system appraisal result schematic diagrames for the embodiment of the present invention, from fig. 5, it can be seen that
From the point of view of the whole network appraisal result, the client perception overall condition of the whole network is good.October 15, QoE evaluated body
Be that score is minimum, close to general, because the weight accounting of retentivity score is relatively low, then final scoring it is main by
To the score influence of access property.Verify reason is mainly influenceed by TCP success rates, and concrete reason is as follows：
October 15, morning GGSN201 was cut into after Ericsson's synthesized gateway, morning LAN1 LAN2 TCP
Success rate, which glides, triggers Level 1Alarming, and it is that portfolio is high caused that provincial company, which is suspected, by cutting away a GGSN
TCP success rates index is recovered afterwards.
Step 305b：The scoring of QoE appraisement systems is associated with complaint amount.
Specifically, step 305b includes：
The local examination item complaint amount of online on October 10 to October 19 is taken, is carried out with QoE appraisal results
Paired observation；
For example, the contrast table of complaint amount and QoE appraisement system appraisal results is as shown in table 6：
Table 6
The whole network level QoE appraisement systems appraisal result to October 10 to October 19 is locally examined with online
Core complaint amount (data are supplied to the data of control centre from customer service) is associated analysis；Through association analysis
It is -0.82 to draw coefficient correlation, represents both in relatively strong negatively correlated.
According to the scoring event in table 6, more intuitive contrast schematic diagram can be drawn out, Fig. 6 is the present invention
The contrast schematic diagram of examination item complaint amount is locally complained in the QoE appraisement systems scoring that embodiment is provided with LTE,
From fig. 6, it can be seen that when LTE locally complain examination item complaint amount it is big when, QoE appraisement systems scoring compared with
It is low；When LTE locally complains examination item complaint amount small, the scoring of QoE appraisement systems is higher.
It is therefore contemplated that the QoE Quantization Index Systems set up can accurately react client perception
Situation.
Step 306：Modifying model, is then back to step 303.
With increasing for terminal applies species, usage behavior and the custom of user constantly change, and are mapped to
Network side is that type of service can constantly change.Commented constantly to improve user terminal online QoE perception
Valency system, it is proposed that surfed the Net based on user and complain the QoE with type of service to perceive appraisement system weight coefficient certainly
Adapt to the model of adjustment, i.e., the dynamic adjustment process of each index coefficient.Thinking is as follows：
Key index in first, KPI system constantly learns to update according to business model, finds influence and perceives
Key index, realize that adaptive index system updates.
Second, can be according to first week every month for the other weight accounting of three major types in QoE appraisement systems
Complaint situation is collected, and redistributes each weight coefficient, is used as the QoE appraisement systems three of this following month
Major class weight coefficient.
3rd, can be according to access service amount model realization index weights for access property KPI weight
Adaptation coefficient, or even realize that the weight of different network elements rank is adaptively adjusted.
4th, the big bag service downloading speed higher to integrality weight accounting and packet services download time delay can
To carry out adaptive adjustment weight coefficient according to the flow accounting of big bag business and packet services；Other time delay roots
The business average delay experience counted according to existing network realizes adaptive adjustment.
Step 307：Draw QoE Quantization Index Systems.
The online of the user terminal based on the equipment index derivation QoE of final output perceives appraisement system such as table 7
It is shown：
Table 7
The Pyatyi standard of client perception is drawn according to scoring event, situation can be perceived with visual evaluation user.QoE
Appraisement system Pyatyi is evaluated as shown in table 8：
Table 8
Certainly, above-mentioned Pyatyi standard can also carry out adaptive tune with the change of QoE evaluation system models
It is whole.
Embodiment three
Fig. 7 is the composition structural representation of user provided in an embodiment of the present invention online QoE evaluating apparatus, such as
Shown in Fig. 7, user's online QoE evaluating apparatus includes including described device：
Collector unit 71, the KPI data for collecting network element；
Map unit 72, the mapping for setting up KPI to QoE according to collected KPI data；
Unit 73 is set up, for setting up QoE evaluation system models based on mapping result.
Preferably, the map unit 72, is additionally operable to：
The KPI data being collected into is classified according to access property, integrality, retentivity；
The KQI of the major classifications of setting influence；
KQI based on the major classifications of influence determines the thin item index of the major classifications of influence.
Preferably, it is described to set up unit 73, it is additionally operable to：
Analyze the weight accounting of major classifications；
The weight accounting of the thin item index of the major classifications of analyzing influence；
Set external influence factor regulation coefficient；
According to the weight accounting of major classifications, the weight accounting of the thin item index of the major classifications of influence, with
And the external influence factor regulation coefficient determines the QoE value calculation formula of QoE evaluation system models.
Preferably, described device also includes：Computing unit 74, is used for：
When being scored according to the QoE values calculation formula QoE appraisement systems, made using nine points, often
Individual index obtains 9 points when reaching the scoring upper limit, and 0 point, when between bound is scored at when scoring lower limit
Linear Score.
Preferably, described device also includes：Amending unit 75, is used for：
Count the type of service delta data in preset time period；
The type of service delta data is analyzed；
The index included based on analysis result to the QoE evaluation system models and the weight system of each index
Number, external influence factor regulation coefficient carry out adaptivity amendment.
In practical application, the collector unit 71, map unit 72, unit 73, computing unit 74 are set up
With amending unit 75 can be surfed the Net by user QoE evaluating apparatus central processing unit in the network device
(CPU, Central Processing Unit), microprocessor (MPU, Micro Processor Unit), number
Word signal processor (DSP, Digital Signal Processor) or field programmable gate array (FPGA,
Field Programmable Gate Array) etc. realize.
User that the present embodiment is provided online QoE evaluating apparatus, can more comprehensively, more accurately obtain user
The use of online is perceived, and realizes efficient, the intelligent, standardization for being surfed the Net to user and perceiving and evaluating.
In several embodiments provided by the present invention, it should be understood that disclosed method, equipment and be
System, can be realized by another way.Apparatus embodiments described above are only schematical, example
Such as, the division of the unit, only a kind of division of logic function can have other draw when actually realizing
The mode of dividing, such as：Multiple units or component can be combined, or be desirably integrated into another system, or some spies
Levying to ignore, or does not perform.In addition, the coupling each other of shown or discussed each part,
Or direct-coupling or communication connection can be by some interfaces, the INDIRECT COUPLING or communication of equipment or unit
Connection, can be electrical, machinery or other forms.
The above-mentioned unit illustrated as separating component can be or may not be it is physically separate, as
The part that unit is shown can be or may not be physical location, you can positioned at a place, also may be used
To be distributed on multiple NEs；Part or all of unit therein can be selected according to the actual needs
Realize the purpose of this embodiment scheme.
In addition, each functional unit in various embodiments of the present invention can be fully integrated into a processing unit,
Can also be each unit individually as a unit, can also two or more units be integrated in one
In individual unit；Above-mentioned integrated unit can both be realized in the form of hardware, it would however also be possible to employ hardware adds soft
The form of part functional unit is realized.
One of ordinary skill in the art will appreciate that：Realize that all or part of step of above method embodiment can
To be completed by the related hardware of programmed instruction, foregoing program can be stored in an embodied on computer readable and deposit
In storage media, the program upon execution, performs the step of including above method embodiment；And foregoing storage
Medium includes：Movable storage device, read-only storage (ROM, Read-Only Memory), magnetic disc or
Person's CD etc. is various can be with the medium of store program codes.
Or, if the above-mentioned integrated unit of the embodiment of the present invention is realized and made in the form of software function module
For independent production marketing or in use, it can also be stored in a computer read/write memory medium.Base
Understand in such, what the technical scheme of the embodiment of the present invention substantially contributed to prior art in other words
Part can be embodied in the form of software product, and the computer software product is stored in a storage medium
In, including some instructions to cause a computer equipment (can be personal computer, server or
Person's network equipment etc.) perform all or part of each of the invention embodiment methods described.And foregoing storage
Medium includes：Movable storage device, ROM, magnetic disc or CD etc. are various can be with Jie of store program codes
Matter.
The foregoing is only a preferred embodiment of the present invention, is not intended to limit the protection model of the present invention
Enclose.Any modifications, equivalent substitutions and improvements made within the spirit and principles of the invention etc., all should
Within protection scope of the present invention.
Claims (10)
1. a kind of user's online experience quality Q oE evaluation methods, it is characterised in that methods described includes：
Collect the Key Performance Indicator KPI data of network element；
KPI to QoE mapping is set up according to collected KPI data；
QoE evaluation system models are set up based on mapping result.
2. according to the method described in claim 1, it is characterised in that described based on collected KPI numbers
According to the mapping for setting up KPI to QoE, including：
The KPI data being collected into is classified according to access property, integrality, retentivity；
The Key Quality Indicator KQI of the major classifications of setting influence；
KQI based on the major classifications of influence determines the thin item index of the major classifications of influence.
3. method according to claim 2, it is characterised in that described that QoE is set up based on mapping result
Evaluation system model, including：
Analyze the weight accounting of major classifications；
The weight accounting of the thin item index of the major classifications of analyzing influence；
Set external influence factor regulation coefficient；
According to the weight accounting of major classifications, the weight accounting of the thin item index of the major classifications of influence, with
And the external influence factor regulation coefficient determines the QoE value calculation formula of QoE evaluation system models.
4. method according to claim 3, it is characterised in that methods described also includes：
When being scored according to the QoE values calculation formula QoE appraisement systems, made using nine points, often
Individual index obtains 9 points when reaching the scoring upper limit, and 0 point, when between bound is scored at when scoring lower limit
Linear Score.
5. method according to claim 3, it is characterised in that described that QoE is set up based on mapping result
After evaluation system model, methods described also includes：
Count the type of service delta data in preset time period；
The type of service delta data is analyzed；
The index included based on analysis result to the QoE evaluation system models and the weight system of each index
Number, external influence factor regulation coefficient carry out adaptivity amendment.
The QoE evaluating apparatus 6. a kind of user surfs the Net, it is characterised in that described device includes：
Collector unit, the KPI data for collecting network element；
Map unit, the mapping for setting up KPI to QoE according to collected KPI data；
Unit is set up, for setting up QoE evaluation system models based on mapping result.
7. device according to claim 6, it is characterised in that the map unit, is additionally operable to：
The KPI data being collected into is classified according to access property, integrality, retentivity；
The KQI of the major classifications of setting influence；
KQI based on the major classifications of influence determines the thin item index of the major classifications of influence.
8. device according to claim 7, it is characterised in that described to set up unit, is additionally operable to：
Analyze the weight accounting of major classifications；
The weight accounting of the thin item index of the major classifications of analyzing influence；
Set external influence factor regulation coefficient；
According to the weight accounting of major classifications, the weight accounting of the thin item index of the major classifications of influence, with
And the external influence factor regulation coefficient determines the QoE value calculation formula of QoE evaluation system models.
9. device according to claim 8, it is characterised in that described device also includes：Computing unit,
For：
When being scored according to the QoE values calculation formula QoE appraisement systems, made using nine points, often
Individual index obtains 9 points when reaching the scoring upper limit, and 0 point, when between bound is scored at when scoring lower limit
Linear Score.
10. device according to claim 8, it is characterised in that described device also includes：Amendment is single
Member, is used for：
Count the type of service delta data in preset time period；
The type of service delta data is analyzed；
The index included based on analysis result to the QoE evaluation system models and the weight system of each index
Number, external influence factor regulation coefficient carry out adaptivity amendment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610074975.8A CN107026750B (en) | 2016-02-02 | 2016-02-02 | User Internet QoE evaluation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610074975.8A CN107026750B (en) | 2016-02-02 | 2016-02-02 | User Internet QoE evaluation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107026750A true CN107026750A (en) | 2017-08-08 |
| CN107026750B CN107026750B (en) | 2020-05-26 |
Family
ID=59524631
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610074975.8A Active CN107026750B (en) | 2016-02-02 | 2016-02-02 | User Internet QoE evaluation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107026750B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107612701A (en) * | 2016-07-11 | 2018-01-19 | 中国电信股份有限公司 | A kind of processing method of QoE parameters, device and customer experience management system |
| CN107920362A (en) * | 2017-12-06 | 2018-04-17 | 南京华苏科技有限公司 | A kind of LTE network performance estimating method based on film micro area |
| CN109428759A (en) * | 2017-09-01 | 2019-03-05 | 中国移动通信集团广西有限公司 | A kind of network quality appraisal procedure and device |
| CN110060093A (en) * | 2019-03-25 | 2019-07-26 | 广州瀚信通信科技股份有限公司 | A kind of terminal marketing method based on 4G high flow capacity client |
| CN110650488A (en) * | 2018-06-26 | 2020-01-03 | 大唐移动通信设备有限公司 | Communication quality monitoring method and system |
| CN111177280A (en) * | 2019-12-26 | 2020-05-19 | 北京亚信数据有限公司 | Data authority evaluation method and device |
| WO2020103737A1 (en) * | 2018-11-22 | 2020-05-28 | 华为技术有限公司 | Network performance bottleneck value determination method and apparatus |
| CN112422534A (en) * | 2020-11-06 | 2021-02-26 | 上海优扬新媒信息技术有限公司 | Credit evaluation method and device of electronic certificate |
| CN116542566A (en) * | 2023-05-09 | 2023-08-04 | 广东圣千科技有限公司 | Interactive scoring method and system for intelligent skin care customer service |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102158879A (en) * | 2011-02-24 | 2011-08-17 | 大唐移动通信设备有限公司 | Essential factor lost score data processing method and equipment |
| CN102625344A (en) * | 2012-03-13 | 2012-08-01 | 重庆信科设计有限公司 | Model and method for evaluating user experience quality of mobile terminal |
| CN102685791A (en) * | 2012-05-22 | 2012-09-19 | 北京东方文骏软件科技有限责任公司 | Method for evaluating user quality of experience (QoE) of WAP (Wireless Application Protocol) services by simulating user behavior |
| WO2013044997A1 (en) * | 2011-09-28 | 2013-04-04 | Telefonica, S.A. | A method to measure quality of experience of a video service |
| CN103227738A (en) * | 2013-04-26 | 2013-07-31 | 华南师范大学 | Intelligent network monitoring system based on self-similar model |
| CN103312540A (en) * | 2013-05-24 | 2013-09-18 | 中国联合网络通信集团有限公司 | User service requirement parameter determining method and device |
| CN104023232A (en) * | 2014-06-27 | 2014-09-03 | 北京邮电大学 | Mobile video quality assessment method based on hierarchy analysis and multiple linear regressions |
| CN104378220A (en) * | 2013-08-14 | 2015-02-25 | 中兴通讯股份有限公司 | Method, device, user terminal and network server for evaluating user experience quality |
| CN104618924A (en) * | 2015-01-30 | 2015-05-13 | 南京邮电大学 | Wireless ubiquitous network-based quality of experience index system and measuring method |
| WO2015144211A1 (en) * | 2014-03-25 | 2015-10-01 | Telefonaktiebolaget L M Ericsson (Publ) | Method and system for monitoring qoe |
| CN104994133A (en) * | 2015-05-22 | 2015-10-21 | 华中科技大学 | Mobile Web webpage access user experience perception evaluating method based on network KPI |
| CN105050125A (en) * | 2015-06-23 | 2015-11-11 | 武汉虹信通信技术有限责任公司 | Method and device for evaluating mobile data service quality oriented to user experience |
-
2016
- 2016-02-02 CN CN201610074975.8A patent/CN107026750B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102158879A (en) * | 2011-02-24 | 2011-08-17 | 大唐移动通信设备有限公司 | Essential factor lost score data processing method and equipment |
| WO2013044997A1 (en) * | 2011-09-28 | 2013-04-04 | Telefonica, S.A. | A method to measure quality of experience of a video service |
| CN102625344A (en) * | 2012-03-13 | 2012-08-01 | 重庆信科设计有限公司 | Model and method for evaluating user experience quality of mobile terminal |
| CN102685791A (en) * | 2012-05-22 | 2012-09-19 | 北京东方文骏软件科技有限责任公司 | Method for evaluating user quality of experience (QoE) of WAP (Wireless Application Protocol) services by simulating user behavior |
| CN103227738A (en) * | 2013-04-26 | 2013-07-31 | 华南师范大学 | Intelligent network monitoring system based on self-similar model |
| CN103312540A (en) * | 2013-05-24 | 2013-09-18 | 中国联合网络通信集团有限公司 | User service requirement parameter determining method and device |
| CN104378220A (en) * | 2013-08-14 | 2015-02-25 | 中兴通讯股份有限公司 | Method, device, user terminal and network server for evaluating user experience quality |
| WO2015144211A1 (en) * | 2014-03-25 | 2015-10-01 | Telefonaktiebolaget L M Ericsson (Publ) | Method and system for monitoring qoe |
| CN104023232A (en) * | 2014-06-27 | 2014-09-03 | 北京邮电大学 | Mobile video quality assessment method based on hierarchy analysis and multiple linear regressions |
| CN104618924A (en) * | 2015-01-30 | 2015-05-13 | 南京邮电大学 | Wireless ubiquitous network-based quality of experience index system and measuring method |
| CN104994133A (en) * | 2015-05-22 | 2015-10-21 | 华中科技大学 | Mobile Web webpage access user experience perception evaluating method based on network KPI |
| CN105050125A (en) * | 2015-06-23 | 2015-11-11 | 武汉虹信通信技术有限责任公司 | Method and device for evaluating mobile data service quality oriented to user experience |
Non-Patent Citations (2)
| Title |
|---|
| 俞俊生: "《从云到终端提升移动流媒体性能技术研究》", 30 September 2015 * |
| 高林: "《计算机公共基础——基本知识和使用》", 31 July 2000 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107612701A (en) * | 2016-07-11 | 2018-01-19 | 中国电信股份有限公司 | A kind of processing method of QoE parameters, device and customer experience management system |
| CN109428759A (en) * | 2017-09-01 | 2019-03-05 | 中国移动通信集团广西有限公司 | A kind of network quality appraisal procedure and device |
| CN107920362A (en) * | 2017-12-06 | 2018-04-17 | 南京华苏科技有限公司 | A kind of LTE network performance estimating method based on film micro area |
| CN107920362B (en) * | 2017-12-06 | 2020-12-01 | 南京华苏科技有限公司 | LTE network performance evaluation method based on micro-area |
| CN110650488A (en) * | 2018-06-26 | 2020-01-03 | 大唐移动通信设备有限公司 | Communication quality monitoring method and system |
| WO2020103737A1 (en) * | 2018-11-22 | 2020-05-28 | 华为技术有限公司 | Network performance bottleneck value determination method and apparatus |
| CN110060093A (en) * | 2019-03-25 | 2019-07-26 | 广州瀚信通信科技股份有限公司 | A kind of terminal marketing method based on 4G high flow capacity client |
| CN110060093B (en) * | 2019-03-25 | 2023-12-05 | 广州瀚信通信科技股份有限公司 | Terminal marketing method based on 4G high-flow clients |
| CN111177280A (en) * | 2019-12-26 | 2020-05-19 | 北京亚信数据有限公司 | Data authority evaluation method and device |
| CN112422534A (en) * | 2020-11-06 | 2021-02-26 | 上海优扬新媒信息技术有限公司 | Credit evaluation method and device of electronic certificate |
| CN112422534B (en) * | 2020-11-06 | 2023-09-22 | 度小满科技(北京)有限公司 | Credit evaluation method and equipment for electronic certificate |
| CN116542566A (en) * | 2023-05-09 | 2023-08-04 | 广东圣千科技有限公司 | Interactive scoring method and system for intelligent skin care customer service |
| CN116542566B (en) * | 2023-05-09 | 2023-11-21 | 广东圣千科技有限公司 | Interactive scoring method and system for intelligent skin care customer service |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107026750B (en) | 2020-05-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107026750A (en) | A kind of user's online QoE evaluation methods and device | |
| CN102158879B (en) | Essential factor lost score data processing method and equipment | |
| CN105357691B (en) | LTE wireless network user perceives monitoring method and system | |
| CN109921941B (en) | Network service quality evaluation and optimization method, device, medium and electronic equipment | |
| CN109803295B (en) | Method and device for evaluating communication cell rectification priority | |
| CN108075925B (en) | Improving performance of a communication network based on end-to-end performance observation and evaluation | |
| CN101562832B (en) | Method and system for calculating key quality index used in communication test | |
| US20030151619A1 (en) | System for analyzing network load and other characteristics of an executable application | |
| EP3035597A1 (en) | Method and device for evaluating quality of user experience, user terminal and network server | |
| CN104756467A (en) | CDN traffic management in the cloud | |
| CN108764332A (en) | A kind of Channel Quality analysis method, computing device and storage medium | |
| CN107818133A (en) | A kind of residential block network capabilities analysis method and system based on big data | |
| CN103607309B (en) | Mapping method for service KQI and QOE | |
| CN105933920A (en) | Method and device for predicting user satisfaction | |
| CN106603340A (en) | Active dial testing method and system of CDN quality | |
| CN102075366B (en) | Method and equipment for processing data in communication network | |
| CN103209422A (en) | Method for accurate spot selection of wireless local area network (WLAN) | |
| CN108495329A (en) | A kind of evaluation method and device of base station reliability | |
| CN108093427A (en) | A kind of VoLTE evaluation the quality method and system | |
| CN106612511A (en) | Wireless network throughput evaluation method and device based on support vector machine | |
| CN106535204A (en) | Service coverage quality evaluation method and device | |
| CN105790866B (en) | Base station rankings method and device | |
| CN110348745A (en) | The ranking method and device of advertising channel | |
| CN109428760B (en) | User credit evaluation method based on operator data | |
| CN111106969A (en) | Automatic evaluation method and system of Internet of things platform and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |