CN107026750B

CN107026750B - User Internet QoE evaluation method and device

Info

Publication number: CN107026750B
Application number: CN201610074975.8A
Authority: CN
Inventors: 韩笑
Original assignee: China Mobile Group Guangdong Co Ltd
Current assignee: China Mobile Group Guangdong Co Ltd
Priority date: 2016-02-02
Filing date: 2016-02-02
Publication date: 2020-05-26
Anticipated expiration: 2036-02-02
Also published as: CN107026750A

Abstract

The invention discloses a method for evaluating user on-line experience quality (QoE), which comprises the following steps: collecting Key Performance Indicator (KPI) data of a network element; establishing a KPI to QoE mapping based on the collected KPI data; and establishing a QoE evaluation system model based on the mapping result. The invention also discloses a device for evaluating the QoE of the user on the Internet. By adopting the technical scheme of the invention, the use perception of the user on the Internet can be more accurately obtained.

Description

User Internet QoE evaluation method and device

Technical Field

The invention relates to a user perception analysis technology in the technical field of communication, in particular to a method and a device for evaluating the Quality of Experience (QoE) of a user on the internet.

Background

The client perception is the subjective perception of a user using a network, and at present, three methods are commonly used in actual network operation and maintenance, namely a client sampling survey statistical method, a current network service index dial-test method and a traditional method for evaluating the client perception by a single network operation and maintenance index.

The customer sampling survey statistical method mostly adopts a questionnaire survey form to collect the perception situation of the customer. However, this evaluation method is heavy in workload, inefficient, and not easy to repeat the measurement. When different networks are evaluated, the evaluation method has large fluctuation and weak comparability between the networks due to difference between regions and humanity.

The present network service index dial-up test method can measure the index condition of any network without a network management system and other operation and maintenance systems, and directly obtain the use perception result of the network, and has the advantages of independence from equipment manufacturers, independence from a bearing layer to a great extent, flexible test process and the like. However, this evaluation method requires a specified place to be reached, which is labor and material intensive, and the test result represents only the local area index condition, and is not universal.

In a traditional method for evaluating customer perception by using a single network operation and maintenance index, a Key Performance Indicator (KPI) of network operation is set based on an operation angle and can be obtained through a network management system. The evaluation method uses independent indexes such as wireless call completing rate, attachment success rate and the like to evaluate the use perception of the user on the internet by the terminal. However, the evaluation method can observe the condition of each index in a subtle way, and the sensitivity of the user to the terminal use perception is far lower than that of the network equipment, so that the operation and maintenance are concentrated on processing the alarm of the relevant equipment, and the influence degree of each index on the user cannot be accurately described. With the continuous expansion of network scale and the continuous update of communication technology, the workload of equipment operation and maintenance is increased, and the labor cost is relatively stable. Therefore, the evaluation method is not beneficial to the evaluation of user perception and the improvement of the working efficiency.

Disclosure of Invention

In view of this, the present invention is intended to provide a method and an apparatus for evaluating a user internet QoE, which can more accurately obtain a user perception of internet access.

In order to achieve the purpose, the technical scheme of the invention is realized as follows:

the invention discloses a user Internet QoE evaluation method, which comprises the following steps:

collecting KPI data of network elements;

establishing KPI to QoE mapping according to the collected KPI data;

and establishing a QoE evaluation system model based on the mapping result.

In the foregoing solution, preferably, the establishing a KPI to QoE mapping based on the collected KPI data includes:

classifying the collected KPI data according to accessibility, completeness and retentivity;

setting Key Quality Indicators (KQI, Key Quality Indicators) influencing each large category;

and determining detailed item indexes influencing each large category based on the KQI influencing each large category.

In the foregoing solution, preferably, the establishing a QoE evaluation system model based on a mapping result includes:

analyzing the weight ratio of each large category;

analyzing the weight ratio of detailed indexes influencing each large category;

setting an external influence factor adjustment coefficient;

and determining a QoE value calculation formula of the QoE evaluation system model according to the weight ratio of each large category, the weight ratio of detailed item indexes influencing each large category and the external influence factor adjustment coefficient.

In the foregoing solution, preferably, the method further includes:

and when the QoE evaluation system is evaluated according to the QoE value calculation formula, a nine-point system is adopted, each index is 9 points when reaching the upper limit of the evaluation, 0 point when reaching the lower limit of the evaluation, and the linear evaluation is carried out between the upper limit and the lower limit.

In the foregoing solution, preferably, after the QoE evaluation system model is established based on the mapping result, the method further includes:

counting the service type change data in a preset time period;

analyzing the service type change data;

and performing adaptive correction on the indexes contained in the QoE evaluation system model, the weight coefficient of each index and the external influence factor adjusting coefficient based on the analysis result.

The invention also discloses a QoE evaluation device for user online, which comprises:

the collecting unit is used for collecting KPI data of the network element;

a mapping unit for establishing KPI to QoE mapping according to the collected KPI data;

and the establishing unit is used for establishing a QoE evaluation system model based on the mapping result.

In the foregoing solution, preferably, the mapping unit is further configured to:

setting KQI affecting each large category;

In the foregoing solution, preferably, the establishing unit is further configured to:

analyzing the weight ratio of each large category;

analyzing the weight ratio of detailed indexes influencing each large category;

setting an external influence factor adjustment coefficient;

In the foregoing solution, preferably, the apparatus further includes: a computing unit to:

In the foregoing solution, preferably, the apparatus further includes: a correction unit for:

counting the service type change data in a preset time period;

analyzing the service type change data;

The user Internet QoE evaluation method and device provided by the invention collect key performance index KPI data of network elements; establishing KPI to QoE mapping according to the collected KPI data; and establishing a QoE evaluation system model based on the mapping result. Therefore, the technical scheme of the invention can more comprehensively and accurately obtain the use perception of the user on the Internet, and realize high efficiency, intellectualization and standardization of the perception evaluation of the user on the Internet.

Drawings

Fig. 1 is a flowchart illustrating an implementation of a method for evaluating a QoE of a user for internet access according to an embodiment of the present invention;

fig. 2 is a schematic diagram of a KPI to QoE mapping according to an embodiment of the present invention;

fig. 3 is a flowchart of a specific implementation of establishing a user internet QoE evaluation system according to an embodiment of the present invention;

fig. 4 is a weight proportion diagram of three categories of QoE evaluation systems obtained according to one month of statistical data according to an embodiment of the present invention;

fig. 5 is a schematic diagram of a QoE evaluation system scoring result provided in the embodiment of the present invention;

fig. 6 is a schematic diagram illustrating comparison between QoE evaluation system scores and complaint amounts of LTE local complaint assessment items provided by an embodiment of the present invention;

fig. 7 is a schematic structural diagram illustrating a composition of a device for evaluating a QoE of a user for internet access according to an embodiment of the present invention.

Detailed Description

So that the manner in which the features and aspects of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings.

Fig. 1 is a flowchart illustrating an implementation of a method for evaluating a QoE of a user accessing internet according to an embodiment of the present invention, where as shown in fig. 1, the method mainly includes the following steps:

step 101: key Performance Indicator (KPI) data of a network element is collected.

In a specific embodiment, KPI data of network elements may be collected by an existing network management system.

Specifically, according to the whole process of user internet access, from the attachment process to the Public Data Network (PDN), to the Transmission Control Protocol (TCP) connection and the final user GET or POST behavior, KPIs on the wireless side, core side, and service side in the data service process are listed one by one.

The GET action refers to requesting specified page information and returning to an entity body.

Where POST behavior refers to requesting that the server accept the specified document as a new subordinate entity to the identified Uniform Resource Identifier (URI).

Step 102: a KPI to QoE mapping is established based on the collected KPI data.

Preferably, the establishing a KPI to QoE mapping based on the collected KPI data may comprise:

Specifically, when determining the detailed item index affecting each large category based on the KQI affecting each large category, the following factors may be considered:

by the number of application times, the size of time delay and the ratio of the times of disconnection or switching.

Fig. 2 shows a schematic diagram of mapping of KPI to QoE, as shown in fig. 2, QoE is user perception, KPI is network performance index, and KQI is introduced to establish a mapping relationship between the two. That is, the conventional mapping established by the QoE evaluation system is the process of KPI → KQI → QoE.

From the network maintenance perspective, the complaint information of the client which most directly reflects the use condition of the network by the client is the complaint information of the client; from the perspective of user perception, the method mainly relates to the accessibility, integrity and retentivity of the network; thus, complaint information can be classified into three major categories, access, integrity, and retention.

Through analysis of the complaint information of the user, corresponding KQI and algorithms of various KQI indexes can be set for each large category by combining with various links of the use process of the internet service of the user. It should be noted that KQI is a set of KPI indicators that can be measured.

Specifically, the KQI index and index meaning corresponding to each large category can be referred to table 1.

TABLE 1

Step 103: and establishing a QoE evaluation system model based on the mapping result.

Preferably, the establishing a QoE evaluation system model based on the mapping result may include:

analyzing the weight ratio of each large category;

analyzing the weight ratio of detailed indexes influencing each large category;

setting an external influence factor adjustment coefficient;

Specifically, the QoE value calculation formula is as follows:

wherein f and lambda respectively represent influence coefficients of external influence factors; i represents one of the major categories, i ═ 1, 2, 3, …, n; k represents a fine term index under each large category, and k is a positive integer greater than or equal to 1; delta_kRepresenting the weight corresponding to each detailed index under each large class; delta_iAnd representing the weight corresponding to each large category index.

In the foregoing solution, preferably, the method further includes:

The mapping to the network side is that the service type will change continuously as the use behavior and habit of the user change continuously; therefore, in order to continuously improve a QoE perception evaluation system for user internet access, after a QoE evaluation system model is established based on the mapping result, the method further includes:

counting the service type change data in a preset time period;

analyzing the service type change data;

Therefore, the QoE perception evaluation system can be continuously improved according to the use habits of the user.

The method for evaluating the user internet QoE in the embodiment at least has the following advantages:

firstly, KPI data of network elements can be collected through a network management system, measurement and monitoring are convenient based on the existing network, index data can be fed back in time, and the method is more suitable for daily monitoring optimization and monitoring and promotion of customer perception of KPI indexes based on the network operation and maintenance management system;

secondly, calculating KPI data of equipment in network operation and maintenance to obtain the weight of the indexes, and calculating through a set KPI-KQI-QoE mapping model to obtain a user online perception QoE scoring result; compared with the traditional method for evaluating the perception of the client by using a single network operation and maintenance index, the QoE scoring result directly reflects the perception condition of the user;

thirdly, the establishment of the QoE evaluation system integrates the factors of the whole flow of the wireless, core and service platforms, and the factors which have important influence on the customer perception in the network operation and maintenance can be more accurately positioned through the distribution of the weight;

fourthly, compared with the traditional method for evaluating the client perception by a single network operation and maintenance index, the QoE evaluation system can be adjusted more flexibly to adapt to the development of the network.

Example two

Fig. 3 is a flowchart of a specific implementation of establishing a user internet QoE evaluation system according to an embodiment of the present invention, and as shown in fig. 3, the flowchart mainly includes the following steps:

step 301: the KPIs are refined from the network performance and step 302 is then performed.

When refining the KPI, factors such as the number of times of application, the size of delay, and the ratio of the times of disconnection or switching can be considered.

Preferably, the KPI may include: indexes of a wireless side, a core side and a service side in a data service flow;

wherein, for the accessibility category, the wireless side index comprises: RRC establishment success rate and E-RAB establishment success rate; the core-side indicators include: an attachment success rate and a service request success rate; the SP-side indices include: TCP establishment success rate, DNS analysis success rate, GET success rate and POST success rate;

wherein, for the integrity category, the wireless side index includes: large packet download rate and small packet download delay; the core-side indicators include: attachment delay, TCP establishment delay and DNS analysis delay; the SP-side indices include: GET first packet delay;

wherein, for the category of retentivity, the wireless-side indicators include: the wireless disconnection times of each GB flow, the E-RAB disconnection times of each GB flow, and the (same frequency/different frequency) switching failure times of each GB flow; the core-side indicators include: number of TAU failures per GB flow.

Wherein SP is an abbreviation of Service Provider, wherein the name is a Service Provider; RRC is an abbreviation of radio resource Control, wherein the literal name is radio resource Control; E-RAB is an abbreviation of Evolved Radio Access bearer, wherein the literal name is Evolved Radio Access bearer; TCP is an abbreviation of Transmission control protocol, wherein the name is a Transmission control protocol; DNS is an abbreviation for Domain Name System, where the literal Name is the Domain Name System; TAU is an abbreviation for Tracking Area Update, wherein the literal name is Tracking Area Update; GB is used to denote a computer memory unit.

Step 302: a mapping of KPIs to qoes is established and then step 303 is performed.

Specifically, how to establish the mapping from the KPI to the QoE specifically can refer to step 102 in the first embodiment, which is not described herein again.

Step 303: the weight of each index is obtained by the analysis, and then step 304 is performed.

Preferably, step 303 comprises:

step 303 a: and calculating the weight ratio of three categories of the QoE evaluation system.

Specifically, for the weight ratio of three major categories of the QoE evaluation system, the calculation method is as follows:

counting the generalized complaint data in a period of time, and classifying the generalized complaint data;

counting the number of complaints of each category;

and calculating the ratio of the number of complaints of each type to the total number of complaints.

For example, in month 10, there were 3900 customers with broad complaints, classified by accessibility, integrity, and retention, accounting for 61.85%, 25.74%, and 12.41% of complaints. Fig. 4 is a weight proportion diagram of three categories of QoE evaluation systems obtained according to one month of statistical data according to an embodiment of the present invention.

By rounding the weight coefficients, the weight ratio of the three major categories of indexes of the QoE evaluation system can be considered as shown in table 2:

QoE three major categories	Accessibility	Integrity of	Retention property
				10 month weight coefficient	62％	26％	12％

TABLE 2

Step 303 b: and calculating the weight ratio of each detailed index of the three categories of the QoE evaluation system.

Because each detailed index is particularly important in the QoE evaluation system scoring, the reliability of the QoE evaluation system can be directly reflected.

Still taking the above generalized complaint data from 3900 clients in common in month 10 as an example, the weight proportion of the three major categories of indicators and the detailed indicators in the QoE evaluation system combined with the KPIs in 301 is shown in table 3:

TABLE 3

Specifically, the weight ratio of each detail index is calculated as follows:

(1) the access performance is as follows: the wireless side, the core side and the service side have 8 key indexes, the number of requests of 8 indexes in the whole network which occur all day is counted, and the total number of requests of each index, which accounts for the number of requests of 8 indexes, is the weight coefficient of the index. For example, the RRC establishes a success rate, and the number of requests occurring all day accounts for more than 8 indexes, i.e. 210581181/900759602 ≈ 23.38% ≈ 23%. Similarly, the weight coefficients can be calculated for the remaining 7 indexes in the same manner.

(2) Integrity: the method mainly comprises a downloading class and a time delay class, and the weight ratio of each class is 50%. The download class mainly comprises 2 types of big packets and small packets, and the ratio weight of each is 25%. The time delay class mainly comprises 4 classes of attachment time delay, TCP (transmission control protocol) establishment time delay, DNS (domain name system) establishment time delay and GET (GET first packet) time delay, and the total time delay of each index accounts for the total time delay of the 4 indexes and is a weight coefficient of the index. For example, the total time delay of the attachment time delay is greater than the total time delay of the 4 indexes, that is, 557.31/5834.11 is 4.78% ≈ 5%, and similarly, the weight coefficients of the remaining 3 indexes can be calculated in the same manner.

(3) Retention property: the wireless side and the core side have 4 key indexes in total, and according to the failure times of each unit flow of the 4 key indexes all day, the failure times of each index unit flow accounts for the total failure times of the 4 indexes and is the weight coefficient of the index. For example, the wireless dropped-line all-antenna dropped-line number is greater than 4 indexes, namely 271677/1313225 ═ 20.67% ≈ 21%, and similarly, the remaining 3 indexes may also calculate the weight coefficient in the same manner.

Step 303 c: and determining the external influence factor adjusting coefficients f and lambda.

Specifically, f is defined as the ratio of the flow on the day of the holiday to the flow on the non-holiday three days before the holiday.

That is, f is adjusted according to whether holidays are provided, and the holidays are holidays such as the Qingming festival, the Hour festival and the like which are specified by the country and are provided on six weeks every week.

In daily work we find that during saturday and holidays, the activity area of people changes due to the relative concentration of people flow, resulting in a degradation of local indicators due to congestion or concentration of use, thus causing a slight degradation of the overall network performance indicator, but this part of the perception is influenced by unforgeable factors. For better evaluation of user perception at the full-network level, we consider that when the day of scoring is a non-holiday, f is 1; and during holidays, f is the current day flow/average flow of the first three non-holidays.

Specifically, λ is defined as an influence coefficient of the cutting operation on the day.

Different departments such as a core network, a wireless network, etc. perform related cutover operations aiming at network structure adjustment in daily work so as to optimize network performance.

(1) When the late operation is successful and the perception of the next day is not influenced, the lambda is 1;

(2) when late cutover fails but the back-over succeeds before 6 am, lambda is 0.9;

(3) if late cutover fails and the back fails 6 am, λ is 0.5.

The coefficient adjusts the relevant index aiming at the cutting network element at the late time. If late cutover is the wireless side cutover operation, all indexes lambda of each index of the core side and the service platform side are 1, all indexes of the wireless side are determined according to the success or failure of the cutover operation, and lambda is x.

Step 304: and calculating to obtain the QoE value of the specific service.

And the QoE index quantification system adopts a nine-point system, each KPI can be fully divided into 9 points when reaching the upper limit of the score, 0 point when reaching the lower limit of the score, and the linear score is realized between the upper limit and the lower limit.

Still taking the above generalized complaint data from 3900 clients in common in month 10 as an example, the setting conditions of the upper score limit and the lower score limit of each detailed index of the QoE evaluation system in combination with each KPI in step 301 are shown in table 4:

TABLE 4

Specifically, the setting method of the upper score limit and the lower score limit of each detailed index of the QoE evaluation system is as follows:

(1) the access performance is as follows: the upper and lower limits of the success rate of 8 indexes are set by mainly referring to daily monitoring thresholds for the corresponding indexes.

(2) Integrity: the method is divided into indexes of a downloading class and a time delay class. The download class has an upper limit of the cell download rate according to the total network rate TOP 80%, and a lower limit of the worst total network TOP 10%. Packet download delay and attachment delay are defined by upper and lower thresholds of 2s and 8s respectively according to 258 principle. Three indexes of TCP establishing time delay, DNS establishing time delay and GET first packet time delay are an overall body, the total time delay is 2 seconds as an upper limit, and 8 seconds as a lower limit. The ratio of each time delay to the sum of the average time delays of the three indexes multiplied by 2s is the upper limit, and the ratio of each time delay to the sum of the average time delays of the three indexes multiplied by 8s is the lower limit. (the process is equivalent to the normal distribution process of indexes to set a threshold.) for example, the average time delay established by TCP is 44.82ms, the average time delay established by DNS is 23.33ms, and the average time delay of GET first packet response is 146.32 ms; the TCP establishes an upper delay bound of: 44.82/(44.82+23.33+146.32) × 2000 ═ 416ms ═ 0.42 s; the lower TCP latency limit is: 44.82/(44.82+23.33+146.32) 8000 ═ 1671ms ═ 1.67 s. Similarly, the upper and lower limits of the DNS establishment delay and the GET first packet delay can be calculated according to corresponding modes.

(3) Retention property: the optimal T0P 20% of the flow index times per GB is taken as an upper limit, and the worst TOP 10% is taken as a lower limit. (again, the normal distribution according to the data, and the upper and lower index limits are set according to probability statistics) are as follows: the optimal TOP 20% of the wireless disconnection times of the whole network cell per GB traffic is within 4.53 times, and the worst TOP 10% of the cells is more than 58.3 times. The upper and lower limits are set to 4.53 times and 58.3 times, respectively.

The TOP mentioned above refers to ranking.

The score of each item is calculated by the method, and a QoE scoring result is calculated by multiplying the score by a weight coefficient, wherein a QoE value calculation formula is as follows:

Step 305: and (6) data verification.

Specifically, if the verification result meets the preset standard, step 307 is executed; if the verification result does not meet the preset standard, step 306 is executed.

Preferably, step 305 comprises:

step 305 a: and evaluating the whole network grade grading result of the QoE evaluation system.

In order to verify the accuracy of the QoE evaluation system in scoring the entire network and the cell level, the QoE calculation formula in step 304 is used to evaluate the entire network index from 10 months and 10 days to 10 months and 19 days, and the specific score and evaluation result are shown in table 5:

TABLE 5

According to the scoring conditions in table 5, a more intuitive QoE evaluation system scoring result diagram can be drawn, fig. 5 is a QoE evaluation system scoring result diagram provided in the embodiment of the present invention, and as can be seen from fig. 5, from the overall network scoring result, the overall perception situation of the entire network of the client is good. The 10-month 15-day QoE evaluation system score is the lowest, and is close to normal, and since the weight of the retention score is relatively low, the final score is mainly influenced by the accessibility score. The checking reason is mainly influenced by the success rate of the TCP, and the specific reasons are as follows:

after the GGSN201 cuts into the Ericsson integrated gateway in the morning of 10 months and 15 days, the success rate of the LAN1\ LAN2 TCP is slid down to trigger a first-level alarm, so that the company is suspected to be caused by high traffic, and the TCP success rate index is recovered after one GGSN is cut away.

Step 305 b: and correlating the QoE evaluation system score with the complaint amount.

Specifically, step 305b includes:

taking the complaint amount of the local assessment items on the Internet from 10 months and 10 days to 10 months and 19 days, and comparing and observing the complaint amount with the QoE scoring result;

for example, the table comparing the complaint amount with the QoE evaluation system score results is shown in table 6:

TABLE 6

Carrying out correlation analysis on the scoring result of the whole-network-level QoE evaluation system from 10 months and 10 days to 10 months and 19 days and the local assessment complaint amount (data from customer service provided to a network dispatching center) on the internet; correlation analysis shows that the correlation coefficient is-0.82, which shows that the two are strongly and negatively correlated.

According to the scoring conditions in table 6, a more intuitive comparison schematic diagram can be drawn, fig. 6 is a schematic diagram for comparing the scores of the QoE evaluation system and the complaint amounts of the LTE local complaint assessment items provided by the embodiment of the present invention, and as can be seen from fig. 6, when the complaint amounts of the LTE local complaint assessment items are large, the scores of the QoE evaluation system are low; when the complaint quantity of the LTE local complaint assessment item is small, the QoE evaluation system has higher score.

Therefore, the established QoE quantitative index system can reflect the condition perceived by the customer more accurately.

Step 306: the model is modified and then returns to step 303.

With the increase of terminal application types, the use behaviors and habits of users are changed continuously, and the mapping to the network side is the service type which is changed continuously. In order to continuously improve a user terminal Internet access QoE perception evaluation system, a QoE perception evaluation system weight coefficient self-adaptive adjustment model based on user Internet access complaints and service types is provided, namely a dynamic adjustment process of each index coefficient. The idea is as follows:

firstly, key indexes in the KPI system are continuously learned and updated according to a service model, key indexes influencing perception are searched, and self-adaptive index system updating is realized.

Secondly, the weight percentage of the three major categories in the QoE evaluation system can be summarized according to the complaints of the first week of each month, and each weight coefficient is redistributed to serve as the three major categories of the QoE evaluation system in the next month.

Thirdly, aiming at the weight of the access KPI, the adaptive coefficient of the index weight can be realized according to the access traffic model, and even the adaptive adjustment of the weight at different network element levels can be realized.

Fourthly, the downloading rate of the big packet service and the downloading time delay of the small packet service with higher integrity weight ratio can be adaptively adjusted according to the traffic ratio of the big packet service and the small packet service; and other time delays realize self-adaptive adjustment according to the service average time delay experience of the current network statistics.

Step 307: and obtaining a QoE quantization index system.

The finally output user terminal internet QoE perception evaluation system based on the device index derivation method is shown in table 7:

TABLE 7

And obtaining a five-level standard of customer perception according to the score condition, and intuitively evaluating the perception condition of the user. QoE evaluation system five-level evaluation is shown in table 8:

TABLE 8

Of course, the above five-level criteria may also be adaptively adjusted according to the change of the QoE evaluation system model.

EXAMPLE III

Fig. 7 is a schematic structural diagram illustrating a composition of a user internet QoE evaluation device according to an embodiment of the present invention, and as shown in fig. 7, the user internet QoE evaluation device includes:

a collecting unit 71, configured to collect KPI data of a network element;

a mapping unit 72 for establishing KPI to QoE mapping based on the collected KPI data;

and the establishing unit 73 is used for establishing a QoE evaluation system model based on the mapping result.

Preferably, the mapping unit 72 is further configured to:

setting KQI affecting each large category;

Preferably, the establishing unit 73 is further configured to:

analyzing the weight ratio of each large category;

analyzing the weight ratio of detailed indexes influencing each large category;

setting an external influence factor adjustment coefficient;

Preferably, the apparatus further comprises: a calculating unit 74 for:

Preferably, the apparatus further comprises: a correction unit 75 for:

counting the service type change data in a preset time period;

analyzing the service type change data;

In practical applications, the collecting Unit 71, the mapping Unit 72, the establishing Unit 73, the calculating Unit 74 and the correcting Unit 75 may be implemented by a Central Processing Unit (CPU), a Micro Processing Unit (MPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), or the like in a network device where the user internet QoE evaluating apparatus is located.

The user internet QoE evaluation device provided by the embodiment can more comprehensively and accurately obtain the user internet use perception, and realize high efficiency, intellectualization and standardization of user internet perception evaluation.

In the embodiments provided by the present invention, it should be understood that the disclosed method, apparatus and system can be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, all the functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: various media that can store program codes, such as a removable Memory device, a Read-Only Memory (ROM), a magnetic disk, or an optical disk.

Alternatively, the integrated unit according to the embodiment of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or a part contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A QoE (quality of experience) evaluation method for user surfing the Internet is characterized by comprising the following steps:

collecting key performance index KPI data of network elements;

establishing KPI to QoE mapping according to the collected KPI data;

establishing a QoE evaluation system model based on the mapping result;

wherein, the establishing of the QoE evaluation system model based on the mapping result comprises:

classifying the collected KPI data according to accessibility, completeness and retentivity; analyzing the weight ratio of each large category; analyzing the weight ratio of detailed indexes influencing each large category; setting an external influence factor adjustment coefficient; determining a QoE value calculation formula of the QoE evaluation system model according to the weight ratio of each large category, the weight ratio of detailed item indexes influencing each large category and the external influence factor adjustment coefficient:

wherein f represents the ratio of the flow on the day of holiday to the flow on the non-holiday three days before holiday; λ represents an influence coefficient of the cutting operation on the same day; i represents one of the major categories, i ═ 1, 2, 3, …, n; k represents a fine term index under each large category, and k is a positive integer greater than or equal to 1; delta_kRepresenting the weight corresponding to each detailed index under each large class; delta_iRepresenting the weight corresponding to each large category index;

the analyzing the weight ratio of each large category comprises:

counting the generalized complaint data in a period of time, and classifying the generalized complaint data; counting the number of complaints of each category; calculating the ratio of the number of complaints of each type to the total number of complaints;

after the QoE evaluation system model is established based on the mapping result, the method further includes:

counting the service type change data in a preset time period;

analyzing the service type change data;

2. The method of claim 1, wherein establishing a KPI to QoE mapping based on the collected KPI data comprises:

setting key quality indexes KQI affecting each large category;

3. The method of claim 1, further comprising:

4. A user online QoE evaluation device is characterized by comprising:

the collecting unit is used for collecting KPI data of the network element;

the establishing unit is used for establishing a QoE evaluation system model based on the mapping result;

the establishing unit is further configured to:

classifying the collected KPI data according to accessibility, integrity and retentivity, and analyzing the weight ratio of each large category; analyzing the weight ratio of detailed indexes influencing each large category; setting an external influence factor adjustment coefficient; determining a QoE value calculation formula of the QoE evaluation system model according to the weight ratio of each large category, the weight ratio of detailed item indexes influencing each large category and the external influence factor adjustment coefficient:

the analyzing the weight ratio of each large category comprises: counting the generalized complaint data in a period of time, and classifying the generalized complaint data; counting the number of complaints of each category; calculating the ratio of the number of complaints of each type to the total number of complaints;

the device further comprises: a correction unit for:

counting the service type change data in a preset time period;

analyzing the service type change data;

5. The apparatus of claim 4, wherein the mapping unit is further configured to:

setting KQI affecting each large category;

6. The apparatus of claim 4, further comprising: a computing unit to: