WO2015101024A1

WO2015101024A1 - Quality of experience evaluation method and device

Info

Publication number: WO2015101024A1
Application number: PCT/CN2014/082763
Authority: WO
Inventors: 赵静波
Original assignee: 中兴通讯股份有限公司
Priority date: 2014-01-06
Filing date: 2014-07-22
Publication date: 2015-07-09
Also published as: CN104767628A

Abstract

Disclosed are a quality of experience evaluation method and device, the method comprising: classifying users according to a user behavior characteristics index to obtain first-class users and second-class users, the first-class users having a closeness with business service equal to or greater than a preset closeness, and the second class user having a closeness with business service less than the preset closeness; performing first quality of experience (QOE) evaluation on the first-class users. The present invention improves QOE accuracy and increases the practical value.

Description

TECHNICAL FIELD The present invention relates to the field of communications, and in particular to a user experience quality assessment method and apparatus. BACKGROUND OF THE INVENTION Currently, mobile Internet services are developing rapidly, mobile Internet users are proliferating, and operators are paying attention to the online experience of mobile Internet users. For operators, when the functions between competing products are not much different, the user experience will be promoted to the core value of the product. All planning activities of operators and the planning and optimization of network services must be centered on the user experience. The user experience will be the core of wireless third-generation mobile communication technology (3G), data services, and broadband private line services. Quality of Experience (QOE) will become a hot topic in the times. How to ensure that customers provide high-quality network service quality, enhance customer recognition of the brand, and maintain customer loyalty become critical issues. For new customers, how to have more intuitive and effective persuasiveness, how to provide network service quality reports to old customers on a regular basis, so that customers can clearly know the quality of network services, and ensure that they will not be lost due to subjective factors. These are all issues that must be resolved as soon as possible in front of mobile operators. QOE is the end-to-end concept of the user. It refers to the user's subjective experience of the business and the overall performance of the system as perceived from the user's perspective. Therefore, QOE is a measure of the customer's overall satisfaction with the service provider.

QOE is closely related to and different from Quality of Service (QOS). QOS mainly reflects the concept: measurable, improved and guaranteed hardware and software features. In contrast, QOE represents the objective and subjective satisfaction of users (that is, QOE is defined at the user level, and the QOS of the network and business that the user actually feels is QOE). QOE is the subjective embodiment of QOS in the minds of users. The current QOE assessment is mainly based on QOS assessment, but QOS can't reflect the differences between users. For example: For the same QQ business service, some users like to hang on QQ all day, send and receive messages frequently, some users just Occasionally used. For users who frequently use the service or users who occasionally use the service, it is likely that the QQ business service has different expectations, which will result in the same perceived experience of different quality of service. The user-aware evaluation technology in the related art only evaluates the user's perception through the service quality, and does not consider the problem of the individual difference of the user. Currently, no effective solution has been proposed. SUMMARY OF THE INVENTION The present invention provides a method and apparatus for evaluating user experience quality, so as to at least solve the problem that the user-aware evaluation technology in the above related art only evaluates the user's perception through the service quality, and does not consider the individual difference of the user. According to an aspect of the present invention, a user experience quality evaluation method is provided, including: classifying a user according to a user's behavior characteristic index, and obtaining a first type user and a second type user, wherein the first type user is a service The user with the closeness of the service is greater than or equal to the preset degree of closeness, and the second type of user is the user whose affinity with the business service is less than the preset degree; the first user experience quality (QOE) assessment is performed for the first type of user. Preferably, the preset closeness includes at least one of the following factors: the number of times the user uses the service service, the time when the user uses the service service, the frequency at which the user uses the service service, and the traffic that the user uses the service service. Preferably, the second type of user is subjected to the second QOE evaluation; or the second type of user is filtered out, and the second type of user is no longer evaluated for QOE. Preferably, before classifying the user according to the behavior characteristic indicator of the user, the method includes: analyzing the behavior characteristic of the user, and acquiring the behavior characteristic indicator, wherein the behavior characteristic indicator can reflect the behavior characteristic. Preferably, the behavior feature comprises at least one of: browsing a service using a webpage, using an instant messaging service, using a microblogging service, using an online video service, and using an online game service. Preferably, when the behavior characteristic is that the webpage browsing service is used, the behavior characteristic indicator includes at least one of the following: a download rate, a homepage open delay, a full webpage downloading rate, and a homepage response delay. According to another aspect of the present invention, a user experience quality evaluation apparatus is provided, including: a classification module, configured to classify a user according to a behavior characteristic indicator of the user, to obtain a first type user and a second type user, wherein, A type of user is a user whose affinity with the business service is greater than or equal to a preset degree, and the second type of user is a user whose affinity with the business service is less than a preset degree; the evaluation module is set to perform the first type of user. First user experience quality QOE assessment. Preferably, the preset closeness includes at least one of the following factors: the number of times the user uses the service service, the time when the user uses the service service, the frequency at which the user uses the service service, and the traffic that the user uses the service service. Preferably, the evaluation module is further configured to perform a second QOE evaluation on the second type of user; the apparatus further includes: a filtering module, configured to filter out the second type of users, and no longer perform QOE evaluation on the second type of users. Preferably, the behavior feature comprises at least one of: browsing a service using a webpage, using an instant messaging service, using a microblogging service, using an online video service, and using an online game service. Through the invention, the user who uses the service service is classified, and the QOE evaluation is performed on different types of users who are different from the service service, and the user perception evaluation technology in the related technology is only used to evaluate the user perception through the service quality. The problem of individual differences of users is not considered. This kind of evaluation method can take into account the individual differences of users, making the results of perceptual evaluation more targeted, and thus achieving the effect of improving the practical value of perceptual evaluation. BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are set to illustrate,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, 1 is a flowchart of a user experience quality evaluation method according to an embodiment of the present invention; FIG. 2 is a structural block diagram of a user experience quality evaluation apparatus according to an embodiment of the present invention; FIG. 3 is a preferred embodiment according to an embodiment of the present invention. FIG. 4 is a schematic diagram of a user-aware evaluation process according to a preferred embodiment of the present invention; FIG. 5 is a schematic diagram of a user-number distribution of webpage browsing service traffic according to a preferred embodiment of the present invention. BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict. The embodiments of the present invention mainly relate to a method for performing different user experience quality evaluations on different types of users that are selected, which can classify and filter users according to user behavior characteristics and usage habits, and experience experiences of different types of users. Analyze separately. Perceptual analysis is carried out after user classification and screening. Perceptual analysis pays more attention to the individual differences of users, and perceptual evaluation is more targeted, which can greatly improve the accuracy and practical value of perceptual evaluation. This embodiment provides a user experience quality evaluation method. 1 is a flowchart of a method for evaluating user experience quality according to an embodiment of the present invention. As shown in FIG. 1, the method mainly includes the following steps (step S102-step S104): Step S102: The user is classified according to the behavior characteristic index of the user, and the first type user and the second type user are obtained, wherein the first type user is a user whose degree of closeness to the business service is greater than or equal to a preset degree, and the second The class user is a user whose degree of closeness to the business service is less than a preset degree; in step S104, the first user experience quality QOE evaluation is performed on the first class user. Through the above steps, the users who use the service service can be classified first, and then the QOE evaluation is performed on different categories of users who are different from the business service. Through such differentiated user experience quality assessment methods, the QOE evaluation can be made. It is closer to user perception, which improves the accuracy and practical value of sensory evaluation. In this embodiment, the preset closeness may include at least one of the following factors: the number of times the user uses the service service, the time when the user uses the service service, the frequency of the user using the service service, and the traffic of the user using the service service. By setting these different consideration factors or a combination thereof, the preset closeness can be made more detailed and specific as a classification index for classifying users. Of course, in practical applications, the selection and setting of these considerations may vary depending on actual needs. In a preferred embodiment of this embodiment, the second QOE evaluation may be performed on the second type of users (of course, the second QOE evaluation may be the same as the first QOE evaluation, or may be different), or may directly be the second Class users are filtered out and no QOE evaluation is performed on the second type of users. The purpose of this is to isolate users who are very close to the current business services (such as rarely using current business services) and no longer consider such users as the object of user experience quality assessment. In this way, you can reduce The evaluation workload of the business service provider saves the evaluation cost and also makes it easier for the service provider to find the user group that they care about. In this embodiment, before performing step S102, the behavior characteristics of the user may be analyzed to obtain a behavior feature indicator, wherein the behavior feature indicator can reflect the behavior feature. The behavior characteristics may include at least one of the following: browsing a service using a webpage, using an instant messaging service, using a microblogging service, using an online audio and video service, and using an online game business. For example, when the behavior is characterized by using a web browsing service, the behavior characteristic indicator may include at least one of the following: a download rate, a homepage open delay, a full webpage download rate, and a homepage response delay. Of course, in practical applications, under different behavior characteristics of users, behavioral characteristics indicators that can reflect behavior characteristics may be different, that is, if the user's behavior characteristics are using instant messaging services, using microblogging services, and using online video and audio. Business, use of online game business, or a variety of other inexhaustible enumeration types of business, its behavior characteristics can be determined according to the characteristics of the specific business. For example, in practical applications, the above user experience quality assessment method can be implemented by the following steps: (1) analyzing the behavior characteristics of the user to obtain an indicator reflecting the behavior characteristics of the user; (2) determining the screening condition and the screening method (3) classify and filter users; (4) conduct QOE assessment for different user groups. The embodiment further provides a user experience quality evaluation apparatus, which is configured to perform the foregoing user experience quality evaluation method. FIG. 2 is a structural block diagram of a user experience quality evaluation apparatus according to an embodiment of the present invention. As shown in FIG. 2, the apparatus mainly includes: a classification module 10 and an evaluation module 20. The classification module 10 is configured to classify the user according to the behavior characteristic indicator of the user, and obtain the first type user and the second type user, wherein the first type user is closer to the business service than the preset degree of closeness. The user of the second type is a user whose degree of closeness to the business service is less than a preset degree; the evaluation module 20 is configured to perform a first user experience quality QOE evaluation on the first type of user. Of course, in practical applications, the evaluation module 20 can also be configured to perform a second QOE evaluation on the second type of users. The second QOE assessment can be used in the same assessment as the first QOE assessment, or it can be a different assessment. 3 is a structural block diagram of a preferred user experience quality evaluation apparatus according to an embodiment of the present invention. As shown in FIG. 3, the preferred user experience quality evaluation apparatus may further include: a filtering module 30 configured to filter out the second type of users, Then conduct a QOE assessment for the second type of users. In the implementation QOE evaluation process using the user experience quality evaluation device shown in FIG. 2 or the preferred user experience quality evaluation device shown in FIG. 3, the preset closeness required to use may include at least one of the following factors: The number of business services, the time the user uses the business service, the frequency with which the user uses the business service, and the traffic that the user uses for the business service. By setting these different consideration factors or a combination thereof, the preset closeness can be made more detailed and specific as a classification index for classifying users. Of course, in practical applications, the selection and setting of these considerations may vary depending on actual needs. Moreover, the behavioral features that are needed may also include at least one of the following: browsing a business using a webpage, using an instant messaging service, using a microblogging service, using an online video business, and using an online gaming business. Of course, in practical applications, under different behavior characteristics of users, behavioral characteristics indicators that can reflect behavior characteristics may be different, that is, if the user's behavior characteristics are using instant messaging services, using microwave services, and using online audio and video services. , using online game business, or a variety of other inexhaustible enumeration types of business, its behavior characteristics can be determined according to the characteristics of the specific business. The user experience quality evaluation method and device provided by the foregoing embodiments may first classify users who use the service service, and then perform QOE evaluation on different types of users who are different from the service service. The evaluation method can take into account the individual differences of the users, making the perceptual evaluation results more targeted and improving the practical value of the perceptual evaluation. The user experience quality evaluation method and apparatus provided by the foregoing embodiments are described or illustrated in more detail below with reference to FIG. 4 and FIG. 5 and a preferred embodiment. The preferred embodiment mainly provides a method for performing different QOE evaluations on different user groups by user screening operations. In the following, the user belongs to different user groups for a brief introduction. Take Weibo as an example. If user A only logs in to Weibo one day, he quits, and User B has been refreshing and publishing Weibo in one day. User Wei's Weibo experience has little or no impact on Weibo's experience, and User B's perceived experience has a greater relationship with Weibo's experience. In the preferred embodiment described below, the implementation goal to be achieved is to filter the user A or simply perform a simple QOE evaluation on the user A by setting the threshold of the number of microblog refresh times, so that the operator has more Pay attention to the perception of user B (this part of users can also be called effective users of Weibo business). The implementation process of the preferred embodiment requires the following evaluation system or function module, which is briefly introduced as follows: A. QOS index system, an indicator system based on business key performance indicators and network key performance indicators; B, QOE evaluation system, A certain algorithm calculates the QOS indicator of the user, and evaluates the user QOE; C. Sets the module to perform user filtering (ie, performs user classification), and can filter and classify the user according to the behavior characteristics of the user. The Execution User Screening module is between the QOS indicator system and the QOE evaluation system. In order to facilitate the evaluation of the perception of different user groups, the screening conditions and screening methods can be set according to the actual situation. 4 is a schematic diagram of a user-aware evaluation process according to a preferred embodiment of the present invention. As shown in FIG. 4, the processing steps of the user-aware evaluation process are as follows: Step 1: Obtain a QOS-aware indicator of a network user by using a common method; Step: Filter and classify network users. The screening method can be determined according to actual needs (for example, users can be classified according to user behavior characteristics). Step 3: Perform QOE perception evaluation on the filtered user groups. It should be noted that the execution of the first step and the second step described above does not need to distinguish the order. FIG. 5 is a schematic diagram showing the distribution of the number of users of the web browsing service traffic according to a preferred embodiment of the present invention. The preferred embodiment is further described by taking the web browsing service as an example in conjunction with FIG. 5: The process of QOE for web browsing service traffic users mainly includes: Step 1: Obtain relevant indicators for service quality assessment of web browsing service, generally including download rate, homepage open delay, full download rate of webpage, home response delay, etc.; Steps: Analyze the user traffic of the web browsing service. If the number of traffic users is as shown in Figure 5, the number of users with web browsing traffic of 60-70M is the largest, and most of the user traffic is between 20-100M. Then we can divide users into 3 categories, users with traffic less than 20M, users with traffic between 20-100M, and users with traffic greater than 100M. The third step is to make a perceptual assessment of the distribution of these three types of people. For users whose traffic is less than 20M, whether the perception is poor or not results in less traffic, and whether it is necessary to take awareness improvement measures. Evaluate the user perception of traffic between 20-100M, understand the perception of most users, and have a comprehensive understanding of the entire network. Evaluate the user perception of traffic greater than 100M. This part of the user is also the user who brings the most commercial value to the operator. The user's perception of this part of the user is grasped, and the user perception is guaranteed to bring more benefits to the operator. More business value. As can be seen from the above example, the perceptual assessment is more targeted by distinguishing between the user's behavioral characteristics and individual differences. Improve the practical value of sensory assessment. It has important significance for actual network operation. The following takes the QQ service as an example to introduce the application of user filtering in locking effective users: For mobile Internet QQ users, there are many different users, the first type of users prefer to use mobile terminal QQ chat; the second type of users only occasionally move The QQ chat on the terminal; the third type of user just installs the QQ client on the mobile terminal, and can test whether it can log in successfully and chat, and there is almost no use later. The QQ business data of these types of users can be captured in massive network data. For operators, the first type of users are real QQ business effective users. If the quality of QQ service services declines, it will have a greater impact on the perception of this group of people; for the second and third types of users, QQ The quality of service of the business has little impact on them. The operator wants to understand the perception of effective users of the QQ service. The following describes how to score effective users of QQ business by screening. The first step is to obtain the QQ service indicator of the mobile Internet user. The second step is to set the QQ service login threshold and the QQ service transmission/reception threshold. The third step is to filter out the QQ service login times exceeding the threshold and the QQ service transmission/reception times. Users who exceed the threshold, this part of the user is a valid user of the QQ business; The fourth step is to evaluate the QQ service perception of the valid QQ user based on the user's QQ service indicator. As can be seen from the above example, by setting a threshold, the invalid users of the QQ service are filtered out, and the data of the invalid users is avoided to affect the user's QQ service group perception evaluation result. Make the perceived evaluation of QQ business more targeted. At the same time, such screening can filter out the garbage data that may exist in the network. In the massive data obtained by the actual operator, there is inevitably garbage data. Taking the QQ service as an example, there is a possibility that data loss may occur. Reasonable circumstances, such as sending and receiving message data, but no login data. This part of the data is called junk data. The garbage data can be filtered out by setting filter conditions. Improve the scientific nature of perception assessment. The following example illustrates the application of user filtering in perceptual evaluation to affect the application of perceptual evaluation algorithms: Take Sina Weibo as an example, Sina Weibo business uses refresh and release operations. Users who don't use have different preferences. Some users like to browse, but rarely publish microblogs. The operations involved in browsing Weibo are only refresh operations. Some users are active users of Weibo, often publishing Weibo, and also Like to browse Weibo, the Weibo operations involved in this part of the user are refreshed and released. For these two types of users, the Weibo operations involved are different because of different usage habits. These two types of users should be scored with different indicators. For users who like to browse Weibo, only use Weibo to refresh relevant indicators for sensory evaluation. For users who like to post Weibo and browse Weibo, Weibo refresh indicators and Weibo release indicators are required to participate in the perceptual evaluation. The actual perceptual evaluation process is as follows: The first step is to obtain the user microblog business usage data; the second step is to examine the user microblog refresh and microblog publishing times, and analyze the proportion of user microblog refresh times and user publishing times; Steps: Set the refresh threshold and the release ratio threshold. The fourth step is to use the microblog refresh indicator to participate in the microblog perception evaluation for the user whose release ratio is less than the threshold, and only use the refresh ratio less than the threshold. Use Weibo to publish indicators to participate in Weibo perception assessment. For users whose refresh ratio and release ratio exceed the threshold, the refresh indicator and the release indicator participate in the Weibo perception assessment. In the above examples, in the existing perceptual evaluation method, since the judging step of the user behavior feature is added, the perceptual evaluation index is more targeted, so that the accuracy of the perceptual evaluation can be greatly improved. It should be noted that the above several methods are only described by way of example, and the user screening method that can be used in practice is not limited thereto. The general idea of the preferred embodiment is to classify and filter users according to the behavior characteristics of the user. The evaluation enhances the practical value of the perceptual evaluation by taking into account the individual differences of the users and making the perceptual evaluation results more targeted. It should be noted that each of the above modules can be implemented by hardware. For example: a processor, including the above modules, or each of the above modules is located in one processor. In another embodiment, a software is provided that is configured to perform the technical solutions described in the above embodiments and preferred embodiments. In another embodiment, a storage medium is provided, the software being stored, including but not limited to: an optical disk, a floppy disk, a hard disk, a rewritable memory, and the like. From the above description, it can be seen that the present invention achieves the following technical effects: By using the method of the present invention, compared with the existing QOE evaluation, it is convenient to evaluate the perception of different user groups by distinguishing the differences in individual user behaviors. Operators screen out the groups that they want to focus on according to actual needs, and improve the accuracy and practical value of perception assessment. Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general-purpose computing device, which can be concentrated on a single computing device or distributed over a network composed of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device, such that they may be stored in the storage device by the computing device and, in some cases, may be different from the order herein. The steps shown or described are performed, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps are fabricated as a single integrated circuit module. Thus, the invention is not limited to any specific combination of hardware and software. The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes can be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention. INDUSTRIAL APPLICABILITY As described above, a user experience quality evaluation method and apparatus provided by an embodiment of the present invention have the following beneficial effects: By distinguishing the differences in individual user behaviors, the perception of different user groups is evaluated, which facilitates operators according to actual needs. Screen out the groups that you want to focus on, and improve the accuracy and practical value of the perception assessment.

Claims

Claim

A user experience quality evaluation method, comprising: classifying a user according to a user behavior characteristic index, and obtaining a first type user and a second type user, wherein the first type user is more closely related to the business service than Or a user equal to a preset degree of closeness, the second type of user being a user who is closer to the business service than the preset degree of closeness;

Performing a first user experience quality QOE assessment for the first type of users.

2. The method according to claim 1, wherein the preset closeness comprises at least a number of times the user uses the service service, a time when the user uses the service service, and a user uses the service service. Frequency, the traffic that the user uses for the business service.

3. The method according to claim 1, wherein the method further comprises:

Performing a second QOE evaluation on the second type of users; or filtering out the second type of users, and no longer performing QOE evaluation on the second type of users.

The method according to any one of claims 1 to 3, wherein before classifying the user according to the user's behavior characteristic index, the method comprises:

The behavior characteristics of the user are analyzed to obtain the behavior characteristic indicator, wherein the behavior characteristic indicator can reflect the behavior characteristic.

The method according to claim 4, wherein the behavior feature comprises at least one of: using a web browsing service, using an instant messaging service, using a microblogging service, using an online video service, and using an online gaming service.

The method according to claim 5, wherein, when the behavior characteristic is the using a webpage browsing service, the behavior characteristic indicator comprises at least one of the following:

Download rate, home page open delay, full page download rate, home page response delay.

7. A user experience quality assessment device, comprising:

The classification module is configured to classify the user according to the behavior characteristic indicator of the user, and obtain the first type user and the second type user, wherein the first type user is closer to the business service than the preset degree The user, the second type of user is a user whose degree of closeness to the service service is less than the preset degree; and the evaluation module is configured to perform a first user experience quality QOE evaluation on the first type of user.

8. The apparatus according to claim 7, wherein the preset closeness includes at least a following factor

The number of times the user uses the service service, the time when the user uses the service service, the frequency with which the user uses the service service, and the traffic that the user uses the service service.

The device according to claim 7, wherein the evaluation module is further configured to perform a second QOE evaluation on the second type of user; the device further includes: a filtering module, configured to set the second Class users are filtered out, and QOE evaluation is no longer performed on the second type of users.

The apparatus according to any one of claims 7 to 9, wherein the behavior characteristic comprises at least one of: browsing a service using a webpage, using an instant messaging service, using a microblogging service, using an online audio and video service, using Online game business.