WO2024018597A1

WO2024018597A1 - User sensory quality estimation device, user sensory quality estimation method, and program

Info

Publication number: WO2024018597A1
Application number: PCT/JP2022/028358
Authority: WO
Inventors: 勇一朗浦田; 太一河野; 則次恵木
Original assignee: 日本電信電話株式会社
Priority date: 2022-07-21
Filing date: 2022-07-21
Publication date: 2024-01-25

Abstract

This user sensory quality estimation device makes it possible to estimate unified user sensory quality between different services by comprising: a comparative evaluation value estimation unit which is configured to calculate a comparative evaluation estimation value obtained by estimating a comparative evaluation of a relative dominance relationship between a first estimation value of user sensory quality pertaining to a first communication service and a certain value on a scale of an estimation value of user sensory quality pertaining to a second communication service; and a correction unit which is configured to output, as a second estimation value, the certain value obtained by correcting the first estimation value on the scale when the comparative evaluation estimation value satisfies a prescribed condition.

Description

User experience quality estimation device, user experience quality estimation method and program

The present invention relates to a user experience quality estimation device, a user experience quality estimation method, and a program.

In recent years, with the development of information and communication technology, users are using various communication services using smartphones, tablet terminals, PCs, etc. As a result, traffic is rapidly increasing.

When providing services such as web browsing via a best effort network, the quality of service varies greatly depending on the time of day and the like. Furthermore, since a decline in service quality causes user dissatisfaction and leads to service cancellation, etc., it is important for network operators and communication service providers to understand the user experience quality when using a service. Therefore, user experience quality estimation technology has been established for the purpose of quality monitoring and quality control of communication services such as video distribution and Web browsing. For example, regarding video distribution services, ITU-T Recommendation P. 1203 (Non-Patent Document 1) has been standardized. Regarding Web browsing, a Web-Browsing QoE Estimation Model (Non-Patent Document 2) has been proposed.

Existing user experience quality estimation techniques are based on data evaluated within the target service, and it is not possible to directly compare the output values of the user experience quality estimation technology for each service. For example, Non-Patent Document 1 discloses a technology that outputs the quality of video distribution in the range of 1 (very poor) to 5 (very good), and Non-Patent Document 2 discloses a technology that outputs the quality of video distribution in the range of 1 (very bad) to 5 (very good), and in Non-Patent Document 2, the quality of web browsing is A technique for outputting in 5 to 5 has been disclosed. In a situation where both technologies are used for quality monitoring, the user experience quality for video distribution users A and B is estimated to be 3.0 and 4.0, respectively, and the user experience quality for user C for web browsing is estimated to be 3.0. Suppose that it is estimated to be 5. At this time, with respect to users A and B whose output values are compared for the same service, it can be said that user A has a lower user experience quality. However, since the output values of different services cannot be directly compared, it is difficult to determine whether user C has a lower or higher user experience quality than users A and B.

Network operators and communication service providers need to understand the user experience quality when using services across services in order to detect services with lower user experience quality through service monitoring. To this end, there is a need for a quality of user experience estimation technique that allows the estimated values of the quality of user experience for each service to be compared on the same scale.

However, as explained above, it is not possible to compare the user experience quality between different services on the same scale simply by using existing user experience quality estimation techniques together.

The present invention has been made in view of the above points, and an object of the present invention is to enable uniform estimation of user experience quality between different services.

Therefore, in order to solve the above problem, the user experience quality estimating device calculates a first estimated value of the user experience quality regarding the first communication service and a certain value in the scale of the estimated value of the user experience quality regarding the second communication service. a comparative evaluation value estimation unit configured to calculate a comparative evaluation estimated value for estimating a comparative evaluation of a relative superiority relationship with the first and a correction unit configured to output the certain value as a second estimated value obtained by correcting the estimated value to the scale.

It is possible to uniformly estimate the quality of user experience between different services.

1 is a diagram showing an example of a hardware configuration of a user experience quality estimation device 10 according to an embodiment of the present invention. 1 is a diagram illustrating an example of a functional configuration of a user experience quality estimation device 10 according to an embodiment of the present invention. 3 is a flowchart for explaining an example of a processing procedure executed by the user experience quality estimation device 10. FIG.

Embodiments of the present invention will be described below based on the drawings. FIG. 1 is a diagram showing an example of the hardware configuration of a user experience quality estimation device 10 according to an embodiment of the present invention. The user experience quality estimating device 10 in FIG. 1 includes a drive device 100, an auxiliary storage device 102, a memory device 103, a processor 104, an interface device 105, etc., which are interconnected via a bus B.

A program that implements the processing in the user experience quality estimation device 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive device 100, the program is installed from the recording medium 101 to the auxiliary storage device 102 via the drive device 100. However, the program does not necessarily need to be installed from the recording medium 101, and may be downloaded from another computer via a network. The auxiliary storage device 102 stores installed programs as well as necessary files, data, and the like.

The memory device 103 reads and stores the program from the auxiliary storage device 102 when there is an instruction to start the program. The processor 104 is a CPU, a GPU (Graphics Processing Unit), or a CPU and a GPU, and executes functions related to the user experience quality estimation device 10 according to a program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network.

In this embodiment, a case will be described in which the user experience quality of service A is used as a standard among two types of services (communication services including video distribution and web browsing) A and B. The user experience quality estimating device 10 starts operation by inputting the quality parameters of the service B whose user experience quality is to be estimated, and converts the estimated value of the quality of the service B based on the quality parameters into the estimated user experience quality of the service A. The corrected user experience quality estimated value of service B corrected on the scale is output. In order to realize such a function, the user experience quality estimation device 10 has a functional configuration as shown in FIG. 2.

FIG. 2 is a diagram showing an example of the functional configuration of the user experience quality estimation device 10 in the embodiment of the present invention. In FIG. 2, the user experience quality estimation device 10 includes a pre-correction quality estimation section 11, a correction section 12, and a comparative evaluation value estimation section 13. Each of these units is realized by one or more programs installed in the user experience quality estimation device 10 causing the processor 104 to execute the process.

The pre-correction quality estimating unit 11 inputs quality parameters that can be acquired (observable) while using service B and calculates a user experience quality value (hereinafter referred to as (referred to as the "pre-correction user experience quality estimate"). As the user experience quality estimation technology corresponding to service B, the user experience quality estimation technology of existing communication services can be applied. Examples of existing communication services include video distribution services, VR video distribution services, voice call services, Web browsing, Web conference systems, online games, and the like.

As a user experience quality estimation technology for video distribution services, ITU-T Recommendation P. 1203 defines a technique for estimating the quality of user experience, and multiple similar techniques are also implemented.

The pre-correction quality estimation unit 11 receives the recommendation P. When using the technology of Recommendation P. 1203 (that is, when service B is a video distribution service etc. to which the technology of Recommendation P. Values regarding audio bit rate and playback stop are input as quality parameters. Based on these quality parameters, the pre-correction quality estimating unit 11 outputs a user experience quality for video viewing from 1 to 5 (the larger the value, the better the quality).

When the pre-correction quality estimating unit 11 uses the technology of Non-Patent Document 2 targeting web browsing (that is, when service B is web browsing etc. to which the technology can be applied), the pre-correction quality estimating unit 11 includes the following: , the waiting time until display is input as a quality parameter. The pre-correction quality estimating unit 11 outputs the user experience quality for Web browsing from 1 to 5 (the larger the value, the better the quality) based on the quality parameter.

The pre-correction quality estimation unit 11 inputs the output value (pre-correction user experience quality estimate of service B) to the correction unit 12.

The correction unit 12 calculates, for each of a plurality of values that can be taken as the estimated user experience quality of the service A, the value and the pre-correction estimated user experience quality of the service B input from the pre-correction quality estimation unit 11. The comparative evaluation value estimation unit 13 receives the comparative evaluation estimated value outputted from the comparative evaluation value estimation unit 13 .

The comparative evaluation estimated value is a value that estimates a comparative evaluation of the relative superiority or inferiority of the user experience quality for two different services. The comparative evaluation estimate ranges, for example, from -3 (service A is worse) to +3 (service B is worse). In this embodiment, the range of the estimated comparative evaluation value is −3 to +3, and the description will be made assuming that the estimated value is 0 and the quality of both services is the same.

Input/output between the correction unit 12 and the comparative evaluation value estimating unit 13 is repeatedly performed for each of a plurality of values that can be taken as the estimated user experience quality of service A. When a predetermined condition for stopping the repetition (hereinafter referred to as "stop condition") is met, the correction unit 12 uses the estimated user experience quality of service A that was last input to the comparison evaluation value estimation unit 13. is output as an estimated value of the user experience quality of service B (hereinafter referred to as a "corrected user experience quality estimated value") in the user experience quality scale of service A.

The stopping conditions include, for example, that the number of inputs and outputs to and from the comparative evaluation value estimation unit 13 exceeds a certain number of times, and that the user experience quality of both services (service A and service B) is considered to be equivalent (comparative evaluation value estimation Possible reasons include that the absolute value of the comparative evaluation estimated value output from the unit 13 is sufficiently small, or that the amount of change in the estimated user experience quality of service A is below a certain level. This prevents unintended and unnecessary repeated calculations, and obtains an estimated user experience quality of service A that is equivalent to the uncorrected user experience quality estimate of service B. An estimated value of perceived quality is obtained.

As a simple operation example, the correction unit 12 increases the user experience quality value x of service A by 0.01, for example, from the minimum value MOS _MIN to the maximum value MOS _MAX that can be taken as the estimated user experience quality of service A. , the user experience quality value x of service A when the absolute value of the comparative evaluation estimate obtained when inputting to the comparative evaluation value estimating unit 13 together with the pre-correction user experience quality estimate of service B is the corrected user experience quality Output as quality estimate. The stopping condition in this case is that x is greater than or equal to MOS _MAX .

An efficient method for searching for an estimated value of the user experience quality of service A is known as a root-finding algorithm, and famous ones include the bisection method and Newton's method.

An example of the operation of the correction unit 12 when using the bisection method will also be described. When parameters representing the lower limit, upper limit, and intermediate value of the search range (hereinafter referred to as "related parameters") are set to low, high, and mid, the correction unit 12 calculates that low=MOS _MIN , high=MOS _MAX , mid=( Initialize each related parameter by (low+high)/2. The correction unit 12 inputs mid as the estimated user experience quality of service A and the pre-correction user experience quality estimate of service B as the estimated user experience quality of service B to the comparative evaluation value estimation unit 13, and performs a comparative evaluation. The comparative evaluation estimated value EMOS _PC is received from the value estimation unit 13. The correction unit 12 stops if |EMOS _PC |<Th, and outputs mid as the corrected user experience quality estimated value. Th is a threshold value used as a stop condition. As the stop condition, EMOS _PC ² <Th may be used or an equal sign may be included. If the stop condition is not satisfied, the correction unit 12 updates each related parameter (low, high, mid) and repeats the above operation. If EMOS _PC is a positive value, the estimated user experience quality of service B is worse, so the correction unit 12 sets high=mid, mid=( high and mid are updated by low+high)/2. If EMOS _PC is a negative value, the estimated user experience quality of service A is worse, so the correction unit 12 sets low=mid, mid=( Update low and mid by low+high)/2.

The comparative evaluation value estimating unit 13 calculates the comparative evaluation estimated value based on the estimated user experience quality of each of the services A and B input from the correcting unit 12. The comparative evaluation value estimation section 13 outputs the calculation result to the correction section 12. A simple example of how to calculate the comparative evaluation estimate EMOS _PC is as follows.

However, EMOS _A is an estimated value of the quality of user experience of service A, EMOS _B is an estimated value of the quality of user experience of service B, and a ₁₁ , a ₁₂ and a ₁₃ are coefficients. Hereinafter, _aij represents a coefficient. The coefficient can be derived by multiple regression analysis, curve fitting, etc. using the results of a subjective quality evaluation experiment that compares the quality of both services, and may take a value of 0 or a negative value.

Alternatively, the comparative evaluation value estimation unit 13 may calculate the comparative evaluation estimated value EMOS _PC by taking into consideration the characteristics at the time of the comparative evaluation by the user. An example in which such characteristics are taken into account will be described below.

For example, calculation can be performed after converting the estimated user experience quality into a psychological scale R from 0 to 100.

However, RfromMOS and MOSfromR are functions that convert MOS and R into each other.

Further, when the quality of one of the services is very poor, the comparative evaluation estimated value EMOS _PC may be calculated as follows, taking into account the strong impression that the quality of one of the services is very poor.

This calculation formula uses conditional branching under a specific condition (x<th). That is, when x<th, the value of δ is 1; otherwise, the value of δ is 0.

Similarly, using R,

You can also use it as

We simply add the square term to correct the estimated user experience quality of both services, taking into account the interaction,

It is also possible to convert it to R in the same way.

Also, instead of the square term, we will use a power term with the exponent part as a coefficient to correct the estimated user experience quality of both services,

It is also possible to convert it to R in the same way.

Furthermore, the comparative evaluation estimate EMOS _PC may be calculated by combining the above.

Further, the comparative evaluation estimated value EMOS _PC may be calculated using machine learning such as Random Forest, for example. In that case, EMOS _A and EMOS _B can be used as feature quantities.

Hereinafter, the processing procedure executed by the user experience quality estimation device 10 will be explained. FIG. 3 is a flowchart for explaining an example of a processing procedure executed by the user experience quality estimation device 10.

In step S11, the pre-correction quality estimating unit 11 calculates the pre-correction user experience quality estimate based on the quality parameters of service B input to the user experience quality estimation device 10. The pre-correction quality estimation unit 11 inputs the calculated pre-correction user experience quality estimate to the correction unit 12 .

Subsequently, the correction unit 12 initializes or updates the related parameters of the estimated user experience quality of service A and the estimated user experience quality of service A, and adjusts the estimated user experience quality of service A and the user experience quality of service B. The estimated quality value is input to the comparative evaluation value estimation unit 13 (S12). When step S12 is executed for the first time, the related parameters and the estimated user experience quality of service A are initialized. If step S12 is executed for the second time, the related parameters and the estimated user experience quality of service A are updated. Furthermore, the value inputted to the comparative evaluation value estimation unit 13 as the estimated user experience quality of service A is the estimated user experience quality of service A after initialization or update. The value inputted to the comparative evaluation value estimating unit 13 as the estimated user experience quality of service B is the corrected user experience quality estimate of service B.

Subsequently, the comparative evaluation value estimating unit 13 calculates a comparative evaluation estimated value based on the input estimated user experience quality of service A and the input estimated user experience quality of service B (S13). The comparative evaluation value estimating unit 13 outputs the calculated comparative evaluation estimated value to the correcting unit 12.

Subsequently, the correction unit 12 determines whether the stop condition is satisfied based on the comparative evaluation estimated value output from the comparative evaluation value estimation unit 13 (S14). If the stop condition is not met (No in S14), the correction unit 12 repeats steps S12 and subsequent steps.

If the stop condition is met (Yes in S14), the correction unit 12 uses the estimated user experience quality of service A when the stop condition is met (that is, last inputted to the comparison evaluation value estimation unit 13) as the service. It is output as the corrected user experience quality estimated value of B (S15).

Note that the user experience quality estimating device 10 has been described above that allows two different services A and B to be compared by correcting the estimated user experience quality value of service B using service A as a reference. By using a plurality of such user experience quality estimation devices 10 (or the functional configuration of the user experience quality estimation device 10), it becomes possible to compare the quality of three or more different services on the same scale. For example, for the third service C, the user experience quality estimation device 10 having the comparative evaluation value estimating unit 13 for services A and C is used to calculate the estimated user experience quality of service C. By making the correction, it becomes possible to compare the estimated user experience quality of service A and the corrected user experience quality estimate of service B on the same scale.

As described above, according to the present embodiment, based on the comparative evaluation value when comparing the user experience quality of different services, the user experience quality of one service is determined based on the user experience quality of the other service. Corrections are made to map to. As a result, it is possible to uniformly estimate the quality of user experience between different services. By referring to the output on the same scale that enables quality comparisons between services, it can be used, for example, to control technology such as resource allocation that optimizes the quality of user experience within limited resources across services. It can be utilized.

Although the embodiments of the present invention have been described in detail above, the present invention is not limited to these specific embodiments, and various modifications can be made within the scope of the gist of the present invention as described in the claims. - Can be changed.

10 User Experience Quality Estimation Device 11 Pre-correction Quality Estimation Unit 12 Correction Unit 13 Comparison Evaluation Value Estimation Unit 100 Drive Device 101 Recording Medium 102 Auxiliary Storage Device 103 Memory Device 104 Processor 105 Interface Device B Bus

Claims

Comparative evaluation for estimating a comparative evaluation of the relative superiority/inferiority between the first estimated value of the quality of user experience regarding the first communication service and a certain value on the scale of the estimated value of the quality of user experience regarding the second communication service. a comparative evaluation value estimator configured to calculate an estimated value;
a correction unit configured to output the certain value as a second estimated value obtained by correcting the first estimated value to the scale when the comparative evaluation estimated value satisfies a predetermined condition;
A user experience quality estimation device comprising:
The comparative evaluation value estimating unit is configured to calculate the comparative evaluation estimated value with the first estimated value for a plurality of values in the scale,
The correction unit is configured to output, as the second estimated value, a value for which the comparative evaluation estimated value satisfies the predetermined condition among the plurality of values.
The user experience quality estimation device according to claim 1.
The comparative evaluation value estimating unit takes into account the characteristics at the time of comparative evaluation by the user, corrects the estimated user experience quality of the first communication service and the second communication service, converts the psychological scale, and calculates the specific conditions. is configured to calculate the comparative evaluation estimated value using any one or more of the conditional branches in;
The user experience quality estimation device according to claim 1 or 2, characterized in that:
Comparative evaluation for estimating a comparative evaluation of the relative superiority/inferiority between the first estimated value of the quality of user experience regarding the first communication service and a certain value on the scale of the estimated value of the quality of user experience regarding the second communication service. a comparative evaluation value estimation procedure for calculating an estimated value;
a correction procedure of outputting the certain value as a second estimated value obtained by correcting the first estimated value to the scale when the comparative evaluation estimated value satisfies a predetermined condition;
A method for estimating quality of user experience, characterized in that the method is performed by a computer.
The comparative evaluation value estimation procedure calculates the comparative evaluation estimated value with the first estimated value for a plurality of values in the scale,
The correction procedure includes outputting a value among the plurality of values for which the comparative evaluation estimated value satisfies the predetermined condition as the second estimated value.
5. The user experience quality estimation method according to claim 4.
The comparative evaluation value estimation procedure takes into account the characteristics at the time of comparative evaluation by the user, corrects the estimated user experience quality of the first communication service and the second communication service, converts the psychological scale, and performs specific conditions. calculating the comparative evaluation estimated value using any one or more of the conditional branches in;
The user experience quality estimation method according to claim 4 or 5, characterized in that:
Comparative evaluation for estimating a comparative evaluation of the relative superiority/inferiority between the first estimated value of the quality of user experience regarding the first communication service and a certain value on the scale of the estimated value of the quality of user experience regarding the second communication service. a comparative evaluation value estimation procedure for calculating an estimated value;
a correction procedure of outputting the certain value as a second estimated value obtained by correcting the first estimated value to the scale when the comparative evaluation estimated value satisfies a predetermined condition;
A program that causes a computer to execute.