WO2016101711A1 - Automatic evaluation method and system for quality of experience of business network service - Google Patents

Abstract

Provided are an automatic evaluation method and system for the quality of experience of a business network service. A business network service is divided into a plurality of response stages, and a response delay of each response stage is acquired; then an appropriate quality-of-experience evaluation function is selected according to a response stage and a response delay of the response stage; the selected quality-of-experience evaluation function evaluates the quality of experience at the end of a present stage according to the quality of experience of the previous stage and the response delay of the present response stage to obtain sequentially a quality-of-experience value of each response stage; and a quality-of-experience value at the end of the last response stage is taken as a quality-of-experience value of the business network service, and the browsing experience of a user is dynamically evaluated from the perspective of user perception. The evaluation method and system for the quality of experience of the present invention not only factor in the effect different response stages have on the experience, but also factor in the experience carry-over effect from one stage to the next, such that the quality of experience of a business network service can be more accurately evaluated.

Description

Method and system for automatically evaluating service network service experience quality

This application claims the priority of the Chinese Patent Application submitted to the China Patent Office on December 25, 2014, the application number is CN201410827613.2, and the invention name is “A Method and System for Automatic Evaluation of Experience Quality of Service Network Service”. This is incorporated herein by reference.

Technical field

The present invention relates to the field of Internet technologies, and in particular, to a method and system for automatically evaluating a service network service experience quality.

Background technique

At present, the network is continuously developing in the direction of high speed, and network applications have also gained wider popularity. At present, web-based web browsing has become the most frequently used business in the daily work, study, and entertainment of the public. Moreover, the public's dependence on web browsing is increasing, and the requirements for browsing experience are getting higher and higher. Faced with high-speed complex, flexible and changeable network environment, how to provide users with a good browsing experience has become a major challenge for network operators. As the most important reference for improving services, the evaluation of service quality has received more and more attention.

In the traditional mode, the network operator usually evaluates the service performance running on it by measuring the quality of service (English: Quality of Service, QoS for short) parameters such as throughput, packet loss rate or delay. However, in practice, users and network operators have different differences in the performance evaluation methods and focuses of web browsing services because of different starting points for service evaluation, that is, different standards are used between users and operators. Determine the performance of the service. Relative to the reference of network media parameters by network operators, users tend to evaluate the performance of services through their own feelings and experiences in a relatively subjective and non-technical context. Often, what users need is an ideal service response time and a good service experience, and they don't care about the technical parameters in the network. In recent years, with the deepening of research on quality evaluation by researchers, the research found that network QoS parameters can not reflect the real experience and perception of upper-level users. This leads to the evaluation method of using service subjective factors to evaluate service quality. Such user satisfaction with the service and evaluation indicators are often referred to as the user's quality of experience (English: Quality of Experience, referred to as QoE). ITU-T defines QoE as the subjective experience and perception of end users. The overall acceptable level of an application or service. Due to the need to obtain the user's subjective experience, it is generally required to organize a large number of personnel to subjectively score the service, and use the evaluation result as a reference value of the evaluation model, which will consume a lot of labor and economic costs.

Summary of the invention

The embodiment of the invention provides an automatic evaluation method and system for service network service experience quality, so as to automatically evaluate the service network service experience quality.

The first aspect provides an automatic evaluation method for service network service experience quality, including:

Obtaining a response delay of at least one response phase of the service network service;

Selecting, according to the response delay evaluation evaluation of each response stage, the relevant parameters of the experience quality evaluation function or the evaluation function of each response stage from at least one experience quality evaluation function of each response stage;

Obtaining an experience quality value for each response phase by using the selected quality of experience evaluation function according to each response delay and the quality of experience value of the previous response phase;

Using the quality of experience value of the previous response phase as the initial quality of experience value of the current response phase, repeating the obtaining step of the quality of experience value for each of the response phases, until the quality of experience value of the last response phase is obtained, as the service network service The quality of the experience.

In a first possible implementation, the initial quality of experience value of the first response phase is an initial value that is pre-configured by the system.

In conjunction with the first aspect or the first possible implementation of the first aspect, in a second possible implementation, the at least one quality of experience evaluation function of each of the response stages includes:

Where f _i (t) is the quality of experience value of the i-th response phase, and T ₁ ⁱ , T ₂ ⁱ , T ₃ ⁱ are three time threshold parameters in the i-th response phase for evaluating functions and parameters Select, currentScore is the quality of experience value f _i-1 (t) of the previous response phase.

In conjunction with the second possible implementation of the first aspect, in a third possible implementation, the f _i (t) is specifically:

Wherein, a ₁ , b ₁ , c ₁ , a ₂ , b ₂ , c ₂ , and d are related parameters of the experience quality evaluation in the same response phase, and the experience quality evaluation parameters in different response phases are different or the same.

In conjunction with the first aspect or the first possible implementation of the first aspect or the second possible implementation of the first aspect or the third possible implementation of the first aspect, in a fourth possible implementation And obtaining the response delay of the at least one response phase of the network service, including:

When the client detects the submit request of the URL, the system time t _{1 is} recorded;

When the client detects the end of the loading title, the system time t _{2 is} recorded;

When the client detects the end of the loaded text, the system time t _{3 is} recorded;

When the client detects the end of the first displayable content in the loading page, the system time t _{4 is} recorded;

When the client detects the end of loading the remaining displayable content, the system time t _{5 is} recorded;

Obtaining response delays of the respective response phases, the response delays of the respective response phases are D ₁ =t ₂ -t ₁ , D ₂ =t ₃ -t ₂ , D ₃ =t ₄ -t ₃ , D4=t ₅ -t ₄ .

In combination with the first aspect or the first possible implementation of the first aspect or the second possible implementation of the first aspect or the third possible implementation of the first aspect, in a fifth possible implementation And obtaining the response delay of the at least one response phase of the network service, including:

When the HTTP GET request for the URL is detected at any network node of the access network or the core network, the system time t _{1 is} recorded;

After the network node detects the first response message of the HTTP GET of the URL, the system time t _{2 is} recorded;

After the network node detects the second HTTP GET request, the system time t _{3 is} recorded;

After the network node detects the last response message of the first picture HTTP GET request, the system time t _{4 is} recorded;

When the network node detects that the entire page transfer is completed, the system records the time t _5;

Obtaining response delays of the respective response phases, the response delays of the respective response phases are D ₁ =t ₂ -t ₁ , D ₂ =t ₃ -t ₂ , D ₃ =t ₄ -t ₃ , D4=t ₅ -t ₄ .

In conjunction with the first aspect or the first possible implementation of the first aspect or the second possible implementation of the first aspect or the third possible implementation of the first aspect, in a sixth possible implementation And obtaining the response delay of the at least one response phase of the network service, including:

After the service server receives the first HTTP GET request, the system time t _{1 is} recorded;

After the service server sends the response containing the header portion in the main file, the system time t _{2 is} recorded;

After the service server receives the next HTTP GET request, the system time t _{3 is} recorded;

After the service server sends the end of the first displayable object, the system time t _{4 is} recorded;

When the service server ends all responses, the system time t _{5 is} recorded;

Obtaining response delays of the respective response phases, the response delays of the respective response phases are D ₁ =t ₂ -t ₁ , D ₂ =t ₃ -t ₂ , D ₃ =t ₄ -t ₃ , D4=t ₅ -t ₄ .

The second aspect provides an automatic evaluation method for service network service experience quality, including:

Obtaining a response delay of at least one response phase of the service network service;

Obtaining an experience quality value and a weighting value for each response phase according to a response delay of each response phase, the weighting value being an effect on the quality of experience value of each response phase according to the response delay of each response phase The weight assigned by the degree;

A weighted sum of the quality of experience values for each response phase is obtained as the quality of experience value of the service network service according to the weighting value of each response phase.

In a third aspect, an automatic network service service quality evaluation system is provided, including:

a delay extraction unit, configured to acquire a response delay of at least one response phase of the service network service;

a selecting unit, configured to select, according to a response delay of each response stage, an experience quality evaluation function or a correlation parameter of the evaluation function of each response stage from at least one experience quality evaluation function of each response stage;

a quality evaluation unit, configured to obtain an experience quality value of each response stage by using the selected experience quality evaluation function according to each response delay and the quality of experience value of the previous response stage;

The quality evaluation unit is further configured to: use the quality of experience value of the previous response stage as the current response The initial experience quality value of the phase repeats the acquisition of the quality of experience value for each of the response phases described above until the quality of experience value of the last response phase is obtained as the quality of experience value of the service network service.

In a first possible implementation, the initial quality of experience value of the first response phase is an initial value that is pre-configured by the system.

With reference to the third aspect or the first possible implementation manner of the third aspect, in a second possible implementation manner, the at least one quality of experience evaluation function of each of the response stages includes:

Where f _i (t) is the quality of experience value of the i-th response phase, and T ₁ ⁱ , T ₂ ⁱ , T ₃ ⁱ are three time threshold parameters in the i-th response phase for evaluating functions and parameters Select, currentScore is the quality of experience value f _i-1 (t) of the previous response phase.

In conjunction with the second possible implementation of the third aspect, in a third possible implementation, the f _i (t) is specifically:

Wherein, a ₁ , b ₁ , c ₁ , a ₂ , b ₂ , c ₂ , and d are related parameters of the experience quality evaluation in the same response phase, and the experience quality evaluation parameters in different response phases are different or the same.

With reference to the third aspect or the first possible implementation manner of the third aspect or the second possible implementation manner of the third aspect or the third possible implementation manner of the third aspect, in a fourth possible implementation manner The delay extraction unit includes:

a first recording unit, configured to record a system time t ₁ when the client detects the submit request of the URL;

The first recording unit is further configured to record the system time t ₂ when the client detects that the loading title ends;

The first recording unit is further configured to record the system time t ₃ when the client detects that the loading of the text ends;

The first recording unit is further configured to record the system time t ₄ when the client detects that the first displayable content in the loading page ends;

The first recording unit is further configured to record the system time t ₅ when the client detects that the remaining displayable content is loaded.

a first obtaining unit, configured to obtain a response delay of each response phase, where the response delay of each response phase is D ₁ =t ₂ -t ₁ , D ₂ =t ₃ -t ₂ , D ₃ =t ₄ -t ₃ , D4=t ₅ -t ₄ .

In combination with the third aspect or the first possible implementation of the third aspect or the second possible implementation of the third aspect or the third possible implementation of the third aspect, in a fifth possible implementation manner The delay extraction unit includes:

a second recording unit, configured to record a system time t ₁ when an HTTP GET request for a URL is detected by any network node of the access network or the core network;

The second recording unit is further configured to record the system time t ₂ after the network node detects the first response message of the HTTP GET of the URL;

The second recording unit is further configured to record the system time t ₃ after the network node detects the second HTTP GET request;

The second recording unit is further configured to record the system time t ₄ after the network node detects the last response message of the first picture HTTP GET request;

The second recording unit is further configured to record the system time t ₅ after the network node detects that the entire webpage is completely transmitted;

a second obtaining unit, configured to obtain a response delay of each response phase, where the response delay of each response phase is D ₁ =t ₂ -t ₁ , D ₂ =t ₃ -t ₂ , D ₃ =t ₄ -t ₃ , D4=t ₅ -t ₄ .

In combination with the third aspect or the first possible implementation of the third aspect or the second possible implementation of the third aspect or the third possible implementation of the third aspect, in a sixth possible implementation manner The delay extraction unit includes:

a third recording unit, configured to record a system time t ₁ after the service server receives the first HTTP GET request;

The third recording unit is further configured to record the system time t ₂ after the service server sends the response including the header portion in the main file;

The third recording unit is further configured to record the system time t ₃ after the service server receives the next HTTP GET request;

The third recording unit is further configured to record the system time t ₄ after the service server ends the sending of the first displayable object;

The third recording unit is further configured to record the system time t ₅ when the service server ends all responses;

a third obtaining unit, configured to obtain a response delay of each response phase, where the response delay of each response phase is D ₁ =t ₂ -t ₁ , D ₂ =t ₃ -t ₂ , D ₃ =t ₄ -t ₃ , D4=t ₅ -t ₄ .

In a fourth aspect, an automatic network service service quality evaluation system is provided, including:

a delay extraction unit, configured to acquire a response delay of at least one response phase of the service network service;

An obtaining unit, configured to obtain an experience quality value and a weighting value of each response phase according to a response delay of each response phase, where the weighting value is according to a response delay of each response phase to each response phase The weight assigned to the degree of impact of the quality value;

The quality evaluation unit is configured to obtain, according to the weighting value of each response phase, a weighted sum of the quality of experience values of each response phase as the quality of experience value of the service network service.

It can be seen that, according to an embodiment of the present invention, a method and system for automatically evaluating a service network service quality of experience, by dividing a service network service into multiple response phases, obtaining a response delay of each response phase, and then according to the response phase and the The appropriate response quality evaluation function is selected according to the response delay of the stage. The selected experience quality evaluation function evaluates the quality of experience at the end of the stage according to the quality of the previous stage and the response delay of the response stage, and sequentially obtains each response stage. The quality of experience value, and the quality of experience value at the end of the last response phase is used as the quality of experience value of the service network service, and the user's browsing experience is dynamically evaluated from the perspective of the user's perception. The method and system for evaluating the quality of experience of the present invention are not only Considering the impact of the different response phases of the business on the experience, and considering the impact of the previous phase of the experience transfer to the next stage, the quality of the service experience of the service network can be more accurately evaluated.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only Some embodiments of the present invention, for those of ordinary skill in the art, do not pay Other drawings can also be obtained from these drawings on the premise of creative labor.

1 is a schematic diagram of a network structure of service network service quality evaluation according to an embodiment of the present invention;

FIG. 2 is a flowchart of a method for automatically evaluating a service network service experience quality according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of an overall service network service experience quality automatic evaluation process according to an embodiment of the present invention; FIG.

Figure 4 is a schematic diagram of the 5-point evaluation criteria;

FIG. 5 is a schematic diagram of dividing a webpage loading phase;

6 is a flowchart of an experience quality evaluation function according to an embodiment of the present invention;

FIG. 7 is a schematic flowchart of monitoring and evaluating in a browser according to an example of an embodiment of the present invention;

Figure 8 is a schematic diagram showing changes in the quality of experience in the first phase;

FIG. 9 is a schematic flowchart of monitoring and evaluating in an access network according to an example of an embodiment of the present invention;

FIG. 10 is a schematic flowchart of monitoring and evaluating on a web server according to an example of an embodiment of the present invention;

FIG. 11 is a flowchart of another method for automatically evaluating an experience quality of a service network service according to an embodiment of the present invention;

FIG. 12 is a schematic structural diagram of a service network service experience quality automatic evaluation system according to an embodiment of the present invention;

FIG. 13 is a schematic structural diagram of a time delay extracting unit 11 shown in FIG. 12;

FIG. 14 is a schematic structural diagram of another delay extraction unit 11 shown in FIG. 12;

FIG. 15 is a schematic structural diagram of still another delay extraction unit 11 shown in FIG. 12;

FIG. 16 is a schematic structural diagram of another service network service experience quality automatic evaluation system according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

The services involved in the embodiments of the present invention include web browsing, but are not limited to web browsing, and may be Other network services that load content in stages. In the embodiment of the present invention, web browsing is taken as an example. In the current research, when evaluating the quality of webpage browsing service, only the overall waiting delay and loading delay are considered, and the whole browsing process is studied as an object of evaluating the quality of service. However, when browsing a webpage, since the browser loads the webpage in stages, the browsing experience of the user is often affected by some loading events in stages. For example, when the page title loading time is long, the user generally closes the webpage directly to terminate the browsing behavior; when the first version of the webpage is loaded, the user browses the first version of the content and is less affected by the subsequent content loading time. Considering the staged real experience process when the user browses the webpage, the embodiment of the present invention divides the entire webpage loading process into multiple stages, and evaluates the browsing process in stages. In addition, considering that the user's perception of the previous stage event will affect its perception of the current stage of the event, even in the same stage, the previous moment has an impact on the current moment. The embodiment of the present invention utilizes the continuity of multi-stage impact, and proposes an evaluation scheme for web-based web browsing service quality. The program converts the entire browsing process into multiple consecutive processes for evaluation, and dynamically evaluates the user's browsing experience from the perspective of user perception. The evaluation scheme proposed by the embodiment of the present invention can reflect the quality of the browsing experience of the user more accurately and in real time, and the loading delay of the webpage is in one-to-one correspondence with the user experience.

The invention can monitor and record the webpage loading delay in real time in the user equipment, the access network, the network node or the server, and evaluate the service quality of the webpage browsed by the user in real time or in batches, and provide an important reference for the network operator to improve the service, and the service network service The network structure of quality evaluation is shown in Figure 1. Here, the user equipment may be a handheld terminal or other terminal device capable of providing a web browsing service. The access network may be a network suitable for packet communication (English: Packet-type Communications) such as a broadcast network, a local area network, a wide area network, or a wireless network. These networks can be used including Digital Subscriber Line (DSL), Cable Broadband, 802.11 Wi-Fi, Global System for Mobile Communication (GSM). Wideband Code Division Multiple Access (WCDMA), CDMA2000 (Code Division Multiple Access 2000), Worldwide Interoperability for Microwave Access (WiMax) or Long Term Evolution (English) English: Long Term Evolution, referred to as LTE). The evaluation in the present invention can be used not only in a client (such as a browser) on a user equipment, an access network, an arbitrary network node or a server, but also can separately set up a module for quality evaluation after extracting evaluation parameters from any of the above locations. In addition, the evaluation process can be performed in real time or after data is collected for later evaluation and analysis.

FIG. 2 is a flowchart of a method for automatically evaluating an experience quality of a service network service according to an embodiment of the present invention. The method includes the following steps:

Step S101: Acquire a response delay of at least one response phase of the service network service.

A network service can be divided into multiple response phases, and the response delay of each response phase reflects the quality of experience of users at this stage.

The manner of obtaining the response delay may be determined according to the location of the response monitoring. For example, the response delay is monitored on the client of the user equipment, and the response delay of each response phase may be obtained by the client; if in the access network or The response delay is monitored on any network node of the core network, and the response delay of each response phase can be obtained by the network node; if the response delay is monitored by the server, the response delay of each response phase can be obtained by the server. If the evaluation is performed on a client, a network node, or a server, the response delay of each response phase is directly obtained by the client, the network node, or the server; if the evaluation is performed in a separate system, the slave, the network The node or server gets the response delay for each response phase.

Step S102: Select an experience quality evaluation function or a correlation parameter of the evaluation function for each response stage from at least one experience quality evaluation function of each response stage according to a response delay of each response stage.

The impact of each response phase in the network service on the user experience is not the same. Different response delays in each response phase have different impacts on the user experience. Therefore, it is required to be based on each response phase in each response phase. In response to the delay, the quality of experience evaluation function of the response stage is selected from one or more quality of experience evaluation functions of each response stage, or related parameters of different evaluation functions are used to conform to the perceived characteristics of the user.

Step S103: Obtain an experience quality value for each response phase by using the selected experience quality evaluation function according to each response delay and the quality of experience value of the previous response phase.

Using the thoughts that influence the accumulation, the user's experience quality in each response stage is dynamically evaluated at any time. The experience quality evaluation function is a function related to the quality of experience value and the response delay of each previous response stage.

Step S104: The experience quality value of the previous response phase is used as the initial experience quality value of the current response phase, and the obtaining step of the quality of experience value of each of the response phases is repeated until the last one is obtained. The quality of experience value of the response phase as the quality of experience value of the service network service.

The initial value of the quality value of each response phase is the quality of experience value at the end of the previous phase. The quality experience value output after the last response phase is the final quality of experience value as the quality of experience value for the entire service network service.

FIG. 3 is a schematic diagram of an overall service network service experience quality automatic evaluation process according to an embodiment of the present invention, which includes four response phases, and the quality experience value outputted by the fourth phase is the final experience quality value, and may also be as needed. , respectively output the quality of experience value at the end of each response phase. MOS (Mean Opinion Score) is one of the most widely used subjective assessment methods, with a score range of 1 to 5. As shown in the schematic diagram of the 5-point evaluation criteria shown in Figure 4, the internationally used 5-point evaluation criteria can be used in the evaluation, and divided into five levels according to the experience experience.

According to an embodiment of the present invention, an automatic evaluation method for service network service experience quality is obtained by dividing a service network service into multiple response phases, obtaining a response delay of each response phase, and then responding according to the response phase and the response time of the phase The appropriate quality of experience evaluation function is selected, and the selected quality of experience evaluation function evaluates the quality of experience at the end of the stage according to the quality of the previous stage and the response delay of the response stage, and sequentially obtains the quality of experience value of each response stage. And the quality of experience value at the end of the last response phase is taken as the quality of experience value of the service network service, and the user's browsing experience is dynamically evaluated from the perspective of the user's perception, not only considering the impact of the different response phases of the service on the experience, but also considering The impact of the previous phase of the experience is passed to the next stage, so that the quality of the service experience of the business network can be more accurately evaluated.

Please refer to FIG. 5, which is a schematic diagram of dividing a webpage loading phase. Here, the client that performs webpage loading can be a browser. On the user device, when the user browses the webpage, the browser (such as Internet Explorer, Firefox, Chrome, Safari, UC, etc.) usually needs to go through multiple loading processes to load the webpage requested by the user. The order in which the loading events occur is divided into four stages: the process of clicking the hyperlink to the page when the user browses the webpage is divided into four stages: 1. The browser parses the received hypertext markup language (English: Hyper Text Markup language, abbreviation: HTML) The title of the document is the loading of the title of the webpage; 2. The browser loads the HTML document, that is, loads the text in the webpage; 3. The browser requests the first HTML embedded resource to receive and load the first embedded resource. Finished, that is, the first object is loaded; 4, the browser for the rest of the embedded resources Request, receive, and load.

Since the user tends to pay more attention to the response time of the service, the time delay of loading in the different stages described above is used as the evaluation parameter of the evaluation method. After the client obtains the loading delay D _i , i=1, 2, 3, 4 of different stages of the browsing webpage by the corresponding technology, the quality evaluation is performed according to each loading delay. In the evaluation, the internationally used 5-point evaluation standard can be used, and the experience is divided into five levels.

Due to the service response time in browsing, that is, the loading delay is obviously as small as possible, so the occurrence of loading delay usually leads to a decline in user experience. However, when the third stage ends, since the text and the first object have been loaded, the user can first browse the loaded content, and the subjective perception of the user will slightly rebound. In addition, the stages of page loading have different impacts on the user experience. For example, users are generally sensitive to delays in the first and fourth phases, so the impact of the first and fourth phases on the user experience is affected. It is greater than the impact of the second and third stage delays. This requires different model parameters to be used at each stage to match the user's perceived characteristics. After obtaining the delay of each stage, the process of service quality evaluation is shown in Figure 3. The initial value of the experience value of each phase is the experience value at the end of the previous phase, and the experience value output after the fourth phase is the final experience score. The model can also output the experience scores at the end of each stage as needed to analyze the impact of each stage in the browsing process, as shown in Figure 3.

Based on the above analysis, the embodiment of the present invention utilizes the idea of influencing the accumulation, and proposes a service quality evaluation method for dynamically evaluating the user's experience quality in the browsing process, and the general experience quality evaluation function in each stage is:

f _i (t)=max(1,min(f _i (t),5))

In the above formula, currentScore is the experience score at the beginning of the current time, and the initial value is 5 points. T ₁ ⁱ , T ₂ ⁱ , T ₃ ⁱ are the three time threshold parameters in the i-th stage, which are used to evaluate the selection of functions and parameters, and evaluate the effects of different response delays on the quality of experience evaluation in the same stage. The time points are divided in different ways. f ₁ ⁱ (t), f ₂ ⁱ (t), and f ₃ ⁱ (t) are the variation functions of the experience values in different partitions, respectively. Taking the calculation process of the quality of experience value in the first response phase as an example, according to the comparison of the response delay of the first response phase with T ₁ ¹ , T ₂ ¹ , and T ₃ ¹ , it is determined that the formula (1) is used. Which evaluation function calculates the quality of experience value for this phase. A flowchart of the experience quality evaluation function is shown in FIG. 6, where D _i is the delay of each response phase.

Specifically, you can take the following form of function:

f _i (t)=max(1,min(f _i (t),5))

The parameters a ₁ , b ₁ , c ₁ , a ₂ , b ₂ , c ₂ , and d are model parameters in the same stage. It should be noted that the values of the parameters may be the same or different in different stages. Different model parameters can reflect different variations of experience values. Since there is no upper bound in the delay in each phase, in any phase, when t>T _threshold , take MOS=MOS _min , for example, MOS _min is 0 or 1, that is, the user experience is unbearable when the delay is too large. , the user will leave the browsing service. After the fourth stage of loading is completed, the output f ₄ (t) is the final user experience MOS value. Specifically, the value of T ₁ ^{1 in the} first stage is generally small to indicate that the experience lasts for 5 points during the user's reaction time. And in the first stage, T ₂ ¹ usually takes the same value as T ₁ ¹ , that is, the two time points coincide, and at this time, it means that there is no interval in which the quality rises in the stage. After the delay is greater than T ₁ ¹ , the experience value will directly enter the falling range between T ₂ ¹ and T ₃ ¹ . In the second and third stages, generally, T ₁ ² = T ₂ ² =0, and T ₁ ³ = T ₂ ³ =0, that is, there is no interval in which the experience value remains unchanged and rises. However, in the fourth stage, considering that there is a case where the experience value rises, T ₁ ⁴ and T ₂ ⁴ take different values. That is, when the delay is greater than T ₁ ⁴ , the experience value enters a section where the experience value between T ₁ ⁴ and T ₂ ⁴ rises.

For the selection of the model parameters, the MOS value can be obtained by a progressive subjective scoring method for each stage. For example, for the same webpage, subjective scoring is performed after the end of the first phase and the second phase respectively, and it is considered that the change of the second phase experience value is based on the subjective experience of the first phase, thereby obtaining the experience in the second phase. The range of variation, and then the specific model parameters are obtained by fitting or the like. The rest of the stages, and so on, can be used to calculate the model parameters in each stage.

The evaluation method is further described by taking the evaluation and monitoring on the client of the user equipment as an example. The evaluation can be completed by integrating the method flow into a software function module of the browser or setting the software function module outside the browser. The embodiment divides the evaluation process into two parts: delay extraction and browsing quality evaluation. First, four stages of delay data are extracted from the client (such as a browser), and the extracted delay data is sent to the evaluation models of each stage for quality evaluation. Specifically, in the evaluation process, the interaction process of each unit is as shown in FIG. 7.

When the browser performs the evaluation, after monitoring the user input URL and submitting, the browser parsing unit notifies the delay extraction unit to record the system time t ₁ , where the browser parsing unit is an existing unit of the browser, and the delay The extracting unit may be regarded as the software functional unit of step S101; notify the delay extracting unit to record the system time t ₂ when it loads the title; and then notify the end of the text loading end, the end of the first displayable content loading, and the end of the entire webpage loading, respectively. The delay extraction unit records system times t ₃ , t ₄ and t ₅ . At this time, the time delay D _i of each stage is calculated and D _{i is} sent to the quality evaluation unit for quality evaluation. During the delay extraction process, a certain unit may not exist. When this occurs, the recording time can be set to the system time recorded by the previous event, and the corresponding delay is also set to 0. For example, when there is only text information in the web page, then t ₄ = t ₃ , t ₅ = t ₃ , and D ₃ =0, D ₄ =0 is set.

In this embodiment, the webpage loading process is divided into four stages, namely: 1. loading of the webpage title; 2. loading of the text in the webpage; 3. loading of the first displayable object in the webpage; 4. loading of the remaining objects. The delay extraction unit obtains the loading delays D _i , i=1, 2, 3, 4 of each stage by analyzing the loading process in a client (such as a browser). Since the time to establish a link is very short, the delay for establishing a link can be ignored here.

If there is no part of the content in the webpage, the judgment is skipped in the flow, and the time of the corresponding record is set to the time recorded in the previous judgment and the delay is set to 0. For example, when there is no text in the web page, the judgment of the end of the text display is skipped and t ₃ = t _{2 is set} , and D ₂ =0.

After the delay extraction is completed, each time delay data is sequentially passed through the browsing quality evaluation unit in the dotted line frame in FIG. 3, and the quality evaluation is performed in stages. The evaluation model in each stage is as shown in formula (1).

The initial phase of the first phase is currentScore=5, and the currentScore starting values of the remaining phases are f(t) at the end of the previous phase, where t=D. It is also possible to add a delay threshold T _threshold = 5 s, that is, the user will take an interrupt browsing operation when the time delay is greater than 5 s in the phase, and the MOS is divided into a minimum value (such as 0). The values of a set of available parameters are shown in Table 1.

Table 1. Parameters related to the quality evaluation function

parameter	Stage one	Stage two	Stage three	Stage four
a 1	3.18	3.41	2.64	2.94
b 1	2.71	1.95	0.04	25.9
c 1	1	1	1	1
a 2	3.13	3.01	2.73	3
b 2	2.41	1.85	0.05	26
c 2	1	1	1	1
d	0	0.6	0	0.6
T 1	0.4s	0s	0s	0s
T 2	0.4s	0s	0s	0.5s
T 3	+∞	2s	+∞	3s

For example, the user perceives that there is no impact in the short delay T ₁ of the first phase, and the currentScore remains at 5 points. When the delay is greater than 0.4s, since the user does not experience the process of improvement at this stage, T ₂ ¹ = T ₁ ¹ , and currentScore will directly enter the descending process. After the delay exceeds the threshold T _threshold = 5s, the user experience immediately drops to 0, that is, the user will leave the browsing operation. Figure 8 shows the changes in the quality of experience during the first phase.

The evaluation method of the present invention is further described by the access network monitoring service network service. The evaluation process is divided into time delay extraction and browsing quality evaluation. Firstly, four stages of delay data are extracted from the access network, and the extracted delay data is sent to the evaluation model function of each stage for quality evaluation.

Specifically, in the evaluation process, the interaction process of each unit is as shown in FIG. 9.

When the access network performs evaluation, after the access network detects the HTTP GET request of a certain URL, the network packet capture and analysis unit notifies the delay extraction unit to record the system time t ₁ , and the network packet capture and analysis unit is the access network. An existing software function unit, the delay extraction unit can be regarded as a software function unit of S101; after detecting the first response message of the HTTP GET of the URL, notifying the delay extraction unit to record the system time t ₂ ; Upon monitoring the second HTTP GET request, the last response message of the first picture HTTP GET request and the end of the entire web page transmission respectively notify the delay extraction unit to record the system times t ₃ , t ₄ and t ₅ . At this time, the time delay D _i of each stage is calculated and D _{i is} sent to the quality evaluation unit for quality evaluation. During the delay extraction process, a certain unit may not exist. When this occurs, the recording time can be set to the system time recorded by the previous event, and the corresponding delay is also set to 0. For example, when the last response message of the first picture HTTP GET request is not detected, then t ₄ = t ₃ , t ₅ = t ₃ , and D ₃ =0, D ₄ =0 is set.

The time delay extraction unit of the embodiment obtains the timestamp and type information of each transport packet by using packet analysis, and further obtains the loading delays D _i , i=1, 2, 3, 4 of each stage.

If there is no part of the content in the webpage, the judgment is skipped in the flow, and the time of the corresponding record is set to the time recorded in the previous judgment and the delay is set to 0. For example, when there is no text in the web page, the judgment of the end of the text display is skipped and t ₃ = t _{2 is set} , and D ₂ =0.

After the delay extraction is completed, each time delay data is sequentially passed through the browsing quality evaluation in the dotted line frame in FIG. 3, and the quality evaluation is performed in stages. The evaluation model in each stage is as shown in formula (1).

The initial currentScore=5 of each stage, that is, the present invention can also simplify the evaluation of each stage independently of each other, where t=D. It is also possible to add a delay threshold T _threshold = 5 s, that is, the user will take an interrupt browsing operation when the time delay is greater than 5 s in the phase, and the MOS is divided into a minimum value (such as 0).

The evaluation method of the present invention is further described by monitoring a business network service at a web server. The evaluation process is divided into time delay extraction and browsing quality evaluation. Firstly, four stages of delay data are extracted from the web server, and the extracted delay data is sent to the evaluation models of each stage for quality evaluation.

In the evaluation process, the interaction process of each unit is as shown in FIG.

When the web server performs the evaluation, when it receives the first HTTP GET request, the server notifies the delay extraction unit to record the system time t ₁ ; when the server sends the response containing the header portion in the main file, the delay extraction unit records are notified. System time t ₂ ; then the delay extraction unit is notified of the system time t ₃ , t ₄ and t ₅ , respectively, upon monitoring the next HTTP GET request, the first displayable object and the end of the entire web page transmission. At this time, the time delay D _i of each stage is calculated and D _{i is} sent to the quality evaluation unit for quality evaluation. During the delay extraction process, a certain unit may not exist. When this occurs, the recording time can be set to the system time recorded by the previous event, and the corresponding delay is also set to 0. For example, when a GET request for a displayable object is not detected, then t ₄ = t ₃ , t ₅ = t ₃ , and D ₃ =0, D ₄ =0 is set.

In this embodiment, the delay extraction unit obtains the loading delays D _i , i=1, 2, 3, 4 of each stage by analyzing the sending process in the web server.

If there is no part of the process in the webpage requested by the user, the judgment of the part is skipped in the process, and the time of the corresponding record is set to the time recorded in the previous judgment and the delay is set to 0. For example, when there is no text in the requested web page, the judgment of the end of the text display is skipped and t ₃ = t _{2 is set} , and D ₂ =0.

After the delay extraction is completed, each time delay data is sequentially passed through the browsing quality evaluation in the dotted line frame in FIG. 3, and the quality evaluation is performed in stages. The evaluation model in each stage is as shown in formula (1).

The initial phase of the first phase is currentScore=5, and the currentScore starting values of the remaining phases are f(t) at the end of the previous phase, where t=D. It is also possible to add a delay threshold T _threshold = 5 s, that is, the user will take an interrupt browsing operation when the time delay is greater than 5 s in the phase, and the MOS is divided into a minimum value (such as 0).

FIG. 11 is a flowchart of another method for automatically evaluating an experience quality of a service network service according to an embodiment of the present invention. The method includes the following steps:

Step S201: Acquire a response delay of at least one response phase of the service network service.

A network service can be divided into multiple response phases, and the response delay of each response phase reflects the quality of experience of users at this stage.

Step S202: Acquire an experience quality value and a weighting value for each response phase according to a response delay of each response phase, where the weighting value is an experience quality according to a response delay of each response phase for each response phase. The weight of the value affected by the assigned value.

The impact of each response phase of the network service on the user experience is not the same. Different response delays in each response phase have different impacts on the user experience. Therefore, according to the response delay of each response phase, Obtaining an experience quality value and a weighting value for each response phase, the weighting value being a weight assigned according to the degree of influence of the response delay of each response phase on the quality of experience value of each response phase.

Step S203: Acquire a weighted sum of the quality of experience values of each response phase according to the weighting value of each response phase, as the quality of experience value of the service network service.

If an application with a low computational complexity is required, the evaluation process can be simplified to the norm for each stage of quality evaluation at each stage:

MOS=||ω·segment_MOS _i || ^p (3)

When p=1 in the above equation, the MOS value is the weighted sum of the experience values of the segments. ω is a weighted value assigned according to the degree of influence of each response phase.

According to an embodiment of the present invention, an automatic evaluation method for service network service experience quality is obtained by dividing a service network service into multiple response phases, obtaining a response delay of each response phase, and then responding according to the response phase and the response time of the phase The appropriate weighting value is selected, and the weighted sum of the quality of experience values of each response phase is used as the quality of experience value of the service network service, and the user's browsing experience is dynamically evaluated from the perspective of the user's perception, and not only the different responses of the service are considered. The impact of the stage on the experience, and considering the impact of the previous stage of experience transfer to the next stage, so that the quality of the service experience of the business network can be more accurately evaluated.

FIG. 12 is a schematic structural diagram of a service network service quality automatic evaluation system according to an embodiment of the present invention. The system 1000 includes:

The delay extraction unit 11 is configured to acquire a response delay of at least one response phase of the service network service.

A network service can be divided into multiple response phases, and the response delay of each response phase reflects the quality of experience of users at this stage.

The manner of obtaining the response delay may be determined according to the location of the response monitoring. For example, the response delay is monitored on the client of the user equipment, and the response delay of each response phase may be obtained by the client; if in the access network or The response delay is monitored on any network node of the core network, and the response delay of each response phase can be obtained by the network node; if the response delay is monitored by the server, the response delay of each response phase can be obtained by the server. If the evaluation is performed on a client, a network node, or a server, the response delay of each response phase is directly obtained by the client, the network node, or the server; if the evaluation is performed in a separate system, the slave, the network The node or server gets the response delay for each response phase.

The selecting unit 12 is configured to select an experience quality evaluation function or a correlation parameter of the evaluation function for each response stage from at least one experience quality evaluation function of each response stage according to a response delay of each response stage.

The impact of each response phase in the network service on the user experience is not the same. Different response delays in each response phase have different impacts on the user experience. Therefore, it is required to be based on each response phase in each response phase. In response to the delay, the quality of experience evaluation function of the response stage is selected from one or more quality of experience evaluation functions of each response stage, or related parameters of different evaluation functions are used to conform to the perceived characteristics of the user.

The quality evaluation unit 13 is configured to obtain an experience quality value for each response phase by using the selected experience quality evaluation function according to each response delay and the quality of experience value of the previous response phase.

Using the thoughts that influence the accumulation, the user's experience quality in each response stage is dynamically evaluated at any time. The experience quality evaluation function is a function related to the quality of experience value and the response delay of each previous response stage.

The quality evaluation unit 13 is further configured to use the quality of experience value of the previous response phase as the initial quality of experience value of the current response phase, and repeat the obtaining of the quality of experience value of each of the response phases until the quality of experience value of the last response phase is obtained. The quality of experience value as the service network service.

The initial value of the quality value of each response phase is the quality of experience value at the end of the previous phase. The quality experience value output after the last response phase is the final quality of experience value as the quality of experience value for the entire service network service.

The schematic diagram of the automatic evaluation process of the service network service experience quality shown in FIG. 3 includes four response phases, and the quality experience value outputted in the fourth phase is the final experience quality value, and may also output the end of each response phase according to needs. The quality of the experience. MOS (Mean Opinion Score) is one of the most widely used subjective assessment methods, with a score range of 1 to 5. As shown in the schematic diagram of the 5-point evaluation criteria shown in Figure 4, the internationally used 5-point evaluation criteria can be used in the evaluation, and divided into five levels according to the experience experience.

According to an embodiment of the present invention, a service network service experience quality automatic evaluation system obtains a response delay of each response phase by dividing the service network service into multiple response phases, and then according to the response phase and the response time of the phase The appropriate quality of experience evaluation function is selected, and the selected quality of experience evaluation function evaluates the quality of experience at the end of the stage according to the quality of the previous stage and the response delay of the response stage, and sequentially obtains the quality of experience value of each response stage. And the quality of experience value at the end of the last response phase is taken as the quality of experience value of the service network service, and the user's browsing experience is dynamically evaluated from the perspective of the user's perception, not only considering the impact of the different response phases of the service on the experience. It also considers the impact of the previous stage of the experience transfer to the next stage, so that the quality of the service experience of the business network can be more accurately evaluated.

The evaluation system is further described below by taking evaluation monitoring at the client of the user equipment as an example. The embodiment divides the evaluation process into two parts: delay extraction and browsing quality evaluation. First, four stages of delay data are extracted from the client (such as a browser), and the extracted delay data is sent to the evaluation models of each stage for quality evaluation. Specifically, in the evaluation process, the interaction process of each unit is as shown in FIG. 7.

Referring to the structure of a delay extraction unit 11 shown in FIG. 13, the delay extraction unit 11 includes: a first recording unit 111, configured to notify the delay extraction of the browser parsing unit after the user input URL is detected and submitted. The unit records the system time t ₁ , where the browser parsing unit is an existing unit of the browser; the first recording unit 111 is further configured to notify the delay extracting unit to record the system time t ₂ when the title is loaded; The recording unit 111 is further configured to notify the delay extraction unit to record the system times t ₃ , t ₄ and t ₅ respectively at the end of the text loading, the end of the first displayable content loading and the end of the entire web page loading. At this time, the first acquisition unit 112 is configured to calculate the delay D _i of each stage and send D _{i to the} quality evaluation unit 13 for quality evaluation. During the delay extraction process, a certain unit may not exist. When this occurs, the recording time can be set to the system time recorded by the previous event, and the corresponding delay is also set to 0. For example, when there is only text information in the web page, then t ₄ = t ₃ , t ₅ = t ₃ , and D ₃ =0, D ₄ =0 is set.

In this embodiment, the webpage loading process is divided into four stages, namely: 1. loading of the webpage title; 2. loading of the text in the webpage; 3. loading of the first displayable object in the webpage; 4. loading of the remaining objects. The delay extraction unit obtains the loading delays D _i , i=1, 2, 3, 4 of each stage by analyzing the loading process in a client (such as a browser). Since the time to establish a link is very short, the delay for establishing a link can be ignored here.

If there is no part of the content in the webpage, the judgment is skipped in the flow, and the time of the corresponding record is set to the time recorded in the previous judgment and the delay is set to 0. For example, when there is no text in the web page, the judgment of the end of the text display is skipped and t ₃ = t _{2 is set} , and D ₂ =0.

After the delay extraction is completed, each time delay data is sequentially passed through the browsing quality evaluation unit in the dotted line frame in FIG. 3, and the quality evaluation is performed in stages. The selection unit 12 selects an appropriate experience quality evaluation function in the evaluation model formula in each stage as shown in the formula (1).

The initial phase of the first phase is currentScore=5, and the currentScore starting values of the remaining phases are f(t) at the end of the previous phase, where t=D. It is also possible to add a delay threshold T _threshold = 5 s, that is, the user will take an interrupt browsing operation when the time delay is greater than 5 s in the phase, and the MOS is divided into a minimum value (such as 0). The values of a set of available parameters are shown in Table 1.

For example, the user perceives that there is no impact in the short delay T ₁ of the first phase, and the currentScore remains at 5 points. When the delay is greater than 0.4s, since the user does not experience the process of improvement at this stage, T ₂ ¹ = T ₁ ¹ , and currentScore will directly enter the descending process. After the delay exceeds the threshold T _threshold = 5s, the user experience immediately drops to 0, that is, the user will leave the browsing operation. Figure 8 shows the changes in the quality of experience during the first phase.

The evaluation system of the present invention is further described by an access network monitoring service network service. The evaluation process is divided into time delay extraction and browsing quality evaluation. Firstly, four stages of delay data are extracted from the access network, and the extracted delay data is sent to the evaluation model function of each stage for quality evaluation.

Specifically, in the evaluation process, the interaction process of each unit is as shown in FIG. 9.

Referring to the structure of another delay extraction unit 11 shown in FIG. 14, when the access network performs evaluation, the delay extraction unit 11 includes: a second recording unit 113, configured to detect a certain URL when the access network detects After the HTTP GET request, the network packet capture and analysis unit notifies the delay extraction unit to record the system time t ₁ , and the network packet capture and analysis unit is an existing software function unit of the access network; the second recording unit 113 is also used to After detecting the first response message of the HTTP GET of the URL, the notification delay extraction unit records the system time t ₂ ; the second recording unit 113 is further configured to subsequently monitor the second HTTP GET request, at the first The last response message of the picture HTTP GET request and the end of the entire web page transmission are respectively notified to the delay extraction unit to record the system times t ₃ , t ₄ and t ₅ . At this time, the second acquisition unit 114 is configured to calculate the delay D _i of each stage and send D _{i to the} quality evaluation unit 13 for quality evaluation. During the delay extraction process, a certain unit may not exist. When this occurs, the recording time can be set to the system time recorded by the previous event, and the corresponding delay is also set to 0. For example, when the first picture GET request is not detected, then t ₄ = t ₃ , t ₅ = t ₃ , and D ₃ =0, D ₄ =0 is set.

The time delay extracting unit 11 of the present embodiment obtains the timestamp and type information of each transport packet by using packet analysis, and obtains the loading delays D _i , i=1, 2, 3, 4 of each stage.

If there is no part of the content in the webpage, the judgment is skipped in the flow, and the time of the corresponding record is set to the time recorded in the previous judgment and the delay is set to 0. For example, when there is no text in the web page, the judgment of the end of the text display is skipped and t ₃ = t _{2 is set} , and D ₂ =0.

After the delay extraction is completed, each time delay data is sequentially passed through the browsing quality evaluation in the dotted line frame in FIG. 3, and the quality evaluation is performed in stages. The selection unit 12 selects an appropriate experience quality evaluation function in the evaluation model formula in each stage as shown in the formula (1).

The initial currentScore=5 of each stage, that is, the present invention can also simplify the evaluation of each stage independently of each other, where t=D. It is also possible to add a delay threshold T _threshold = 5 s, that is, the user will take an interrupt browsing operation when the time delay is greater than 5 s in the phase, and the MOS is divided into a minimum value (such as 0).

The evaluation system of the present invention is further described in terms of monitoring a business network service at a web server. The evaluation process is divided into time delay extraction and browsing quality evaluation. Firstly, four stages of delay data are extracted from the web server, and the extracted delay data is sent to the evaluation models of each stage for quality evaluation.

In the evaluation process, the interaction process of each unit is as shown in FIG.

Referring to the structure diagram of another delay extraction unit 11 shown in FIG. 15, when the web server performs evaluation, the delay extraction unit 11 includes: a third recording unit 115, when it receives the first HTTP GET request. The server notifies the delay extraction unit to record the system time t ₁ ; the third recording unit 115 is further configured to notify the delay extraction unit to record the system time t ₂ after the server sends the response containing the header portion in the main file; The next HTTP GET request, the third recording unit 115 is also used to notify the delay extraction unit to record the system time t ₃ , t ₄ and t ₅ respectively at the end of the first displayable object and the entire web page transmission. At this time, the third acquisition unit 116 is configured to calculate the delay D _i of each stage and send D _{i to the} quality evaluation unit 13 for quality evaluation. During the delay extraction process, a certain unit may not exist. When this occurs, the recording time can be set to the system time recorded by the previous event, and the corresponding delay is also set to 0. For example, when a GET request for a displayable object is not detected, then t ₄ = t ₃ , t ₅ = t ₃ , and D ₃ =0, D ₄ =0 is set.

In this embodiment, the delay extraction unit obtains the loading delays D _i , i=1, 2, 3, 4 of each stage by analyzing the sending process in the web server.

If there is no part of the process in the webpage requested by the user, the judgment of the part is skipped in the process, and the time of the corresponding record is set to the time recorded in the previous judgment and the delay is set to 0. For example, when there is no text in the requested web page, the judgment of the end of the text display is skipped and t ₃ = t ₂ and D ₂ =0 are set.

After the delay extraction is completed, each time delay data is sequentially passed through the browsing quality evaluation in the dotted line frame in FIG. 3, and the quality evaluation is performed in stages. The selection unit 12 selects an appropriate experience quality evaluation function in the evaluation model formula in each stage as shown in the formula (1).

The initial phase of the first phase is currentScore=5, and the currentScore starting values of the remaining phases are f(t) at the end of the previous phase, where t=D. It is also possible to add a delay threshold T _threshold = 5 s, that is, the user will take an interrupt browsing operation when the time delay is greater than 5 s in the phase, and the MOS is divided into a minimum value (such as 0).

FIG. 16 is a schematic structural diagram of another service network service quality automatic evaluation system according to an embodiment of the present invention. The system 2000 includes:

The delay extraction unit 21 is configured to acquire a response delay of at least one response phase of the service network service.

A network service can be divided into multiple response phases, and the response delay of each response phase reflects the quality of experience of users at this stage.

The obtaining unit 22 is configured to obtain an experience quality value and a weighting value for each response phase according to a response delay of each response phase, where the weighting value is according to a response delay of each of the response phases for each response phase The weight of the experience quality value is assigned to the degree of influence.

The impact of each response phase of the network service on the user experience is not the same. Different response delays in each response phase have different impacts on the user experience. Therefore, according to the response delay of each response phase, Obtaining an experience quality value and a weighting value for each response phase, the weighting value being a weight assigned according to the degree of influence of the response delay of each response phase on the quality of experience value of each response phase.

The quality evaluation unit 23 is configured to obtain, according to the weighting value of each response phase, a weighted sum of the quality of experience values of each response phase as the quality of experience value of the service network service.

If an application with a low computational complexity is required, the evaluation process can be simplified to the norm for each stage of quality evaluation at each stage:

MOS=||ω·segment_MOS _i || ^p (3)

When p=1 in the above equation, the MOS value is the weighted sum of the experience values of the segments. ω is a weighted value assigned according to the degree of influence of each response phase.

According to an embodiment of the present invention, a service network service experience quality automatic evaluation system obtains a response delay of each response phase by dividing the service network service into multiple response phases, and then according to the response phase and the response time of the phase The appropriate weighting value is selected, and the weighted sum of the quality of experience values of each response phase is used as the quality of experience value of the service network service, and the user's browsing experience is dynamically evaluated from the perspective of the user's perception, and not only the different responses of the service are considered. The impact of the stage on the experience, and considering the impact of the previous stage of experience transfer to the next stage, so that the quality of the service experience of the business network can be more accurately evaluated.

It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should understand that the present invention is not limited by the described action sequence. Because certain steps may be performed in other sequences or concurrently in accordance with the present invention. In the following, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and units involved are not necessarily required by the present invention.

In the above embodiments, the descriptions of the various embodiments are different, and the details that are not detailed in a certain embodiment can be referred to the related descriptions of other embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented in hardware, firmware implementation, or a combination thereof. When implemented in software, the functions described above may be stored in or transmitted as one or more instructions or code on a computer readable medium. Computer readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one location to another. A storage medium may be any available media that can be accessed by a computer. For example, but not limited to, the computer readable medium may include a random access memory (RAM), a read-only memory (ROM), and an electrically erasable programmable read-only memory (Electrically Erasable Programmable). Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disc storage, disk storage media or other magnetic storage devices, or can be used to carry or store an instruction or data structure. The desired program code and any other medium that can be accessed by the computer. Also. Any connection may suitably be a computer readable medium. For example, if the software is using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (Digital Subscriber Line, DSL) or wireless technology such as infrared, radio and microwave transmission from a website, server or other remote source, then coaxial cable, fiber optic cable, twisted pair, DSL or such as infrared, wireless and microwave Wireless technologies such as those included in the fixing of the associated medium. As used in the present invention, a disk and a disc include a compact disc (CD), a laser disc, a compact disc, a digital versatile disc (DVD), a floppy disk, and a Blu-ray disc, wherein the disc is usually magnetically copied, and the disc is The laser is used to optically replicate the data. Combinations of the above should also be included within the scope of the computer readable media.

In summary, the above description is only a preferred embodiment of the technical solution of the present invention, and is not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Claims (16)

1. An automatic evaluation method for service network service experience quality, characterized in that:

   Obtaining a response delay of at least one response phase of the service network service;

   Selecting, according to the response delay evaluation evaluation of each response stage, the relevant parameters of the experience quality evaluation function or the evaluation function of each response stage from at least one experience quality evaluation function of each response stage;

   Obtaining an experience quality value for each response phase by using the selected quality of experience evaluation function according to each response delay and the quality of experience value of the previous response phase;

   Using the quality of experience value of the previous response phase as the initial quality of experience value of the current response phase, repeating the obtaining step of the quality of experience value for each of the response phases, until the quality of experience value of the last response phase is obtained, as the service network service The quality of the experience.
2. The method of claim 1 wherein the initial quality of experience value of the first response phase is an initial value of the system pre-configured.
3. The method of claim 1 or 2, wherein the at least one quality of experience evaluation function for each of the response stages comprises:

   

   Where f _i (t) is the quality of experience value of the i-th response phase, and T ₁ ⁱ , T ₂ ⁱ , T ₃ ⁱ are three time threshold parameters in the i-th response phase for evaluating functions and parameters Select, currentScore is the quality of experience value f _i-1 (t) of the previous response phase.
4. The method of claim 3 wherein said f _i (t) is:

   

   Wherein, a ₁ , b ₁ , c ₁ , a ₂ , b ₂ , c ₂ , and d are related parameters of the experience quality evaluation in the same response phase, and the experience quality evaluation parameters in different response phases are different or the same.
5. The method according to any one of claims 1 to 4, wherein the obtaining a response delay of at least one response phase of the network service comprises:

   When the client detects the submit request of the URL, the system time t _{1 is} recorded;

   When the client detects the end of the loading title, the system time t _{2 is} recorded;

   When the client detects the end of the loaded text, the system time t _{3 is} recorded;

   When the client detects the end of the first displayable content in the loading page, the system time t _{4 is} recorded;

   When the client detects the end of loading the remaining displayable content, the system time t _{5 is} recorded;

   Obtaining response delays of the respective response phases, the response delays of the respective response phases are D ₁ =t ₂ -t ₁ , D ₂ =t ₃ -t ₂ , D ₃ =t ₄ -t ₃ , D4=t ₅ -t ₄ .
6. The method according to any one of claims 1 to 4, wherein the obtaining a response delay of at least one response phase of the network service comprises:

   When the HTTP GET request for the URL is detected at any network node of the access network or the core network, the system time t _{1 is} recorded;

   After the network node detects the first response message of the HTTP GET of the URL, the system time t _{2 is} recorded;

   After the network node detects the second HTTP GET request, the system time t _{3 is} recorded;

   After the network node detects the last response message of the first picture HTTP GET request, the system time t _{4 is} recorded;

   When the network node detects that the entire page transfer is completed, the system records the time t _5;

   Obtaining response delays of the respective response phases, the response delays of the respective response phases are D ₁ =t ₂ -t ₁ , D ₂ =t ₃ -t ₂ , D ₃ =t ₄ -t ₃ , D4=t ₅ -t ₄ .
7. The method according to any one of claims 1 to 4, wherein the obtaining a response delay of at least one response phase of the network service comprises:

   After the service server receives the first HTTP GET request, the system time t _{1 is} recorded;

   After the service server sends the response containing the header portion in the main file, the system time t _{2 is} recorded;

   After the service server receives the next HTTP GET request, the system time t _{3 is} recorded;

   After the service server sends the end of the first displayable object, the system time t _{4 is} recorded;

   When the service server ends all responses, the system time t _{5 is} recorded;

   Obtaining response delays of the respective response phases, the response delays of the respective response phases are D ₁ =t ₂ -t ₁ , D ₂ =t ₃ -t ₂ , D ₃ =t ₄ -t ₃ , D4=t ₅ -t ₄ .
8. An automatic evaluation method for service network service experience quality, characterized in that:

   Obtaining a response delay of at least one response phase of the service network service;

   Obtaining an experience quality value and a weighting value for each response phase according to a response delay of each response phase, the weighting value being an effect on the quality of experience value of each response phase according to the response delay of each response phase The weight assigned by the degree;

   A weighted sum of the quality of experience values for each response phase is obtained as the quality of experience value of the service network service according to the weighting value of each response phase.
9. An automatic evaluation system for service network service experience quality, characterized in that:

   a delay extraction unit, configured to acquire a response delay of at least one response phase of the service network service;

   a selecting unit, configured to select, according to a response delay of each response stage, an experience quality evaluation function or a correlation parameter of the evaluation function of each response stage from at least one experience quality evaluation function of each response stage;

   a quality evaluation unit, configured to obtain an experience quality value of each response stage by using the selected experience quality evaluation function according to each response delay and the quality of experience value of the previous response stage;

   The quality evaluation unit is further configured to: use the quality of experience value of the previous response phase as the initial quality of experience value of the current response phase, and repeat the obtaining of the quality of experience value of each of the response phases until the quality of the experience of the last response phase is obtained. Value, as the quality of experience value of the service network service.
10. The system of claim 9 wherein the initial quality of experience value for the first response phase is an initial value of the system pre-configured.
11. The system of claim 9 or 10, wherein the at least one quality of experience evaluation function for each of the response phases comprises:

    

    Where f _i (t) is the quality of experience value of the i-th response phase, and T ₁ ⁱ , T ₂ ⁱ , T ₃ ⁱ are three time threshold parameters in the i-th response phase for evaluating functions and parameters Select, currentScore is the quality of experience value f _i-1 (t) of the previous response phase.
12. The system of claim 11 wherein said f _i (t) is:

    

    Wherein, a ₁ , b ₁ , c ₁ , a ₂ , b ₂ , c ₂ , and d are related parameters of the experience quality evaluation in the same response phase, and the experience quality evaluation parameters in different response phases are different or the same.
13. The system according to any one of claims 9 to 12, wherein the delay extraction unit comprises:

    a first recording unit, configured to record a system time t ₁ when the client detects the submit request of the URL;

    The first recording unit is further configured to record the system time t ₂ when the client detects that the loading title ends;

    The first recording unit is further configured to record the system time t ₃ when the client detects that the loading of the text ends;

    The first recording unit is further configured to record the system time t ₄ when the client detects that the first displayable content in the loading page ends;

    The first recording unit is further configured to record the system time t ₅ when the client detects that the remaining displayable content is loaded.

    a first obtaining unit, configured to obtain a response delay of each response phase, where the response delay of each response phase is D ₁ =t ₂ -t ₁ , D ₂ =t ₃ -t ₂ , D ₃ =t ₄ -t ₃ , D4=t ₅ -t ₄ .
14. The system according to any one of claims 9 to 12, wherein the delay extraction unit comprises:

    a second recording unit, configured to record a system time t ₁ when an HTTP GET request for a URL is detected by any network node of the access network or the core network;

    The second recording unit is further configured to record the system time t ₂ after the network node detects the first response message of the HTTP GET of the URL;

    The second recording unit is further configured to record the system time t ₃ after the network node detects the second HTTP GET request;

    The second recording unit is further configured to record the system time t ₄ after the network node detects the last response message of the first picture HTTP GET request;

    The second recording unit is further configured to record the system time t ₅ after the network node detects that the entire webpage is completely transmitted;

    a second obtaining unit, configured to obtain a response delay of each response phase, where the response delay of each response phase is D ₁ =t ₂ -t ₁ , D ₂ =t ₃ -t ₂ , D ₃ =t ₄ -t ₃ , D4=t ₅ -t ₄ .
15. The system according to any one of claims 9 to 12, wherein the delay extraction unit comprises:

    a third recording unit, configured to record the system time t ₁ after the service server receives the first HTTP GET request;

    The third recording unit is further configured to record the system time t ₂ after the service server sends the response including the header portion in the main file;

    The third recording unit is further configured to record the system time t ₃ after the service server receives the next HTTP GET request;

    The third recording unit is further configured to record the system time t ₄ after the service server ends the sending of the first displayable object;

    The third recording unit is further configured to record the system time t ₅ when the service server ends all responses;

    a third obtaining unit, configured to obtain a response delay of each response phase, where the response delay of each response phase is D ₁ =t ₂ -t ₁ , D ₂ =t ₃ -t ₂ , D ₃ =t ₄ -t ₃ , D4=t ₅ -t ₄ .
16. An automatic evaluation system for service network service experience quality, characterized in that:

    a delay extraction unit, configured to acquire a response delay of at least one response phase of the service network service;

    An obtaining unit, configured to obtain an experience quality value and a weighting value of each response phase according to a response delay of each response phase, where the weighting value is according to a response delay of each response phase to each response phase The weight assigned to the degree of impact of the quality value;

    The quality evaluation unit is configured to obtain, according to the weighting value of each response phase, a weighted sum of the quality of experience values of each response phase as the quality of experience value of the service network service.

Images