WO2019101193A1

WO2019101193A1 - Method and apparatus for predicting a quality of experience of a service in a wireless network

Info

Publication number: WO2019101193A1
Application number: PCT/CN2018/117394
Authority: WO
Inventors: Ting Zhu; Chao Wang
Original assignee: Telefonaktiebolaget Lm Ericsson (Publ)
Priority date: 2017-11-27
Filing date: 2018-11-26
Publication date: 2019-05-31

Abstract

Method, apparatus and computer program product for predicting a quality of experience of a service in a wireless network. A method comprises obtaining training data regarding a plurality of first sessions of the service, wherein the training data comprises at least one parameter and a quality level for each of the plurality of first sessions, and the at least one parameter comprises at least one radio parameter; and generating, based on the training data, a prediction model of the quality of experience of the service by using a machine learning method; wherein the prediction model is used to predict the quality of experience of a second session of the service.

Description

METHOD AND APPARATUS FOR PREDICTING A QUALITY OF EXPERIENCE OF A SERVICE IN A WIRELESS NETWORK

Technical Field

Embodiments of the disclosure generally relate to wireless communication, and, more particularly, to method, apparatus and computer program product for predicting a quality of experience (QoE) of a service in a wireless network.

Background

Given the ubiquitous availability of portable wireless devices for various services such as image and video capture and mobile gaming, there has been a dramatic shift towards wireless services such as Long Term Evolution (LTE) video service and over-the-top (OTT) video streaming. For example, half of the billions of daily video views are already on mobile devices. Due to limits on wireless network capacity and increasing consumer users demanding better quality wireless service such as video display services, an end user's QoE has become an essential measure of network performance. QoE refers to a user's holistic perception and satisfaction with a given service such as LTE video service, OTT service and mobile gaming.

Generating an objective model of the service that can accurately predict an end user's QoE can help wireless network providers better understand the service and high-performance trouble shooting to help operators to fix problems, such as user complain, and network benchmark. Such model can also assist in the reduction of network operational costs by encouraging the design and deployment of efficient “quality-aware” network solutions. Therefore, it would be desirable to provide a solution for predicting the QoE of the service in the wireless network.

Summary

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

According to a first aspect of the disclosure, it is provided a method for predicting a quality of experience of a service in a wireless network. The method comprises: obtaining training data regarding a plurality of first sessions of the service, wherein the training data comprises at least one parameter and a quality level for each of the plurality of first sessions, and the at least one parameter comprises at least one radio parameter; and generating, based on the training data, a prediction model of the quality of experience of the service by using a machine learning method; wherein the prediction model is used to predict the quality of experience of a second session of the service.

According to a second aspect of the disclosure, it is provided an apparatus for predicting a quality of experience of a service in a wireless network. The apparatus comprises a processor; and a memory, the memory containing instructions executable by the processor, whereby the apparatus is operative to: obtain training data regarding a plurality of first sessions of the service, wherein the training data comprises at least one parameter and a quality level for each of the plurality of first sessions, and the at least one parameter comprises at least one radio parameter; and generate, based on the training data, a prediction model of the quality of experience of the service by using a machine learning method; wherein the prediction model is used to predict the quality of experience of a second session of the service.

In an embodiment, the method further comprises: identifying a correlation between the quality level and the at least one parameter by using at least one of Pearson’s correlation coefficient, parallel coordinates, random forest and gradient boosting machines.

In an embodiment, the method further comprises: removing a parameter with weak correlation from the training data.

In an embodiment, the apparatus is further operative to identify a correlation between the quality level and the at least one parameter by using at least one of Pearson’s correlation coefficient, parallel coordinates, random forest and gradient boosting machines.

In an embodiment, the apparatus is further operative to remove a parameter with weak correlation from the training data.

According to a third aspect of the disclosure, it is provided an apparatus for predicting a quality of experience of a service in a wireless network. The apparatus comprises an obtaining unit configured to obtain training data regarding a plurality of first sessions of the service, wherein the training data comprises at least one parameter and a quality level for each of the plurality of first sessions, and the at least one parameter comprises at least one radio parameter; and a generating unit configured to generate, based on the training data, a prediction model of the quality of experience of the service by using a machine learning method; wherein the prediction model is used to predict the quality of experience of a second session of the service.

In an embodiment, the apparatus further comprises an identifying unit configured to identify a correlation between the quality level and the at least one parameter by using at least one of Pearson’s correlation coefficient, parallel coordinates, random forest and gradient boosting machines.

In an embodiment, the apparatus further comprises a removing unit configured to remove a parameter with weak correlation from the training data.

According to a fourth aspect of the disclosure, it is provided a computer program product. The computer program product comprises instructions which when executed by at least one processor, cause the at least one processor to obtain training data regarding a plurality of first sessions of the service, wherein the training data comprises at least one parameter and a quality level for each of the plurality of first sessions, and the at least one parameter comprises at least one radio parameter; and generate, based on the training data, a prediction model of the quality of experience of the service by using a machine learning method; wherein the prediction model is used to predict the quality of experience of a second session of the service.

In an embodiment, the computer program product comprises instructions which when executed by at least one processor, cause the at least one processor to identify a correlation between the quality level and the at least one parameter by using at least one of Pearson’s correlation coefficient, parallel coordinates, random forest and gradient boosting machines.

In an embodiment, the computer program product comprises instructions which when executed by at least one processor, cause the at least one processor to remove a parameter with weak correlation from the training data.

According to a fifth aspect of the disclosure, it is provided a computer readable storage medium. The computer readable storage medium comprises instructions which when executed by at least one processor, cause the at least one processor to obtain training data regarding a plurality of first sessions of the service, wherein the training data comprises at least one parameter and a quality level for each of the plurality of first sessions, and the at least one parameter comprises at least one radio parameter; and generate, based on the training data, a prediction model of the quality of experience of the service by using a machine learning method; wherein the prediction model is used to predict the quality of experience of a second session of the service.

In an embodiment, the computer readable storage medium comprises instructions which when executed by at least one processor, cause the at least one processor to identify a correlation between the quality level and the at least one parameter by using at least one of Pearson’s correlation coefficient, parallel coordinates, random forest and gradient boosting machines.

In an embodiment, the computer readable storage medium comprises instructions which when executed by at least one processor, cause the at least one processor to remove a parameter with weak correlation from the training data..

In an embodiment, the at least one radio parameter is collected from a network device of the wireless network.

In an embodiment, the at least one parameter further comprises a parameter obtained from a signaling message for each of the plurality of first sessions.

In an embodiment, the signaling message comprises at least one of session initial protocol message, session description protocol message and realtime transport control protocol message.

In an embodiment, the at least one radio parameter comprise at least one of user equipment (UE) transmission power headroom, channel quality indicator, signal to interference plus noise ratio, the number of physical resource blocks used for the latest uplink transmission, the number of successful radio link control (RLC) protocol data unit (PDU) transmissions in a downlink direction, the number of successful RLC PDU transmissions in an uplink direction, the number of unsuccessful RLC PDU and RLC PDU segment transmissions in the downlink direction, the number of unsuccessful RLC PDU and RLC PDU segment transmissions in the uplink direction, the downlink delay, the total number of successful hybrid automatic repeat request (HARQ) transmissions in the downlink/uplink direction using a specific modulation, the total successfully transferred data volume on media access control (MAC) level in the downlink, a total amount of physical resources used for transmission in the uplink/downlink, UE throughput of the downlink/uplink direction; and the parameter obtained from the signaling message comprises at least one of transmitted octet and packet counts, packet loss, packet delay variation, bit rate, frame arte, profile-level-id, jitter, and round-trip delay time.

In an embodiment, the machine learning method comprises at least one of random forest, gradient boosting machines and neural networks.

In an embodiment, the wireless network comprises a long term evolution (LTE) network, the service comprises a LTE video service and the quality level comprises mean opinion score (MOS) .

These and other objects, features and advantages of the disclosure will become apparent from the following detailed description of illustrative embodiments thereof, which are to be read in connection with the accompanying drawings.

Brief Description of the Drawings

Fig. 1 depicts a schematic system, in which some embodiments of the present disclosure can be implemented;

Fig. 2 is a flow chart depicting a method according to an embodiment of the present disclosure;

Fig. 3 is a flow chart depicting a method according to another embodiment of the present disclosure;

Fig. 4 is a block diagram illustrating an apparatus according to an embodiment of the disclosure; and

Fig. 5 is a block diagram illustrating an apparatus according to another embodiment of the disclosure.

Detailed Description

For the purpose of explanation, details are set forth in the following description in order to provide a thorough understanding of the embodiments disclosed. It is apparent, however, to those skilled in the art that the embodiments may be implemented without these specific details or with an equivalent arrangement.

As used herein, the term “wireless network” refers to a network following any suitable communication standards, such as LTE-Advanced (LTE-A) , LTE, Wideband Code Division Multiple Access (WCDMA) , High-Speed Packet Access (HSPA) , New Radio (NR) and so on. Furthermore, the communications between a terminal device and a network device in the wireless network may be performed according to any suitable generation communication protocols, including, but not limited to, Global System for Mobile Communications (GSM) , Universal Mobile Telecommunications System (UMTS) , Long Term Evolution (LTE) , and/or other suitable the first generation (1G) , the second generation (2G) , 2.5G, 2.75G, the third generation (3G) , the fourth generation (4G) , 4.5G, the future fifth generation (5G) communication protocols, wireless local area network (WLAN) standards, such as the IEEE 802.11 standards; and/or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax) , Bluetooth, and/or ZigBee standards, and/or any other protocols either currently known or to be developed in the future.

The term “network device” refers to a device in a wireless network via which a terminal device accesses the network and receives services therefrom. The network device refers to a base station (BS) , an access point (AP) , or any other suitable device in the wireless network. The BS may be, for example, a node B (NodeB or NB) , an evolved NodeB (eNodeB or eNB) , or gNB, a Remote Radio Unit (RRU) , a radio header (RH) , a remote radio head (RRH) , a relay, a low power node such as a femto, a pico, and so forth. Yet further examples of the network device may include multi-standard radio (MSR) radio equipment such as MSR BSs, network controllers such as radio network controllers (RNCs) or base station controllers (BSCs) , base transceiver stations (BTSs) , transmission points, transmission nodes. More generally, however, the network device may represent any suitable device (or group of devices) capable, configured, arranged, and/or operable to enable and/or provide a terminal device access to the wireless network or to provide some service to a terminal device that has accessed the wireless network.

The term “terminal device” refers to any end device that can access a wireless network and receive services therefrom. By way of example and not limitation, the terminal device refers to a mobile terminal, user equipment (UE) , or other suitable devices. The UE may be, for example, a Subscriber Station (SS) , a Portable Subscriber Station, a Mobile Station (MS) , or an Access Terminal (AT) . The terminal device may include, but not limited to, portable computers, image capture terminal devices such as digital cameras, gaming terminal devices, music storage and playback appliances, a mobile phone, a cellular phone, a smart phone, voice over IP (VoIP) phones, wireless local loop phones, a tablet, a wearable device, a personal digital assistant (PDA) , portable computers, desktop computer, image capture terminal devices such as digital cameras, gaming terminal devices, music storage and playback appliances, wearable terminal devices, vehicle-mounted wireless terminal devices, wireless endpoints, mobile stations, laptop-embedded equipment (LEE) , laptop-mounted equipment (LME) , USB dongles, smart devices, wireless customer-premises equipment (CPE) and the like. In the following description, the terms “terminal device” , “terminal” , “user equipment” and “UE” may be used interchangeably. As one example, a terminal device may represent a UE configured for communication in accordance with one or more communication standards promulgated by the 3rd Generation Partnership Project (3GPP) , such as 3GPP’s GSM, UMTS, LTE, and/or 5G standards. As used herein, a “user equipment” or “UE” may not necessarily have a “user” in the sense of a human user who owns and/or operates the relevant device. In some embodiments, a terminal device may be configured to transmit and/or receive information without direct human interaction. For instance, a terminal device may be designed to transmit information to a network on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the wireless network. Instead, a UE may represent a device that is intended for sale to, or operation by, a human user but that may not initially be associated with a specific human user.

As used herein, a downlink, DL transmission refers to a transmission from the network device to a terminal device, and an uplink, UL transmission refers to a transmission in an opposite direction.

As used herein, a service refers to any suitable service that can be provided by the wireless network, such as video service, audio service, voice service, multimedia service, data service, gaming service, Internet of vehicles service, and OTT service.

References in the specification to “one embodiment, ” “an embodiment, ” “an example embodiment, ” and the like indicate that the embodiment described may include a particular feature, structure, or characteristic, but it is not necessary that every embodiment includes the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

It shall be understood that although the terms “first” and “second” etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and similarly, a second element could be termed a first element, without departing from the scope of example embodiments. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed terms.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be liming of example embodiments. As used herein, the singular forms “a” , “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” , “comprising” , “has” , “having” , “includes” and/or “including” , when used herein, specify the presence of stated features, elements, and/or components etc., but do not preclude the presence or addition of one or more other features, elements, components and/or combinations thereof.

In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skills in the art to which this disclosure belongs.

An end user's QoE is highly subjective and is the result of a combined effect produced by various factors. Most of the existing QoE models are designed on datasets containing real media packets, which causes heavy computation and storage overhead and requires expensive software and hardware. To overcome or mitigate at least one of the above-mentioned problems or other problems, the embodiments of the disclosure propose a solution for predicting the QoE of the service in the wireless network.

It is noted that though the embodiments are mainly described in the context of LTE video service, they are not limited to this but can be applied to any suitable service provided by the wireless network, such as audio service, voice service, multimedia service, data service, gaming service, Internet of vehicles service, and OTT service.

Now some exemplary embodiments of the present disclosure will be described below with reference to the figures.

Fig. 1 depicts a schematic system, in which some embodiments of the present disclosure can be implemented. As shown in Fig. 1, the wireless system 100 comprises a network device 110 such as a cellular base station. The network device 110 may refer to a function element on the network side as compared to a terminal device or UE.For example, the network device 110 may comprise an eNB, a Home eNode B, a femto BS, a pico BS, gNB or any other node capable to serve terminal devices 104-10n in the system 100. It is well known that a cellular radio system may comprise a network of radio cells each served by a transmitting station, known as a cell site or base transceiver station. The radio network provides wireless communications service for a plurality of transceivers (in most cases mobile) . The network of network devices working in collaboration allows for wireless service which is greater than the radio coverage provided by a single network device. The individual network device may be connected by another network (in many cases a wired network, not shown) , which includes additional controllers for resource management and in some cases access to other network systems (such as the Internet) or metropolitan area networks (MANs) . The circle 130 schematically indicates a coverage range of the network device 110.

As shown in Fig. 1, the system 100 may further comprise one or more UEs or terminal devices 104-10n, each of which may operably communicate with the network device 110 such as a cellular base station through a wireless link, such as

link

120 and 124. The terminal devices 104-10n can be fixed or moveable. Terminal devices 104-10n may include, but not limited to, cellular telephones, smart phones, and computers, whether desktop, laptop, or otherwise, as well as mobile devices or terminals such as cellular network UEs, handheld computers, personal digital assistants (PDAs) , wearable devices, video cameras, set-top boxes, personal media devices, or any combinations of the foregoing, which may be provided with wireless communication functionality and run with any kind of operating system including, but not limited to, Windows, Linux, UNIX, Android, iOS and their variants.

In addition, though only one network device 110 is shown in Fig. 1, there may be two or more network devices such that some terminal devices are within the coverage range of first network device, some terminal devices are within the coverage range of second network device, and some terminal devices are at the border of the coverage ranges of two or more network devices, and so on. In the latter case, the terminal devices may receive signals from each of the two or more network devices.

As shown in Fig. 1, the system 100 may further comprise a service provider node 140, which may operably communicate with the terminal devices 104-10n through the network device 110. The service provider node 140 is configured to provide service such as LTE video service to the users of the terminal devices 104-10n. The service provider node 140 can be implemented in form of hardware, software or their combination, including but not limited to, cloud computer, distributed computing system, virtual computer, smart phones, tablets, laptops, servers, thin clients, set-top boxes and PCs. The service provider node 140 may run with any kind of operating system including, but not limited to, Windows, Linux, UNIX, Android, iOS and their variants. It is noted that though the service provider node 140 is shown to be located within the wireless network, it may be located outside the wireless network, for example within another wireless or wired network. In the latter case, the service provided by the service provider node 140 may be referred to as OTT service. In addition, though the service provider node 140 and the network device 110 are shown as two independent entities, they may also be integrated together in some embodiments.

Fig. 2 is a flow chart depicting a method 200 according to an embodiment of the present disclosure, which may be performed at an apparatus, wherein the apparatus may be the network device 110, the service provider node 140, any other suitable entity, or a module of one of them. As such, the apparatus may provide means for accomplishing various parts of the method 200 as well as means for accomplishing other processes in conjunction with other components.

As shown in Fig. 2, the method 200 may start at block 202 where the apparatus obtains training data regarding a plurality of first sessions of the service, wherein the training data comprises at least one parameter and a quality level for each of the plurality of first sessions, and the at least one parameter comprises at least one radio parameter.

The service may be provided by the service provider node 140. As described above, the service may be any suitable service that can be provided by the wireless network, such as video service, audio service, voice service, multimedia service, data service, gaming service, Internet of vehicles service, and OTT service.

The session may be a dialogue, a conversation or a meeting between two or more communicating parties, such as the service provider node 140 and the terminal devices 101-10n. A session is set up or established at a certain point in time, and then torn down at some later point. An established communication session may involve more than one message in each direction. A session is typically, but not always, stateful, meaning that at least one of the communicating parties needs to save information about the session history in order to be able to communicate, as opposed to stateless communication, where the communication consists of independent requests with responses. An established session is the basic requirement to perform a connection-oriented communication. A session also is the basic step to transmit in connectionless communication modes.

The quality level of the service may be measured by the performance of subjective tests or difference analysis between the source and the received streams. For example, the Mean Opinion Score (MOS) test has been used for decades in telephony networks to obtain the user's QoE of a voice service. The MOS provides a numerical indication of the perceived quality from the users' perspective of received media. The MOS is expressed as a single number in the range 1 to 5, where 1 is the lowest perceived quality, and 5 is the highest perceived quality. For example, the MOS data may be obtained from the end-user or a MOS measuring device. In addition to MOS, the quality level may also be measured by other suitable methods in other embodiments.

In this embodiment of the present disclosure, the at least one parameter may comprise at least one radio parameter. In general, the at least one parameter may comprise any radio parameter that can be collected from the wireless network or the UE. In an embodiment, the at least one radio parameter may be collected from the network device of the wireless network. For example, the at least one radio parameter may be collected from the eNodeB in the LTE network by menas of measurement report message and/or a network device-specific message. Different types of network device, different manufacturers' network device, or different versions' network device may provide different radio parameters, therefore the type of the at least one radio parameter may depend on the specific network device.

In an embodiment, the at least one radio parameter may comprise at least one of user equipment (UE) transmission power headroom, channel quality indicator (CQI) , signal to interference plus noise ratio (SINR) , the number of physical resource blocks used for the latest uplink transmission, the number of successful radio link control (RLC) protocol data unit (PDU) transmissions in a downlink direction, the number of successful RLC PDU transmissions in an uplink direction, the number of unsuccessful RLC PDU and RLC PDU segment transmissions in the downlink direction, the number of unsuccessful RLC PDU and RLC PDU segment transmissions in the uplink direction, the downlink delay, the total number of successful hybrid automatic repeat request (HARQ) transmissions in the downlink/uplink direction using a specific modulation, the total successfully transferred data volume on media access control (MAC) level in the downlink, the total amount of physical resources used for transmission in the uplink/downlink, UE throughput of the downlink/uplink direction.

In another embodiment, the at least one parameter may further comprise a parameter obtained from a signaling message for each of the plurality of first sessions. In general, the at least one parameter may comprise any suitable parameter that can be obtained from the signaling message. The type of the signaling message may depend on the specific service. The signaling message may be collected from the service provider node or captured by the apparatus or the network device 110 from which the apparatus can obtained the signaling message. The signaling message may comprise at least one of session initial protocol (SIP) message, session description protocol (SDP) message and realtime transport control protocol (RTCP) message. The SIP message may comprise the identities of two communication entities. The SDP message may comprise information for describing streaming media communications parameters, such as session description, time descriptions and media descriptions. RTCP may provide out-of-band statistics and control information for a realtime transport protocol (RTP) session. For example, the RTCP may provide feedback on the quality of service (QoS) in media distribution by periodically sending statistics information such as transmitted octet and packet counts, packet loss, packet delay variation, jitter, and round-trip delay time to participants in a streaming multimedia session. In an embodiment, the parameter obtained from the signaling message may comprise at least one of transmitted octet and packet counts, packet loss, packet delay variation, bit rate, frame arte, profile-level-id, jitter, and round-trip delay time. The profile-level-id is a parameter in SDP that indicate a degree of required decoder performance for a profile. For example, a level of support within a profile specifies the maximum picture resolution, frame rate, and bit rate that a decoder may use.

The training data may be generated for a particular type of the network device, for the network device at a particular location, for multiple types of network devices for example from different manufacturers.

At block 204, the apparatus may generate, based on the training data, a prediction model of the quality of experience (QoE) of the service by using a machine learning method. The machine learning method may comprise any suitable machine learning methods that can produce the prediction model of QoE of the service by using the training data.

In an embodiment, the machine learning method may comprises at least one of random forest (RF) , gradient boosting machines (GBM) and neural networks. GBM is based on the binary decision tree as base learner algorithm, the upper algorithm is Gradient Boosting Machine, which used for regression and classification problems, which produces a prediction model in the form of an ensemble of weak prediction models, typically decision trees. RF is based on the binary decision tree as base learner algorithm, the upper algorithm is random forest, which is an ensemble learning method for classification, regression and other tasks, that operate by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or mean prediction (regression) of the individual trees. The neural network may be trained to recognize complex associations between inputs and outputs that were presented during a supervised training cycle.

After generating the prediction model, the prediction model can be used to predict the QoE of a second session of the service. For example, the prediction model may be used by the network device such as eNodeB and/or the service provider node to predict the QoE of the service in real time and/or non-real time.

Fig. 3 is a flow chart depicting a method 300 according to another embodiment of the present disclosure, which may be performed at an apparatus, wherein the apparatus may be the network device 110, the service provider node 140, any other suitable entity, or a module of one of them. As such, the apparatus may provide means for accomplishing various parts of the method 300 as well as means for accomplishing other processes in conjunction with other components. For some parts which have been described in the above embodiments, detailed description thereof is omitted here for brevity.

At block 302, the apparatus may obtain training data regarding a plurality of first sessions of the service, wherein the training data comprises at least one parameter and a quality level for each of the plurality of first sessions, and the at least one parameter comprises at least one radio parameter. The block 302 is similar to block 202 of Figure 2, therefore detailed description thereof is omitted here for brevity.

At block 304, the apparatus may identify a correlation between the quality level and the at least one parameter by using at least one of Pearson’s correlation coefficient, parallel coordinates, random forest and gradient boosting machines. Pearson’s correlation coefficient may be used to investigate the dependence between multiple variables at the same time, and the result is a table containing the correlation coefficients between each variable and the others. Parallel coordinates can be used to visualize high-dimensional geometry and analyzing multivariate data, and the result is a coordinate containing the multivariate data and the target result. #The correlation between the quality level and the at least one parameter can be used to guide network optimization and troubleshooting for the service such as LTE video.

For example, by using Pearson’s correlation coefficient or parallel coordinates, the applicant found that, for LTE video, successful HARQ transport by quadrature phase shift keying (QPSK) is more positive correlated with the LTE video MOS; and failed HARQ transport by 64 quadrature amplitude modulation (64QAM) and throughput per user for DL are more negative correlated with the LTE video MOS. By using random forest or gradient boosting machines, the applicant found that, for LTE video, throughput per user for DL is the most influential factor for the LTE video MOS, successful HARQ transport by QPSK is the second influential factor, and the third influential factor is UE transmission power headroom. These results can be used to guide network optimization and troubleshooting for the service such as LTE video.

At block 306, the apparatus may remove a parameter with weak correlation from the training data. The ‘weak correlation’ means that the parameter with weak correlation may have a weaker effect on the quality level of the service. For example, the machine learning method may use n parameters as input, then top n parameters with stronger correlation may be deemed as stronge correlation and the other parameters may be deemed as weak correlation. In this way, it can save computation and storage overhead.

At block 308, the apparatus may generate, based on the training data, a prediction model of the quality of experience of the service by using a machine learning method, wherein the prediction model is used to predict the quality of experience of a second session of the service. The block 308 is similar to block 204 of Figure 2, therefore detailed description thereof is omitted here for brevity.

In various embodiments, the wireless network comprises the LTE network, the service comprises the LTE video service and the quality level comprises MOS.

Fig. 4 depicts an apparatus capable of implementing the methods for predicting a QoE of a service in a wireless network as described above, wherein the apparatus may be implemented by or included in the network device, the service provider node or an independent entity. As shown in Fig. 4, the apparatus 400 comprises a processing device 404, a memory 405, and a transceiver 401 in operative communication with the processor 404. The transceiver 401 comprises at least one transmitter 402 and at least one receiver 403. While only one processor is illustrated in Fig. 4, the processing device 404 may comprises a plurality of processors or multi-core processor (s) . Additionally, the processing device 404 may also comprise cache to facilitate processing operations.

Computer-executable instructions can be loaded in the memory 405 and, when executed by the processing device 404, cause the apparatus 400 to implement the above-described methods for predicting a QoE of a service in a wireless network. In particular, the computer-executable instructions can cause the apparatus 400 to obtain training data regarding a plurality of first sessions of the service, wherein the training data comprises at least one parameter and a quality level for each of the plurality of first sessions, and the at least one parameter comprises at least one radio parameter; and generate, based on the training data, a prediction model of the quality of experience of the service by using a machine learning method; wherein the prediction model is used to predict the quality of experience of a second session of the service.

In an embodiment, the at least one radio parameter comprise at least one of user equipment (UE) transmission power headroom, channel quality indicator, signal to interference plus noise ratio, the number of physical resource blocks used for the latest uplink transmission, the number of successful radio link control (RLC) protocol data unit (PDU) transmissions in a downlink direction, the number of successful RLC PDU transmissions in an uplink direction, the number of unsuccessful RLC PDU and RLC PDU segment transmissions in the downlink direction, the number of unsuccessful RLC PDU and RLC PDU segment transmissions in the uplink direction, the downlink delay, the total number of successful hybrid automatic repeat request (HARQ) transmissions in the downlink/uplink direction using a specific modulation, the total successfully transferred data volume on media access control (MAC) level in the downlink, the total amount of physical resources used for transmission in the uplink/downlink, UE throughput of the downlink/uplink direction.

In an embodiment, the parameter obtained from the signaling message comprises at least one of transmitted octet and packet counts, packet loss, packet delay variation, bit rate, frame arte, profile-level-id, jitter, and round-trip delay time.

In an embodiment, the computer-executable instructions can cause the apparatus 400 to identify a correlation between the quality level and the at least one parameter by using at least one of Pearson’s correlation coefficient, parallel coordinates, random forest and gradient boosting machines.

In an embodiment, the computer-executable instructions can cause the apparatus 400 to remove a parameter with weak correlation from the training data.

Fig. 5 depicts an apparatus capable of implementing the methods for predicting a QoE of a service in a wireless network as described above, wherein the apparatus may be implemented by or included in network device, the service provider node or an independent entity. As shown in Fig. 5, the apparatus 500 comprises an obtaining unit 502 configured to obtain training data regarding a plurality of first sessions of the service, wherein the training data comprises at least one parameter and a quality level for each of the plurality of first sessions, and the at least one parameter comprises at least one radio parameter; and a generating unit 504 configured to generate, based on the training data, a prediction model of the quality of experience of the service by using a machine learning method; wherein the prediction model is used to predict the quality of experience of a second session of the service.

In an embodiment, the parameter obtained from the signaling message comprises at least one of transmitted octet and packet counts, packet loss, packet delay variation, bit rate, frame arte, level id, jitter, and round-trip delay time.

In an embodiment, the apparatus 500 further comprises an identifying unit 506 (optional) configured to identify a correlation between the quality level and the at least one parameter by using at least one of pearson’s correlation coefficient, parallel coordinates, random forest and gradient boosting machines.

In an embodiment, the apparatus 500 further comprises a removing unit 508 (optional) configured to remove a parameter with weak correlation from the training data.

According to an aspect of the disclosure it is provided a computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program instructions stored therein, the computer- executable instructions being configured to, when being executed, cause an apparatus to operate as described above.

According to an aspect of the disclosure it is provided a computer readable storage medium comprising instructions which when executed by at least one processor, cause the at least one processor to perform the method as described above.

The embodiments of the disclosure may have the following advantages. Instead of existing estimation solution involving real media packet such as RTP packet with heavy payload and the specific hardware, in the embodiments of the disclosure, only the signaling/control messages are collected, therefore the overhead of messages may be reduced to 1%and the methods of the disclosure can be run on any common computing device. For example, by avoiding to collect the massive RTP (Real Time Packet) packets, in contrast, the control message for RTP (RTCP) is collected with 5%data volume of total video volume. The embodiments of the disclosure may help to build the expert knowledge database based on machine learning and guideline the network optimization for the service such as video service. In addition, the key factors for the QoE can be identified by machine learning methods, which are more important for better QoE. Moreover, the troubleshooting including the radio and service provider sides is possible to estimate, then the connection between user experience and performance index is transparent and clear.

It is noted that any of the components of the network device and terminal device can be implemented as hardware or software modules. In the case of software modules, they can be embodied on a tangible computer-readable recordable storage medium. All of the software modules (or any subset thereof) can be on the same medium, or each can be on a different medium, for example. The software modules can run, for example, on a hardware processor. The method steps can then be carried out using the distinct software modules, as described above, executing on a hardware processor.

The terms “computer program” , “software” and “computer program code” are meant to include any sequences or human or machine cognizable steps which perform a function. Such program may be rendered in virtually any programming language or environment including, for example, C/C++, Fortran, COBOL, PASCAL, assembly language, markup languages (e.g., HTML, SGML, XML) , and the like, as well as object-oriented environments such as the Common Object Request Broker Architecture (CORBA) , JavaTM (including J2ME, Java Beans, etc. ) , Binary Runtime Environment (BREW) , and the like.

The terms “memory” and “storage device” are meant to include, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the memory or storage device would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM) , a read-only memory (ROM) , an erasable programmable read-only memory (EPROM or Flash memory) , an optical fiber, a portable compact disc read-only memory (CD-ROM) , an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

In any case, it should be understood that the components illustrated herein may be implemented in various forms of hardware, software, or combinations thereof, for example, application specific integrated circuit (s) (ASICS) , functional circuitry, an appropriately programmed general purpose digital computer with associated memory, and the like. Given the teachings of the disclosure provided herein, one of ordinary skill in the related art will be able to contemplate other implementations of the components of the disclosure.

The descriptions of the various embodiments have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

Claims

A method (200, 300) for predicting a quality of experience of a service in a wireless network, comprising:

obtaining (202, 302) training data regarding a plurality of first sessions of the service, wherein the training data comprises at least one parameter and a quality level for each of the plurality of first sessions, and the at least one parameter comprises at least one radio parameter; and

generating (204, 308) , based on the training data, a prediction model of the quality of experience of the service by using a machine learning method;

wherein the prediction model is used to predict the quality of experience of a second session of the service.
The method according to claim 1, wherein the at least one radio parameter is collected from a network device of the wireless network.
The method according to claim 1 or 2, wherein the at least one parameter further comprises a parameter obtained from a signaling message for each of the plurality of first sessions.
The method according to claim 3, wherein the signaling message comprises at least one of session initial protocol message, session description protocol message and realtime transport control protocol message.
The method according to claim 3 or 4, wherein the at least one radio parameter comprise at least one of user equipment (UE) transmission power headroom, channel quality indicator, signal to interference plus noise ratio, a number of physical resource blocks used for the latest uplink transmission, a number of successful radio link control (RLC) protocol data unit (PDU) transmissions in a downlink direction, a number of successful RLC PDU transmissions in an uplink direction, a number of unsuccessful RLC PDU and RLC PDU segment transmissions in the downlink direction, a number of unsuccessful RLC PDU and RLC PDU segment transmissions in the uplink direction, a downlink delay, a total number of successful hybrid automatic repeat request (HARQ) transmissions in the downlink/uplink direction using a specific modulation, a total successfully transferred data volume on media access control (MAC) level in the downlink, a total amount of physical resources used for transmission in the uplink/downlink, UE throughput of the downlink/uplink direction;

wherein the parameter obtained from the signaling message comprises at least one of transmitted octet and packet counts, packet loss, packet delay variation, bit rate, frame arte, profile-level-id, jitter, and round-trip delay time.
The method according to any one of claims 1-5, further comprising:

identifying (304) a correlation between the quality level and the at least one parameter by using at least one of Pearson’s correlation coefficient, parallel coordinates, random forest and gradient boosting machines.
The method according to claim 6, further comprising:

removing (306) a parameter with weak correlation from the training data.
The method according to any one of claims 1-7, wherein the machine learning method comprises at least one of random forest, gradient boosting machines and neural networks.
The method according to any of claims 1-8, wherein the wireless network comprises a long term evolution (LTE) network, the service comprises a LTE video service and the quality level comprises mean opinion score (MOS) .
An apparatus (400) for predicting a quality of experience of a service in a wireless network, comprising:

a processor (404) ; and

a memory (405) , the memory (405) containing instructions executable by the processor (404) , whereby the apparatus (400) is operative to:

obtain training data regarding a plurality of first sessions of the service, wherein the training data comprises at least one parameter and a quality level for each of the plurality of first sessions, and the at least one parameter comprises at least one radio parameter; and

generate, based on the training data, a prediction model of the quality of experience of the service by using a machine learning method;

wherein the prediction model is used to predict the quality of experience of a second session of the service.
The apparatus according to claim 10, wherein the apparatus is operative to perform the method of any one of claims 2 to 9.
An apparatus (500) for predicting a quality of experience of a service in a wireless network, comprising:

an obtaining unit (502) configured to obtain training data regarding a plurality of first sessions of the service, wherein the training data comprises at least one parameter and a quality level for each of the plurality of first sessions, and the at least one parameter comprises at least one radio parameter; and

a generating unit (504) configured to generate, based on the training data, a prediction model of the quality of experience of the service by using a machine learning method;

wherein the prediction model is used to predict the quality of experience of a second session of the service.
The apparatus according to claim 12, wherein the apparatus (500) is operative to perform the method of any one of claims 2 to 9.
A computer program product comprising instructions which when executed by at least one processor, cause the at least one processor to perform the method according to any one of claims 1 to 9.
A computer readable storage medium comprising instructions which when executed by at least one processor, cause the at least one processor to perform the method according to any one of claims 1 to 9.