WO2023024105A1

WO2023024105A1 - Improvement of quality of experience measurement

Info

Publication number: WO2023024105A1
Application number: PCT/CN2021/115139
Authority: WO
Inventors: Jing He; Jing PING
Original assignee: Nokia Shanghai Bell Co., Ltd.; Nokia Solutions And Networks Oy; Nokia Technologies Oy
Priority date: 2021-08-27
Filing date: 2021-08-27
Publication date: 2023-03-02
Also published as: EP4393197A1; CN117882432A

Abstract

A method, device and computer readable storage medium for improvement of QoE measurement. Specifically, in case that the first device (210) determines that there is at least one overlapped metric for the second device (220), the first device (210) may generate the at least one measurement configuration according to some example embodiments of the present disclosure, where a measurement configuration of the least one measurement configuration comprising at least one overlapped metric. Alternatively, in case that the second device (220) determines that there is at least one overlapped metric be configured to the second device (220), when reporting the measurement results, the second device (220) generates at least one measurement report some example embodiments of the present disclosure, wherein a measurement report of the at least one measurement report comprises measurement reports correspond to at least one overlapped metric. In this way, unnecessary and redundant transmission in air interface may be avoided.

Description

IMPROVEMENT OF QUALITY OF EXPERIENCE MEASUREMENT

FIELD

Embodiments of the present disclosure generally relate to the field of telecommunication and in particular, to devices, methods and computer readable storage media for improvement of quality of experience (QoE) measurement.

BACKGROUND

The QoE measurement results indicate objective experience of a user on the satisfaction of communication service (s) in an application (APP) layer. As a result, it is usually used as one of the key indicators for evaluating and assessing the quality of the communication service (s) . Further, a service provider or an operator may obtain the quality of the communication service (s) by collecting the measurement results from a terminal device of the user, and improve the quality of corresponding communication service (s) according to the collected measurement results.

In wireless communication, the management system (MS) may transmit a QoE measurement configuration (such as, QoE measurement collection, QMC) to activate a QoE measurement at the terminal device via an access network device. Further, the QoE measurement configuration may indicate at least one measurement metric. After receiving the QoE measurement configuration, the APP layer of the terminal device will perform the QoE measurement and convey the measurement results to an access layer/stratum of the terminal device. The measurement results will be processed and transmitted to the access network device (such as, a gNB) , and the access network device will forward the measurement results to respective QoE data consumers (such as, message collection entities, MCEs) .

Recently, it is proposed to enable configuring more than one QMC for one certain service type to one terminal device. In this event, the metric (s) comprised in different QMCs may be overlapped. In other words, there may be one or more common metric (s) comprised in different QMCs. As a result, the terminal device will report redundancy and reduplicative measurement results, which brings unexpected and undesirable resource wasting in air interface. Thus, it may be desirable to improve and optimize the QoE procedures, such that the redundancy and reduplicative data transmission (such as, the measurement results) may be avoided.

SUMMARY

In general, example embodiments of the present disclosure provide a solution for improvement of QoE measurement.

In a first aspect, there is provided a first device. The first device comprises at least one processor; and at least one memory including computer program code; where the at least one memory and the computer program code are configured to, with the at least one processor, cause the first device to obtain a first measurement requirement for a QoE measurement at a second device from a first data collector, the first measurement requirement comprising at least one metric; obtain a second measurement requirement for a QoE measurement at the second device from a second data collector, the second measurement requirement comprising at least one metric; and if the at least one metric comprised in the first measurement requirement is at least partially overlapped with the at least one metric comprised in the second measurement requirement, generate at least one measurement configuration based on the first and second measurement requirements, a measurement configuration of the least one measurement configuration comprising at least one overlapped metric.

In a second aspect, there is provided a third device. The third device comprises at least one processor; and at least one memory including computer program code; where the at least one memory and the computer program code are configured to, with the at least one processor, cause the third device to receive, from a first device, a first message for activating a measurement configuration for a QoE measurement at a second device, the first message indicating: the measurement configuration, session identity assigned for a measurement session corresponding to the measurement configuration, and information of a plurality of data collectors corresponding to the measurement configuration, create a correspondence between an identity used for identifying the session identity and the information of the plurality of data collectors; and transmit, based on the received first message, a configuration message to the second device.

In a third aspect, there is provided a second device. The second device comprises at least one processor; and at least one memory including computer program code; where the at least one memory and the computer program code are configured to, with the at least one processor, cause the second device to receive, from a third device, a fourth measurement configuration for a QoE measurement at the second device, the fourth measurement requirement indicating at least one metric and corresponding to a first identity to identity a first session identity; receive, from the third device, a fifth measurement configuration for a QoE measurement at for the second device, the fifth measurement requirement indicating at least one metric and corresponding to a second identity to identity a second session identity; and if the at least one metric indicated by the fourth measurement configuration is at least partially overlapped with the at least one metric indicated by the fifth measurement configuration, generate at least one measurement report based on the fourth and fifth measurement configuration, wherein a measurement report of the at least one measurement report comprises measurement reports correspond to at least one overlapped metric.

In a fourth aspect, there is provided there is provided a third device. The third device comprises at least one processor; and at least one memory including computer program code; where the at least one memory and the computer program code are configured to, with the at least one processor, cause the third device to receive, from a second device, a measurement report message of a QoE measurement at the second device, the measurement report message comprising: a measurement report, and a plurality of identities to identify session identities; and transmit, based on a correspondence between the plurality of identities to identify a plurality of session identities and a plurality of identities of respective data collectors, the measurement report to the respective data collectors.

In a fifth aspect, there is provided a method. The method comprises obtaining, at a first device, a first measurement requirement for a QoE measurement at a second device from a first data collector, the first measurement requirement comprising at least one metric; obtaining a second measurement requirement for a QoE measurement at the second device from a second data collector, the second measurement requirement comprising at least one metric; and if the at least one metric comprised in the first measurement requirement is at least partially overlapped with the at least one metric comprised in the second measurement requirement, generating at least one measurement configuration based on the first and second measurement requirements, a measurement configuration of the least one measurement configuration comprising at least one overlapped metric.

In a sixth aspect, there is provided a method. The method comprises receiving, at a third device and from a first device, a first message for activating a measurement configuration for a QoE measurement at a second device, the first message indicating: the measurement configuration, session identity assigned for a measurement session corresponding to the measurement configuration, and information of a plurality of data collectors corresponding to the measurement configuration, creating a correspondence between an identity used for identifying the session identity and the information of the plurality of data collectors; and transmitting, based on the received first message, a configuration message to the second device.

In a seventh aspect, there is provided a method. The method comprises receiving, at a second device and from a third device, a fourth measurement configuration for a QoE measurement at the second device, the fourth measurement requirement indicating at least one metric and corresponding to a first identity to identity a first session identity; receiving, from the third device, a fifth measurement configuration for a QoE measurement at for the second device, the fifth measurement requirement indicating at least one metric and corresponding to a second identity to identity a second session identity; and if the at least one metric indicated by the fourth measurement configuration is at least partially overlapped with the at least one metric indicated by the fifth measurement configuration, generating at least one measurement report based on the fourth and fifth measurement configuration, wherein a measurement report of the at least one measurement report comprises measurement reports correspond to at least one overlapped metric.

In an eighth aspect, there is provided a method. The method comprises receiving, at a third device and from a second device, a measurement report message of a QoE measurement at the second device, the measurement report message comprising: a measurement report, and a plurality of identities to identify session identities; and transmitting, based on a correspondence between plurality of identities to identify session identities and a plurality of identities of respective data collectors, the measurement report to the respective data collectors.

In ninth aspect, there is provided a first apparatus. The first apparatus comprises means for means for obtaining, at a first apparatus, a first measurement requirement for a QoE measurement at a second apparatus from a first data collector, the first measurement requirement comprising at least one metric; means for obtaining a second measurement requirement for a QoE measurement at the second apparatus from a second data collector, the second measurement requirement comprising at least one metric; and means for generating at least one measurement configuration based on the first and second measurement requirements if the at least one metric comprised in the first measurement requirement is at least partially overlapped with the at least one metric comprised in the second measurement requirement, a measurement configuration of the least one measurement configuration comprising at least one overlapped metric.

In a tenth aspect, there is provided a third apparatus. The third apparatus comprises means for receiving, at a third apparatus and from a first apparatus, a first message for activating a measurement configuration for a QoE measurement at a second apparatus, the first message indicating: the measurement configuration, session identity assigned for a measurement session corresponding to the measurement configuration, and information of a plurality of data collectors corresponding to the measurement configuration; means for creating a correspondence between the identity to identity the session identity and the information of the plurality of data collectors; and means for transmitting, based on the received first message, a configuration message to the second apparatus.

In an eleventh aspect, there is provided a second apparatus. The second apparatus comprises means for means for receiving, at a second apparatus and from a third apparatus, a fourth measurement configuration for a QoE measurement at the second apparatus, the fourth measurement requirement indicating at least one metric and corresponding to a first identity to identity a first session identity; means for receiving, from the third apparatus, a fifth measurement configuration for a QoE measurement at for the second apparatus, the fifth measurement requirement indicating at least one metric and corresponding to a first identity to identity a first session identity; and means for generating at least one measurement report based on the fourth and fifth measurement configuration if the at least one metric indicated by the fourth measurement configuration is at least partially overlapped with the at least one metric indicated by the fifth measurement configuration, wherein a measurement report of the at least one measurement report comprises measurement reports correspond to at least one overlapped metric.

In a twelfth aspect, there is provided a third apparatus. The third apparatus comprises means for means for receiving, at a third apparatus and from a second apparatus, a measurement report message of a QoE measurement at the second apparatus, the measurement report message comprising: a measurement report, and a plurality of identities to identify session identities; and means for transmitting, based on a correspondence between the plurality of identities to identity a plurality of session identities and a plurality of identities of respective data collectors, the measurement report to the respective data collectors.

In a thirteenth aspect, there is provided a computer readable medium. The computer readable medium comprises program instructions for causing an apparatus to perform at least the method according to the fifth aspect.

In a fourteenth aspect, there is provided a computer readable medium. The computer readable medium comprises program instructions for causing an apparatus to perform at least the method according to the sixth aspect.

In a fifteenth aspect, there is provided a computer readable medium. The computer readable medium comprises program instructions for causing an apparatus to perform at least the method according to the seventh aspect.

In a sixteenth aspect, there is provided a computer readable medium. The computer readable medium comprises program instructions for causing an apparatus to perform at least the method according to the eighth aspect.

It is to be understood that the summary section is not intended to identify key or essential features of embodiments of the present disclosure, nor is it intended to be used to limit the scope of the present disclosure. Other features of the present disclosure will become easily comprehensible through the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

Some example embodiments will now be described with reference to the accompanying drawings, where:

Fig. 1A illustrates a conventional signaling flow of process of QMC activation and reporting in LTE;

Fig. 1B illustrates a conventional format of the QMC configuration file;

Fig. 2 illustrates an example communication network in which example embodiments of the present disclosure may be implemented;

Fig. 3 illustrates a signaling chart of an example process of the QoE measurement according to some embodiments of the present disclosure;

Fig. 4 illustrates a signaling chart of another example process of the QoE measurement according to some embodiments of the present disclosure;

Fig. 5 illustrates a flowchart of an example method performed at the first device according to some embodiments of the present disclosure;

Fig. 6 illustrates a flowchart of an example method performed at the third device according to some embodiments of the present disclosure;

Fig. 7 illustrates a flowchart of an example method performed at the second device according to some embodiments of the present disclosure;

Fig. 8 illustrates a flowchart of an example method performed at the third device according to some embodiments of the present disclosure;

Fig. 9 illustrates a simplified block diagram of a device that is suitable for implementing example embodiments of the present disclosure; and

Fig. 10 illustrates a block diagram of an example computer readable medium in accordance with example embodiments of the present disclosure.

Throughout the drawings, the same or similar reference numerals represent the same or similar element.

DETAILED DESCRIPTION

Principle of the present disclosure will now be described with reference to some example embodiments. It is to be understood that these embodiments are described only for the purpose of illustration and help those skilled in the art to understand and implement the present disclosure, without suggesting any limitation as to the scope of the disclosure. The disclosure described herein can be implemented in various manners other than the ones described below.

In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skills in the art to which this disclosure belongs.

References in the present disclosure to “one embodiment, ” “an embodiment, ” “an example embodiment, ” and the like indicate that the embodiment described may include a particular feature, structure, or characteristic, but it is not necessary that every embodiment includes the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

It shall be understood that although the terms “first” and “second” etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and similarly, a second element could be termed a first element, without departing from the scope of example embodiments. As used herein, the term “and/or” includes any and all combinations of one or more of the listed terms.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms “a” , “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” , “comprising” , “has” , “having” , “includes” and/or “including” , when used herein, specify the presence of stated features, elements, and/or components etc., but do not preclude the presence or addition of one or more other features, elements, components and/or combinations thereof.

As used herein, the term "determine/determining" (and grammatical variants thereof) can include, not least: calculating, computing, processing, deriving, measuring, investigating, looking up (for example, looking up in a table, a database or another data structure) , ascertaining and the like. Also, "determining" can include receiving (for example, receiving information) , accessing (for example, accessing data in a memory) , obtaining and the like. Also, "determine/determining" can include resolving, selecting, choosing, establishing, and the like.

As used in this application, the term “circuitry” may refer to one or more or all of the following:

(a) hardware-only circuit implementations (such as implementations in only analog and/or digital circuitry) and

(b) combinations of hardware circuits and software, such as (as applicable) :

(i) a combination of analog and/or digital hardware circuit (s) with software/firmware and

(ii) any portions of hardware processor (s) with software (including digital signal processor (s) ) , software, and memory (ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions) and

(c) hardware circuit (s) and or processor (s) , such as a microprocessor (s) or a portion of a microprocessor (s) , that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.

This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and/or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.

As used herein, the term “communication network” refers to a network following any suitable communication standards, such as Long Term Evolution (LTE) , LTE-Advanced (LTE-A) , Wideband Code Division Multiple Access (WCDMA) , High-Speed Packet Access (HSPA) , Narrow Band Internet of Things (NB-IoT) and so on. Furthermore, the communications between a terminal device and a network device in the communication network may be performed according to any suitable generation communication protocols, including, but not limited to, the first generation (1G) , the second generation (2G) , 2.5G, 2.75G, the third generation (3G) , the fourth generation (4G) , 4.5G, the future fifth generation (5G) communication protocols, and/or any other protocols either currently known or to be developed in the future. Embodiments of the present disclosure may be applied in various communication systems. Given the rapid development in communications, there will of course also be future type communication technologies and systems with which the present disclosure may be embodied. It should not be seen as limiting the scope of the present disclosure to only the aforementioned system.

As used herein, the term “access network device” refers to a node in a communication network via which a terminal device accesses the network and receives services therefrom. The network device may refer to a base station (BS) or an access point (AP) , for example, a node B (NodeB or NB) , an evolved NodeB (eNodeB or eNB) , a NR NB (also referred to as a gNB) , a Remote Radio Unit (RRU) , a radio header (RH) , a remote radio head (RRH) , a relay, a low power node such as a femto, a pico, and so forth, depending on the applied terminology and technology.

The term “terminal device” refers to any end device that may be capable of wireless communication. By way of example rather than limitation, a terminal device may also be referred to as a communication device, user equipment (UE) , a Subscriber Station (SS) , a Portable Subscriber Station, a Mobile Station (MS) , or an Access Terminal (AT) . The terminal device may include, but not limited to, a mobile phone, a cellular phone, a smart phone, voice over IP (VoIP) phones, wireless local loop phones, a tablet, a wearable terminal device, a personal digital assistant (PDA) , portable computers, desktop computer, image capture terminal devices such as digital cameras, gaming terminal devices, music storage and playback appliances, vehicle-mounted wireless terminal devices, wireless endpoints, mobile stations, laptop-embedded equipment (LEE) , laptop-mounted equipment (LME) , USB dongles, smart devices, wireless customer-premises equipment (CPE) , an Internet of Things (loT) device, a watch or other wearable, a head-mounted display (HMD) , a vehicle, a drone, a medical device and applications (e.g., remote surgery) , an industrial device and applications (e.g., a robot and/or other wireless devices operating in an industrial and/or an automated processing chain contexts) , a consumer electronics device, a device operating on commercial and/or industrial wireless networks, and the like. In the following description, the terms “terminal device” , “communication device” , “terminal” , “user equipment” and “UE” may be used interchangeably.

The term “MS device” refers to any device or entity that provides management functions in the core network (CN) . By way of example rather than limitation, the MS device may be a device providing functions of operation, administration and maintenance (also referred to as “OAM device” ) , a home subscriber server (HSS) , a mobility management entity (MME) , an access and mobility management function (AMF) , a session management function (SMF) , user plane function (UPF) , etc. In other embodiments, the MS device may be any other suitable device or entity.

Although functionalities described herein can be performed, in various example embodiments, in a fixed and/or a wireless network node may, in other example embodiments, functionalities may be implemented in a user equipment apparatus (such as a cell phone or tablet computer or laptop computer or desktop computer or mobile IOT device or fixed IOT device) . This user equipment apparatus can, for example, be furnished with corresponding capabilities as described in connection with the fixed and/or the wireless network node (s) , as appropriate. The user equipment apparatus may be the user equipment and/or or a control device, such as a chipset or processor, configured to control the user equipment when installed therein. Examples of such functionalities include the bootstrapping server function and/or the home subscriber server, which may be implemented in the user equipment apparatus by providing the user equipment apparatus with software configured to cause the user equipment apparatus to perform from the point of view of these functions/nodes.

A work item (WI) was conducted in the third Generation Partnership Project (3GPP) for QoE measurement in New Radio (NR) . The detailed objectives of the WI are as follows:

● Specify the support for QoE measurement collection in NR standalone mode.

Specify configuration, activation, and deactivation procedures for both signaling-based and management-based QoE measurement collection and reporting, taking LTE QoE solutions as baseline, as defined in the 3GPP standard of TR 38.890.

Specify configuration and reporting for multiple simultaneous QoE measurements at a UE.

Specify The QoE measurement handling in a situation of RAN overload, including pause and resume of QoE measurement reporting.

Specify The QoE measurement handling in RRC_INACTIVE, i.e., keeping the QoE measurement configuration without measuring and reusing the same configuration upon transition from RRC_INACTIVE to RRC_CONNECTED.

● The support for QoE measurement collection and reporting continuity in intra-system intra-RAT mobility scenario for signaling based QoE.

● To support RAN visible QoE, evaluate and specify an initial relevant set of RAN-visible QoE parameters, then specify configuration and reporting.

● Specify the support for per-slice QoE measurement.

● Specify the necessary mechanism to support alignment of radio-related measurement (i.e., minimisation of drive tests, MDT) and QoE measurement.

Reference is made to Fig. 1A, which illustrates a conventional signaling flow of process 100 of QMC activation and reporting in LTE. The process 100 comprises the following operations:

1. The network manager (NM) sends activateAreaQMCJob to Domain Management/Element Management (DM/EM) that controls the eNB, and the activateAreaQMCJob includes the parameters: Service Type, AreaScope, QoE CE Address, PLMN target, QoE Target, QoE Reference and QMC configuration file.

2. The DM/EM forwards activateAreaQMCJob to the eNB, and the activateAreaQMCJob includes the parameters: Service Type, AreaScope, QoE CE Address, PLMN target, QoE Target, QoE Reference and QMC configuration file.

3. The eNB finds a UE that matches the criteria.

4. The eNB sends the RRCConnectionReconfiguration message to the UE, and the RRCConnectionReconfiguration message includes the following: Service Type, QoE Reference and QMC configuration file.

5. The access stratum in the UE sends the AT command +CAPPLEVMC to application level and the AT command +CAPPLEVMC includes the following: Service Type, QoE Reference and QMC configuration file.

6. The application at the UE is started, and the QoE measurement collection is initiated.

7. The APP stratum of the UE sends the AT command +CAPPLEVMR including a Recording Session Indication to the access stratum of the UE.

8. The UE sends the message MeasurementReport including the Recording Session Indication to the eNB.

9. The eNB sends a Notification to the NM.

10. When the QoE measurement collection is completed, the application layer sends the AT command +CAPPLEVMR including QoE Reference and the Report to the access stratum. The Report includes QoE Reference and Recording Session Indication.

11. The UE sends the message MeasurementReport including QoE Reference and the Report to the eNB.

12. The eNB sends the report to the MCE associated to the QoE Reference. The Report includes QoE Reference and Recording Session Indication.

According to the conventional solution, the NM may inform/indicate the objects to be measured through the QMC configuration file (such as, activateAreaQMCJob) . Further, according to the conventional solution when configuring/activating the QoE for the terminal device, each activateAreaQMCJob can merely include one QoE CE Address and one QoE Reference, which means that one QoE Reference merely may be corresponded to one MCE. Further, when reporting the measurement results, one MeasurementReport merely may include one QoE Reference (which is associated with one MCE) , which means that one MeasurementReport may be only transmitted to one MCE.

In release 17 of 3GPP, there will be analytics, assurance and other automated functions using the QMC mechanism in 5G, and further those automated functions may need to collect different QoE measurement results from the same UE. As a result, it is proposed to enable configuring more than one QMC for one certain service type to one terminal device.

According to the conventional solution as illustrated in Fig. 1A, when multiple QMCs are required by different MCEs (for example, different automated functions) , the measurement results are needed to be sent to the different MCEs. Specifically, multiple QMCs for one service type may be configured to one gNB and subsequently to one certain UE in cell. All the configured QMCs are performed in UE’s application layer and the measurement results of each configured QMC are reported to the gNB via QoE reporting container (s) with respective QoE Reference ID and Service Type. The gNB will forward each received QoE reporting container to one of the different MCEs according to QoE Reference ID attached to the container.

Further, to support multiple MCEs served by one or multiple operators, the MS of an operator may configure multiple QMCs to one UE for one service type to support different QoE metrics requirements of different MCEs. Reference is made to Fig. 1B, which illustrates an example conventional format 150 of the QMC configuration file.

According to the conventional format 150 of QMC configuration file as illustrated in Fig. 1B, when multiple QMCs are configured, these QMCs may have overlapped/common measurement metrics for one certain service type.

One specific example is illustrated by referring to Fig. 1B. In the specific example, for Multimedia Telephony Service for IMS (MTSI) service type, the UE is configured to measure QoE metrics for “Speech” and “Video” (i.e., , indicated in QoE configuration file 1) and the measurement results (i.e., QoE reporting container 1) is configured to be reported to a first MCE (i.e., MCE 1) . Further, the UE is also configured to measure metrics for “Video” and “Text” (i.e., , indicated in QoE configuration file 2) and the measurement result (i.e., QoE reporting container 2) is configured to be reported to a second MCE (i.e., MCE 2) . Therefore, the UE configured to measure metrics for “Video” twice. According to the conventional solution, the measurement result of “Video” will be reported twice through two reporting containers to the gNB, which brings unexpected and undesirable resource wasting in air interface.

The above specific example is discussed merely for the purpose of illustration. In real communication scenario, in addition to “Text” , “Video” and “Speech” , there are still other metrics that may be configured to the UE. Further, as illustrated in Fig. 1B, for any of “Text” , “Video” and “Speech” , more than one metric may be further configured. Therefore, when multiple MCEs are enabled, it would be a common scenario that the metrics indicated by some of the MCEs are at least partially overlapped with the metrics indicated by the other MCE (s) . Further, the QoE measurement reporting data is transmitted to gNB by low priority SRB compared with normal RRC signaling. Thus, the QoE measurement data in uplink should be optimized to avoid unnecessary and duplicative data transmission in air interface to save resource as much as possible.

In view of the above, the procedures of configuring QoE measurement and reporting QoE measurement results are desirable to be improved and optimized.

In accordance with some example embodiments of the present disclosure, there is provided a solution for improvement of QoE measurement. In this solution, avoiding unnecessary data redundant transmission of measurement results in air interface may be achieved either at the first device (such as, a MS device) or at the second device (such as, a terminal device) . Specifically, in case that the first device determines that there is at least one overlapped/common metric, the first device may generate the at least one measurement configuration according to some example embodiments of the present disclosure, where a measurement configuration of the least one measurement configuration comprising at least one overlapped metric. In this way, the overlapped/common metric (s) is indicated separately and the unnecessary and duplicative data transmission is avoided thereby. Alternatively, in case that the second device determines that there is at least one overlapped metric/common metric, when reporting the measurement results, the second device generates at least one measurement report according to some example embodiments of the present disclosure, where a measurement report of the at least one measurement report comprises measurement reports correspond to at least one overlapped metric. In this way, the measurement results correspond to the overlapped metric (s) may be transmitted once and the unnecessary and duplicative data transmission is avoided thereby. The above proposed processes will be discussed in detail in the following.

Fig. 2 shows an example communication network 200 in which example embodiments of the present disclosure can be implemented.

The communication network 200 includes a first device 210, a second device 220, a third device 230, data collectors 240-1 and 240-2 (hereinafter collectively referred to as the data collectors 240 or individually referred to as a data collector 240) . Further, in the following text, the data collector 240-1 also referred to as the first data collector 240-1 while the data collector 240-2 also referred to as the second data collector 240-2.

In the communication environment 200, a first device 210 can communicate with a second device 220 and a third device 230 via physical communication channels or links.

In the example of Fig. 2, the first device 210 is illustrated as a MS device, the second device 220 is illustrated as terminal device while the third device 230 is illustrated as access network device serving the second device 220.

In the specific example of Fig. 2, either the first data collector 240-1 or the second first data collector 240-2 may transmit a measurement requirement to the first device 210, where in the measurement requirement comprise at least one metric. In one example, the first device 210 provides a public interface and the first data collector 240-1 or the second first data collector 240-2 invokes the public interface to create the respective measurement requirement. In another example, the first data collector 240-1 or the second first data collector 240-2 transmits a message with a pre-defined structure to the first device 210 to indicate the respective measurement requirement.

In some example embodiments, in addition to the metric (s) , each measurement requirement also may include information of the data collector 240. One example of the information of the data collector 240 is an identifier of the data collector 240. Another example of the information of the data collector 240 is an Internet Protocol (IP) address or media access control (MAC) address of the data collector 240. In other examples, the information of the data collector 240 may be any identity that may be used for identifying the data collector 240.

In some example embodiments, the first device 210 may generate measurement configuration (s) based on the measurement requirement (s) , and then transmit the generated measurement configuration (s) to the third device 230.

The third device 230 may forward the measurement configuration (s) to the second device 220. Then, the second device 220 (i.e., the APP layer of the second device 220) performs corresponding QoE measurement. The QoE measurements results are transmitted to the third device 230 and then forwarded to respective data collector (s) 240.

It also should be appreciated that the numbers of first device, second device, third device and data collector are only for the purpose of illustration without suggesting any limitations. The communication environment 200 may include any suitable numbers of first device, second device, third device and data collector adapted for implementing embodiments of the present disclosure.

Principle and implementations of the present disclosure will be described in detail below with reference to Figs. 3 and 4.

In the following text, the example processes are discussed with regards to two measurement requirements (i.e., the first and second measurement requirements) . It is to be understood that the specific number (i.e., two) of the measurement requirements are only for the purpose of illustration without suggesting any limitations. Actually, the example processes discussed herein may be suitable to the scenario of any suitable number of measurement requirements by adaptive adjustment.

Similarly, the example processes discussed with regards to two measurement configurations (i.e., the fourth and fifth measurement configurations) also are for the purpose of illustration without suggesting any limitations. Actually, the example processes discussed herein may be suitable to the scenario of any suitable number of measurement configurations by adaptive adjustment.

As discussed above, avoiding unnecessary and redundant measurement result transmission in air interface may be achieved either at the first device 210 (such as, a MS device) through an improved configuring procedure or at the second device 220 (such as, a terminal device) through an improved reporting procedure. In following text, these two manners will be discussed in detail separately.

It is to be understood that the below illustrated IEs/parameters are merely for purpose of illustration without any limitation. In some other embodiments, when any of the information element (IE) /parameter is pre-configured or may be obtained from another manner, the IE/parameter may be absent, while some additional parameters may be added due other specific scenarios or requirements.

It is to be understood that the specific values of IEs/parameters (such as, service type, metrics, session identity, measurement configuration, measurement results and so on) illustrated in below specific examples are only for the purpose of illustration without suggesting any limitations. In other example embodiments, the (IE) s/parameters may be any suitable values.

In some example embodiments, when the third device 230 receives a first message which comprises such as, a measurement configuration, information of the data collector 240 and a session identity, the third device 230 will generate a configuration message based on the first message and transmit the configuration message to the second device 220. In some of the example embodiments, the third device 230 indicates information associated with the session identity to the second device 220. In one example, the third device 230 transmits the session identity to the second device 220 directly. In another example, the third device 230 will configure one or more identities/identifications (such as, RRC ID and the like) that correspond to the session identity/the data collector (s) 240, and transmit the one or more identifications rather than the session identity to the second device 220. Accordingly, the second device 220 will attach the one or more identifications rather than the session identity to the measurement report. It should be understood that the identifications may be used to identify the session and may be represented in any suitable manner.

Further, the third device 230 also may create a correspondence or an association between the one or more identifications and the data collector (s) 240/session identities, such that, when receiving the measurement report message, the third device 230 may transmit the measurement report to the respective data collector (s) 240 based on the created correspondence.

Therefore, as for the communication between the second device 220 and the third device 230, the session identity and the one or more identifications are equivalent with each other. In the following text, examples where the third device 230 transmits the session identity in the configuration message are illustrated only for the purpose of illustration. It should be understood that the session identity communicated between the second device 220 and the third device 230 may be replaced with the one or more identifications equivalently. Merely for brevity, the same or similar descriptions are omitted here.

Example processes for the improved configuring procedure

Principle and implementations for the improved configuring procedure of the present disclosure will be described in detail below with reference to Fig. 3, which illustrates a signaling chart of an example process 300 of the QoE measurement according to some embodiments of the present disclosure.

For the purpose of discussion, the example process 300 will be described with reference to Fig. 2. The example process 300 may involve the first device 210, the second device 220, the third device 230, the first data collector 240-1 and the second data collector 240-2.

In operation, the first device 210 may obtain 305 a first measurement requirement for a QoE at the second device 220 from the first data collector 240-1, where the first measurement requirement comprises at least one metric. Further, as illustrated in Fig. 3, the first device 210 further obtains 320 a second measurement requirement for a QoE at the second device 220 from the second data collector 240-2, where the second measurement requirement comprise at least one metric. In one example, the first device 210 obtains the first and second measurement requirements by receiving respective messages from the first data collector 240-1 and the second data collector 240-2. In another example, the first device 210 provides a public interface, and the first data collector 240-1 and the second first data collector 240-2 invoke the public interface to create respective measurement requirement.

In addition to the metric (s) , some IEs/parameters also may be comprised in the first and second measurement requirements. One example of the IEs/parameters is service type (such as, dynamic adaptive streaming over hypertext transfer protocol (HTTP) , DASH) . Another example of the IEs/parameters is information of the data collector 240 (such as, QoE CE Address) . As discussed, the information of the data collector 240 may be identifier, IP address, MAC address and any suitable identity that may be used for identifying the data collector 240.

Two specific examples of part of the first and second measurement requirements are illustrated as below.

specific example of part of the first measurement requirement

QoE data request

- Service Type (DASH)

- QoE CE Address (MCE1)

- QoE metrics (Video, Speech)

specific example of part of the second measurement requirement

QoE data request

- Service Type (DASH)

- QoE CE Address (MCE2)

- QoE metrics (Video, Text)

After obtaining the first and second measurement requirements, the first device 210 may determine whether the at least one metric comprised in the first measurement requirement is at least partially overlapped (including fully overlapped or partially overlapped) with the at least one metric comprised in the second measurement requirement. In other words, the first device 210 may determine whether there are common metric (s) comprised by both the first and second measurement requirements. Additionally, in some embodiments, the first device 210 performs the determining operation only in case that the first and second measurement requirements comprise the same value of Service Type. In this way, the overlapped/common metric (s) may be quickly determined without introducing significant additional calculation since different Service Types usually correspond to different metrics.

In some example embodiments, if there are no overlapped or common metrics, the first device 210 generates two measurement configurations corresponding to the first and second measurement requirements, respectively.

In some example embodiments, if the at least one metric comprised in the first measurement requirement is at least partially overlapped with the at least one metric comprised in the second measurement requirement, the first device 210 generates 325 at least one measurement configuration based on the first and second measurement requirements. In particular, a measurement configuration of the least one measurement configuration comprises at least one overlapped metric.

In some example embodiments, as for the first and second measurement requirements, three measurement configurations may be generated. Specifically, the first device 210 may generate a first measurement configuration indicating metrics comprised in the first measurement requirement while not comprised in the second measurement requirement, a second measurement configuration indicating metrics comprised in the second measurement requirement while not comprised in the first measurement requirement, and a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.

Different from the conventional solution where the first device 210 generates two measurement configurations corresponding to the first and second measurement requirements respectively, the first device 210 restructures the first and second measurement requirements according to the present discourse, such that each metric is comprised/indicated merely one time.

Further, the overlap relationship of the metric may be various, which may allow some of the above first, second and third measurement configurations to not be generated. Specifically, in some scenarios, the first or second measurement configuration would not be generated. Some example embodiments for such scenario will be discussed below.

In some example embodiments, if the at least one metric comprised in the first measurement requirement is a subset of the at least one metric comprised in the second measurement requirement, the first device 210 needs to generate the second and the third measurement configurations.

Alternatively, in some example embodiments, if the at least one metric comprised in the second measurement requirement is a subset of the at least one metric comprised in the first measurement requirement, the first device 210 needs to generate the first and third measurement configurations.

Alternatively, in some example embodiments, if the at least one metric comprised in the first measurement requirement is identical with the at least one metric comprised in the second measurement requirement, the first device 210 merely needs to generate the third configuration indicating metrics comprised in both the first and second measurement requirements.

Further, in some example embodiments, each measurement configuration may correspond to a respective measurement session. During the measurement session, the measurement configuration may be configured, while the measurement results associated with the measurement configuration may be reported.

In some example embodiments, a session identity may be determined/assigned by the first device 210. In the above specific example, a first session identity (represented as “ID3-1” ) is determined by the first device 210 assigned for a measurement session corresponding to the first measurement configuration. Similarly, a second session identity (represented as “ID3-2” ) is determined by the first device 210 assigned for a measurement session corresponding to the second measurement configuration, and a third session identity (represented as “ID3-c” ) is determined by the first device 210 assigned for a measurement session corresponding to the third measurement configuration.

Then, the first device 210 may transmit at least one first message to activate the respective generated measurement configuration (s) (such as, the first, second and third configurations) .

In some example embodiments, the generated measurement configuration is transmitted via a container comprised in the first message. That is, the measurement configuration is transparent with respect to the third device 230.

In some example embodiments, in addition to the measurement configuration, the first message also comprises further some other IEs/parameters. Examples of the other IEs/parameters includes but not limited to, service type and information of at least one data collector 240 corresponding to the respective measurement configuration which are obtained/derived from the measurement requirement.

Additionally, in some example embodiments, the first message also comprises the session identity assigned for the measurement session corresponding to the respective measurement configuration.

In some example embodiments, the first message for activating the third measurement configuration indicates: the first measurement configuration, information of the first collector 240-1, and the first session identity. Similarly, the first message for activating the second measurement configuration indicates: the second measurement configuration, information of the second collector 240-2, and the second session identity, while the first message for activating the third measurement configuration indicates: the third measurement configuration, information of both the first data collector 240-1 and the second collector 240-2, and the third session identity.

Three specific examples of part of the first messages corresponding to the first, second and third measurement configurations are illustrated as below.

specific example of part of the first message corresponding to the first measurement requirement

activateAreaQMCJob

- Service Type (DASH)

- QoE CE Address (MCE1)

- QoE reference ID (ID3-1)

- Container of QoE Config (Speech)

specific example of part of the first message corresponding to the second measurement requirement

activateAreaQMCJob

- Service Type (DASH)

- QoE CE Address (MCE1)

- QoE reference ID (ID3-2)

- Container of QoE Config (Text)

specific example of part of the first message corresponding to the third measurement requirement

activateAreaQMCJob

- Service Type (DASH)

- QoE CE Address (MCE1, MCE2)

- QoE reference ID (ID3-c)

- Container of QoE Config (Text)

Then the generated first message (s) is transmitted to the third device 230. In the specific example of Fig. 3, the first device 210 transmits 340-3 the first message corresponding to the third measurement configuration to the third device 230. Further, if the first measurement configuration is generated, the first device 210 will transmit 340-1 the first message corresponding to the first measurement configuration to the third device 230, while if the second measurement configuration is generated, the first device 210 will transmit 340-2 the first message corresponding to the second measurement configuration to the third device 230.

In addition, in the actual communication scenario, the first measurement requirement and the second measurement requirement usually are obtained at different time points. That is, when obtaining the second measurement requirement, the first measurement requirement may have been processed by the first device 210. For example, the first device 210 may have generated a respective first message to activate the first measurement requirement. As one specific example, the first device 210 has transmitted 310 a first message (such as, activateAreaQMCJob) to the third device 230, and the first message includes the parameters: Service Type (DASH) , QoE CE Address (MCE1) , QoE Reference (ID1) and Container of QMC configuration file (Video, Speech) . Further, the third device 230 also may have processed the first message, for example, has transmitted 315 a RRCReconfiguration message to the second device 220, and the RRCReconfiguration message includes the following: Service Type (DASH) , QoE Reference (ID1) and container of QMC configuration file (Video, Speech) .

In case that the first measurement requirement has been activated by the first device 210, some additional operations are needed. Specifically, the first device 210 needs to transmit 330 a second message to deactivate the first measurement requirement.

As one specific example, the first device 210 transmits a second message (such as, dectivateAreaQMCJob) to the third device 230, and the second message includes the parameters: Service Type (DASH) and QoE Reference (ID1) . Similarly, in case that the third device 230 has configured the first measurement requirement to the second device 220 (such as, has transmitted the above RRCReconfiguration message) , some additional operations are needed by the third device 230. Specifically, the third device 230 needs to transmit 335 a RRCReconfiguration message to the second device 220, and the RRCReconfiguration message includes the following: Service Type (DASH) , QoE Reference (ID1) and Other Config (measConfigAppLayer=discard) . When the second device 220 receives this RRCReconfiguration, the second device 220 may terminate the QoE measurement accordingly.

In this way, even if the first measurement requirement has been activated by the first device 210, the improved solution for handling overlapped/common metric (s) also may be proceeded.

Continue to refer to Fig. 3, after receiving the respective first message, the third device 230 stores the session identity (such as, QoE reference Id) , the information of the data collector (s) 240 and optionally stores other visible QMC parameters. Further, the third device 230 may create a correspondence or an association between the session identity and the information of the data collector (s) 240. Then, the third device 230 forwards a configuration message to the second device 220 based on the received first message. For example, the third device 230 transmits an RRCReconfiguration message to the second device 220, and the RRCReconfiguration message includes the following: Service Type, QoE reference ID and Container of QoE Config which are comprised in the first message.

Additionally, the third device 230 also may configure one or more identifications corresponding to the session identities/the data collector (s) 240. In this event, the third device 230 may transmit the identification (s) rather than the session identity to the second device 220. As one specific example, the third device 230 transmits RRC signaling for QoE measurements in RRC_CONNECTED, where identification (s) (such as, a RRC ID or MeasConfigAppLayerId) is used to identify the QoE configuration between the second device 220 and the third device 230.

More specifically, the third device 230 configures a first identity/identification corresponding to the first session identity, configures a second identity/identification corresponding to the second session identity, while configuring two identifications (referred to as a third identification and a fourth identification) /identities corresponding to the third session identity.

In case that the third device 230 receives the first message corresponding to the first measurement configuration, the third device may create 345-1 a correspondence between the first session identity (or the first identification) and the information of the first data collector 240-1. Further, the third device 230 transmits 350-1 an RRCReconfiguration message to the second device 220, and the RRCReconfiguration message includes the following: Service Type (DASH) , QoE reference ID (ID3-1) (or the first identification) and Container of QoE Config comprised (Speech) .

Similarly, in case that the third device 230 receives the first message corresponding to the second measurement configuration, the third device 230 may create 345-2 a correspondence between the second session identity (or the second identification) and the information of the second data collector 240-3, Further, the third device 230 transmits 350-2 an RRCReconfiguration message to the second device 220, and the RRCReconfiguration message includes the following: Service Type (DASH) , QoE reference ID (ID3-2) (or the second identification) and Container of QoE Config comprised (Text) .

Finally, in case that the third device 230 receives the first message corresponding to the third measurement configuration, the third device 230 may create 345-3 a correspondence between the third session identity and the information of both the first data collector 240-1 and the second data collector 240-2. Further, the third device 230 transmits 350-3 an RRCReconfiguration message to the second device 220, and the RRCReconfiguration message includes the following: Service Type (DASH) , QoE reference ID (ID3-c) and Container of QoE Config comprised (Video) .

Alternatively, when the identification (s) are used for communications between the second device 220 and the third device 230, there may be two identifications (i.e., a third identification and a fourth identification) associated with the third session identity. The third device 230 may create a correspondence between the third and fourth identifications and the information of the first and second data collectors 240. For example, create a correspondence between the third identification and the information of the first data collector 240-1, and a correspondence between the fourth identification and the information of the second data collector 240-2. After that, the third device 230 transmits 350-3 an RRCReconfiguration message to the second device 220, and the RRCReconfiguration message includes the following: Service Type (DASH) , QoE reference ID (ID3-c) (or the third and fourth identifications) and Container of QoE Config comprised (Video) .

In this way, one session identity may correspond to more than one data collector.

After receiving the respective RRCReconfiguration message, the second device 220 may perform respective QoE measurement and obtains the measurement results. The measurement results (i.e., measurement report) may be transmitted via a container to the third device 230. Specifically, the second device 220 may generate a measurement report message comprising session identity (such as, QoE reference ID) and the measurements report (such as, Container of Report) .

Then, after receiving the measurement report message comprising the session identity, the third device 230 may transmit the measurement report to the respective data collector (s) based on the created correspondence between the session identity and the information of the data collectors 240.

As illustrated in the Fig. 3, if a measurements report message (such as, MeasReportAppLayer comprising the first session identity (such as, QoE reference ID (ID3-1) ) (or a first identity to identify the first session identity) and the measurements report (such as, Container of Report (Speech) ) is transmitted 355-1 to the third device 230, the third device 230 determines 360-1 that the first session identity (or a first identity to identify the first session identity) is corresponding the first data collector 240-1, and forwards 365-1 the measurement report to the first data collector 240-1 accordingly.

Further, if a measurements report message (such as, MeasReportAppLayer comprising the second session identity (such as, QoE reference ID (ID3-2) ) (or a second identity to identify the second session identity) and the measurements report (such as, Container of Report (Text) ) is transmitted 355-2 to the third device 230, the third device 230 determines 360-2 that the second session identity (or a second identity to identify the second session identity) is corresponding the second data collector 240-2, and forwards 365-2 the measurement report to the second data collector 240-2 accordingly.

In addition, if a measurements report message (such as, MeasReportAppLayer comprising the third session identity (such as, QoE reference ID (ID3-c) ) or the list of identities corresponding to the third session identity (such as, QoE reference ID (ID3-1 and ID3-2) , or identities corresponding to the first and second identities) and the measurements report (such as, Container of Report (Video) ) is transmitted 355-3 to the third device 230, the third device 230 determines 360-3 the third session identity is corresponding both the first data collector 240-1 and the second data collector 240-2, and forwards 365-3-1, 365-3-2 the measurement report to the first data collector 240-1 and the second data collector 240-2, respectively.

In this way, the measurement result (corresponding to the overlapped/common metric) is transmitted merely once and the resource in air interface may be saved. In addition, by optimizing QoE configuration files at the first device 210 (such as, the MS) , the additional enhancement for the third device 230 (such as, the network access device) is minimized. In particular, no additional enhancement is required for the second device 220 (such as, the terminal device) .

Example processes for the improved reporting procedure

Principle and implementations for the improved reporting procedure of the present disclosure will be described in detail below with reference to Fig. 4, which illustrates a signaling chart of an example process 400 of the QoE measurement according to some embodiments of the present disclosure.

For the purpose of discussion, the example process 400 will be described with reference to Fig. 2. The example process 400 may involve the first device 210, the second device 220, the third device 230, the first data collector 240-1 and the second data collector 240-2.

It is to be understood that processes (such as, obtaining the measurement requirement, generating the first message corresponding to the respective measurement configuration, transmitting the first message, creating the correspondence or association between the session identity and the data collector 240-2, transmitting the configuration message) illustrated in Fig. 4 are similar with those discussed with respect to Fig. 3. For brevity, the same or similar descriptions are omitted here.

In operation, the second device 220 receives 420 a fourth measurement configuration indicating at least one metric for a QoE at the second device 220. Further, the second device 220 receives 440 a fifth measurement configuration indicating at least one metric. In the specific example of Fig. 4, the fourth and fifth measurement configurations are configured for one same service type and correspond to different session identifies. In the following text, the session identity corresponds to the fourth configuration is referred to as the first session identity (be represented as “ID1” ) , while the session identity corresponds to the fifth configuration is referred to as the second session identity (be represented as “ID2” ) .

As one specific example, the fourth measurement configurations comprises metric for Speech and Video while the fourth measurement configurations comprises metric for Text and Video.

In some embodiments, the fourth and fifth measurement configurations may be comprised in different configuration messages (such as, RRCReconfiguration message) as an IE/parameter (such as, Container of QoE Config) transmitted from the third device 230.

Two specific examples of part of the configuration messages corresponding to the fourth and fifth measurement configurations are illustrated as below.

specific example of part of the configuration message corresponding to the fourth measurement configuration

RRCReconfiguration

- Service Type (DASH)

- QoE reference ID (ID1)

- Container of QoE Config (Video, Speech)

specific example of part of the configuration message corresponding to the fifth measurement configuration

RRCReconfiguration

- Service Type (DASH)

- QoE reference ID (ID1)

- Container of QoE Config (Video, Text)

Further, the fourth and fifth measurement configurations may originate from two measurement requirements from different data collectors 240. Specifically, as illustrated in Fig. 4, the first device 210 obtains 405 a first measurement requirement from the first data collector 240-1 and further obtains 425 a second measurement requirement from the second data collector 240-2. Then, the first device 210 generates two measurement configurations (i.e., fourth and fifth measurement configuration) based on the first and second measurement requirements, respectively.

Further, the first device 210 transmits 410 a first message for activating the fourth measurement configuration to the third device 230, where the first message indicates: the fourth measurement configuration, information of the first collector 240-1, and the first session identity. The third device 230 creates 415 a correspondence between the first session identity and the information of the first collector 240-1.

Similarly, the first device 210 transmits 430 a first message for activating the fifth measurement configuration to the third device 230, where the first message indicates: the fifth measurement configuration, information of the second collector 240-2, and the second session identity. The third device 230 creates 435 a correspondence between the second session identity and the information of the second collector 240-2.

After obtaining the fourth and fifth measurement configurations, the second device 220 (such as, the application layer of the second device 220) may determine whether the at least one metric comprised in the at least one metric indicated by the fourth measurement configuration is at least partially overlapped (including fully overlapped and partially overlapped) with the at least one metric indicated by the fifth measurement configuration. In other words, the second device 220 may determine whether there are common metric (s) indicates by both fourth and fifth measurement configurations.

In some example embodiments, if there is no overlapped/common metrics, the second device 220 generates the QoE reporting container according to fourth and fifth measurement configurations separately and QoE reporting container separately with the corresponding session identities. Specifically, the QoE reporting container corresponding to the fourth device is attached with first session identity and the QoE reporting container corresponding to the fifth device is attached with second session identity.

In some example embodiments, if the at least one metric comprised in the at least one metric indicated by the fourth measurement configuration is at least partially overlapped with the at least one metric indicated by the fifth measurement configuration, the third device 230 generates at least one measurement report based on the fourth and fifth measurement configuration. In particular, a measurement report of the at least one measurement report comprises measurement reports correspond to at least one overlapped/common metric.

In some example embodiments, as for the fourth and fifth measurement configurations, three measurement reports may be generated. Specifically, the third device 230 may generate 450 a first measurement report comprising measurement results corresponding to metrics comprised in the fourth measurement configuration while not comprised in fifth measurement configuration, a second measurement report comprising measurement results corresponding to metrics comprised in the fifth measurement configuration while not comprised in fourth configuration requirement, and a third measurement report comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.

Further, the overlap relationship of the metric may be various, which may allow some of the above first, second and third measurement reports to not be generated as discussed below. Specifically, in some scenarios, the first or second measurement report would not be generated. Some example embodiments for such scenario will be discussed below.

In some example embodiments, if the at least one metric indicated by the fourth measurement configuration is a subset of the at least one metric indicated by the fifth measurement configuration, the second device 220 performs a measurement based on the fifth measurement configuration, and further generates the second measurement report and the third measurement report.

Alternatively, in some example embodiments, if the at least one metric indicated by the fifth measurement configuration is a subset of the at least one metric indicated by the fourth measurement configuration, the second device 220 performs a measurement based on the fourth measurement configuration and further generates the first measurement report and the third measurement report.

Alternatively, in some example embodiments, if the at least one metric indicated by the fifth measurement configuration is identical with the at least one metric indicated by the fifth measurement configuration, the second device 220 performs a measurement based on either the fourth measurement configuration or the fifth measurement configuration, and further merely generates the third measurement report.

In some example embodiments, the generated measurement report may be transmitted via a container comprised in the measurement report message. That is, the measurement report is transparent with respect to the third device 230.

In some example embodiments, in addition to the measurement report, the measurement report message further comprises at least one identity, where the at least one session identity corresponds to the respective measurement report and is used for identifying at least one corresponding session identity. More specifically, the first measurement report corresponds to the first session identity, the second measurement report corresponds to the second session identity, while the third measurement report corresponds to both the first and second session identities.

Three specific examples of part of the measurement report message corresponding to the first, second and third measurement report are illustrated as below.

specific example of part of the measurement report message corresponding to the first measurement report

MeasReportAppLayer

- QoE reference ID (ID1)

- Container of Report (Speech)

specific example of part of the measurement report message corresponding to the second measurement report

MeasReportAppLayer

- QoE reference ID (ID2)

- Container of Report (Text)

specific example of part of the measurement report message corresponding to the third measurement report

MeasReportAppLayer

- QoE reference ID (ID1, ID2)

- Container of Report (Video)

Then the generated measurement report message (s) is transmitted to the third device 230. In the specific example of Fig. 4, the second device 220 transmits 455-3 the measurement report message corresponding to the third measurement report to the third device 230. Further, if the first measurement report is generated, the second device 220 will transmit 455-2 the measurement report message corresponding to the first measurement report to the third device 230, while if the second measurement report is generated, the first device 210 will transmit 455-3 the measurement report message corresponding to the third measurement report to the third device 230.

After receiving the measurement report message, the third device 230 may transmit the measurement report to the data collector (s) based on the correspondence between the created correspondence between the session identity and the information of the data collectors 240.

As illustrated in the Fig. 4, if a measurements report message (such as, MeasReportAppLayer comprising the first session identity (such as, QoE reference ID (ID 1) ) and the first measurements report (such as, Container of Report (Speech) ) is transmitted 455-1 to the third device 230, the third device 230 determines 460-1 that the first session identity is corresponding the first data collector 240-1, and forwards 465-1 the first measurement report to the first data collector 240-1 accordingly.

Further, if a measurements report message (such as, MeasReportAppLayer comprising the second session identity (such as, QoE reference ID (ID2) ) and the second measurements report (such as, Container of Report (Text) ) is transmitted 455-2 to the third device 230, the third device 230 determines 460-2 that the second session identity is corresponding the second data collector 240-2, and forwards 465-2 the second measurement report to the second data collector 240-2 accordingly.

In addition, if a measurements report message (such as, MeasReportAppLayer comprising the both of the first and second session identities (such as, QoE reference ID (ID1, ID2) ) and the third measurements report (such as, Container of Report (Video) ) is transmitted 455-3 to the third device 230, the third device 230 determines460-3 that the third measurements report is corresponding both the first data collector 240-1 and the second data collector 240-2, and forwards 465-3-1, 465-3-2 the third measurement report to first data collector 240-1 and the second data collector 240-2, respectively.

In this way, the measurement results (corresponding to the overlapped/common metric) are transmitted merely once and the resource in air interface may be saved.

In addition, by filtering QoE measurement data at the second device 220 (such as, the terminal device) , the additional enhancement for the third device 230 (such as, the network access device) is minimized. In particular, no additional enhancement is required for the first device 210 (such as, the MS) .

Fig. 5 shows a flowchart of an example method 500 implemented at a first device 210 in accordance with some example embodiments of the present disclosure. For the purpose of discussion, the method 500 will be described from the perspective of the first device 210 with respect to Fig. 2. It is to be understood that the method 500 may include additional blocks not shown and/or may omit some shown blocks, and the scope of the present disclosure is not limited in this regard.

At block 510, the first device 210 obtains a first measurement requirement for a QoE measurement at a second device 220 from a first data collector 240-1, the first measurement requirement comprising at least one metric.

At block 530, the first device 210 obtains a second measurement requirement for a QoE measurement at the second device 220 from a second data collector 240-2, the second measurement requirement comprising at least one metric.

At block 530, if the at least one metric comprised in the first measurement requirement is at least partially overlapped with the at least one metric comprised in the second measurement requirement, the first device 210 obtains generates at least one measurement configuration based on the first and second measurement requirements, a measurement configuration of the least one measurement configuration comprising at least one overlapped metric.

In some example embodiments, the first device 210 transmits at least one first message to a third device 230 serving the second device 220, each first message being used for activating a respective measurement configuration of the at least one measurement configuration and comprising: the respective measurement configuration, session identity assigned for a measurement session corresponding to the respective measurement configuration, and information of at least one data collector corresponding to the respective measurement configuration.

In some example embodiments, if the at least one metric comprised in the first measurement requirement is partially overlapped with the at least one metric comprised in the second measurement requirement, the first device 210 generates at least one measurement configuration by: generating the following: a first measurement configuration indicating metrics comprised in the first measurement requirement while not comprised in the second measurement requirement, a second measurement configuration indicating metrics comprised in the second measurement requirement while not comprised in the first measurement requirement, and a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.

In some example embodiments, if the at least one metric comprised in the first measurement requirement is a subset of the at least one metric comprised in the second measurement requirement, the first device 210 generates at least one measurement configuration by: generating the following: a second measurement configuration indicating metrics comprised in the second measurement requirement while not comprised in the first measurement requirement, and a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.

In some example embodiments, if the at least one metric comprised in the second measurement requirement is a subset of the at least one metric comprised in the first measurement requirement, the first device 210 generates at least one measurement configuration by: generating the following: a first measurement configuration indicating metrics comprised in the first measurement requirement while not comprised in second measurement requirement, and a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.

In some example embodiments, if the at least one metric comprised in the first measurement requirement is identical with the at least one metric comprised in the second measurement requirement, the first device 210 generates at least one measurement configuration by: generating a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.

In some example embodiments, the first measurement configuration is corresponding to the first data collector 240-1, the second measurement configuration is corresponding to the second data collector 240-2 and the third measurement configuration is corresponding to both the first data collector 240-1 and second data collector 240-1.

In some example embodiments, the first measurement requirement is to be activated or has been activated by the first deice, and the second measurement requirement is to be activated by the first deice.

In some example embodiments, if the first measurement requirement has been activated by the first deice, the first device 210 transmits a second message to deactivate the first measurement requirement.

In some example embodiments, the first device 210 is a management system device and the second device 220 is a terminal device.

A first apparatus capable of performing the method 500 may comprise means for performing the respective operations of the method 500. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module. The first apparatus may be implemented as or included in the first device 210.

In some example embodiments, the first apparatus comprises means for means for obtaining, at a first apparatus, a first measurement requirement for a QoE measurement at a second apparatus from a first data collector, the first measurement requirement comprising at least one metric; means for obtaining a second measurement requirement for a QoE measurement at the second apparatus from a second data collector, the second measurement requirement comprising at least one metric; and means for generating at least one measurement configuration based on the first and second measurement requirements if the at least one metric comprised in the first measurement requirement is at least partially overlapped with the at least one metric comprised in the second measurement requirement, a measurement configuration of the least one measurement configuration comprising at least one overlapped metric

In some example embodiments, the first apparatus further comprises: means for transmitting at least one first message to a third apparatus serving the second apparatus, each first message being used for activating a respective measurement configuration of the at least one measurement configuration and comprising: the respective measurement configuration, session identity assigned for a measurement session corresponding to the respective measurement configuration, and information of at least one data collector corresponding to the respective measurement configuration.

In some example embodiments, if the at least one metric comprised in the first measurement requirement is partially overlapped with the at least one metric comprised in the second measurement requirement, the means for generating the at least one measurement configuration comprises: means for generating the following: a first measurement configuration indicating metrics comprised in the first measurement requirement while not comprised in the second measurement requirement, a second measurement configuration indicating metrics comprised in the second measurement requirement while not comprised in the first measurement requirement, and a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.

In some example embodiments, if the at least one metric comprised in the first measurement requirement is a subset of the at least one metric comprised in the second measurement requirement, the means for generating the at least one measurement configuration comprises: generating the following: a second measurement configuration indicating metrics comprised in the second measurement requirement while not comprised in the first measurement requirement, and a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.

In some example embodiments, if the at least one metric comprised in the second measurement requirement is a subset of the at least one metric comprised in the first measurement requirement, the means for generating the at least one measurement configuration comprises: generating the following: a first measurement configuration indicating metrics comprised in the first measurement requirement while not comprised in second measurement requirement, and a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.

In some example embodiments, if the at least one metric comprised in the first measurement requirement is identical with the at least one metric comprised in the second measurement requirement, the means for generating the at least one measurement configuration comprises: generating a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.

In some example embodiments, the first measurement requirement is to be activated or has been activated by the first apparatus, and the second measurement requirement is to be activated by the first apparatus.

In some example embodiments, the first apparatus further comprises: means for transmitting a second message to deactivate the first measurement requirement.

In some example embodiments, the first apparatus is a management system apparatus and the second apparatus is a terminal apparatus.

Fig. 6 shows a flowchart of an example method 600 implemented at a third device 230 in accordance with some example embodiments of the present disclosure. For the purpose of discussion, the method 600 will be described from the perspective of the third device 230 with respect to Fig. 2. It is to be understood that the method 600 may include additional blocks not shown and/or may omit some shown blocks, and the scope of the present disclosure is not limited in this regard.

At block 610, the third deice 230 receives, from a first device 210, a first message for activating a measurement configuration for a QoE measurement at a second device 220, the first message indicating: the measurement configuration, session identity assigned for a measurement session corresponding to the measurement configuration, and information of a plurality of data collectors 240 corresponding to the measurement configuration,

At block 620, the third deice 230 creates a correspondence or an association between the session identity and the information of the plurality of data collectors 240.

At block 630, the third deice 230 transmits, based on the received first message, a configuration message to the second device 220.

In some example embodiments, the third device 230 receives, from the second device 220, a measurement report message comprising: a measurement report, and an identity to identify the measurement session, and transmits, based on the correspondence, the measurement report to the plurality of data collectors 240, respectively.

In some example embodiments, the first device 210 is a management system device, the second device 220 is a terminal device and the third device 230 an access network device.

A third apparatus capable of performing the method 600 may comprise means for performing the respective operations of the method 600. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module. The third apparatus may be implemented as or included in the third device 230.

In some example embodiments, the third apparatus comprises means for receiving, at a third apparatus and from a first apparatus, a first message for activating a measurement configuration for a QoE measurement at a second apparatus, the first message indicating: the measurement configuration, session identity assigned for a measurement session corresponding to the measurement configuration, and information of a plurality of data collectors corresponding to the measurement configuration; means for creating a correspondence between the session identity and the information of the plurality of data collectors; and means for transmitting, based on the received first message, a configuration message to the second apparatus.

In some example embodiments, the third apparatus further comprises: means for receiving, from the second apparatus, a measurement report message comprising: a measurement report, and an identity to identify the measurement session; and means for transmitting, based on the correspondence, the measurement report to the plurality of data collectors, respectively.

In some example embodiments, the first apparatus is a management system apparatus, the second apparatus is a terminal apparatus and the third apparatus an access network apparatus.

Fig. 7 shows a flowchart of an example method 700 implemented at a second device 220 in accordance with some example embodiments of the present disclosure. For the purpose of discussion, the method 700 will be described from the perspective of the second 220 with respect to Fig. 2. It is to be understood that the method 700 may include additional blocks not shown and/or may omit some shown blocks, and the scope of the present disclosure is not limited in this regard.

At block 710, the second device 220, receives, from a third device 230, a fourth measurement configuration for a QoE measurement at the second device 220, the fourth measurement requirement indicating at least one metric and corresponding to a first identity to identity a first session identity;

At block 720, the second device 220 receives, from the third device 230, a fifth measurement configuration for a QoE measurement at for the second device 220, the fifth measurement requirement indicating at least one metric and corresponding to a second identity to identity a second session identity.

At block 730, the second device 220 generates at least one measurement report based on the fourth and fifth measurement configuration if the at least one metric indicated by the fourth measurement configuration is at least partially overlapped with the at least one metric indicated by the fifth measurement configuration, wherein a measurement report of the at least one measurement report comprises measurement reports correspond to at least one overlapped metric.

In some example embodiments, the second device 220 transmits at least one measurement report message to the third device 230, each measurement report message corresponding to a respective measurement report of the at least one measurement report and comprising: the respective measurement report and at least one identity, where the at least one identity corresponds to the respective measurement report and is used for identifying at least one session identity.

In some example embodiments, if the at least one metric indicated by the fourth measurement configuration is partially overlapped with the at least one metric indicated by the fifth measurement configuration, the second device 220 generates at least one measurement report by: performing a measurement based on the fourth and fifth measurement configurations; and generating the following: a first measurement report, comprising measurement results corresponding to metrics comprised in the fourth measurement configuration while not comprised in fifth measurement configuration, a second measurement report, comprising measurement results corresponding to metrics comprised in the fifth measurement configuration while not comprised in fourth configuration requirement, and a third measurement report, comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.

In some example embodiments, if the at least one metric indicated by the fourth measurement configuration is a subset of the at least one metric indicated by the fifth measurement configuration, the second device 220 generates at least one measurement report by: performing a measurement based on the fifth measurement configuration; and generating the following: a second measurement report, comprising measurement results corresponding to metrics comprised in the fifth measurement configuration while not comprised in fourth measurement configuration, and a third measurement report, comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.

In some example embodiments, if the at least one metric indicated by the fifth measurement configuration is a subset of the at least one metric indicated by the fourth measurement configuration, the at least one memory and the computer program code are configured to, with the at least one processor, the second device 220 generates at least one measurement report by: performing a measurement based on the fourth measurement configuration; and generating the following: a first measurement report, comprising measurement results corresponding to metrics comprised in the fourth measurement configuration while not comprised in second measurement configuration, and a third measurement report, comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.

In some example embodiments, if the at least one metric indicated by the fifth measurement configuration is identical with the at least one metric indicated by the fifth measurement configuration, the second device 220 generates at least one measurement report by: performing a measurement based on either the fourth measurement configuration or the fifth measurement configuration; and generating a third measurement report comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.

In some example embodiments, the second device 220 transmits, to the third device 230, a measurement report message comprising: the third measurement report, the first identity to identify the first session identity, and the second identity to identify the second session identity.

In some example embodiments, the second device 220 is a terminal device and the third device 230 an access network device.

A second apparatus capable of performing the method 700 may comprise means for performing the respective operations of the method 700. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module. The second apparatus may be implemented as or included in the second device 220.

In some example embodiments, the second apparatus comprises means for means for receiving, at a second apparatus and from a third apparatus, a fourth measurement configuration for a QoE measurement at the second apparatus, the fourth measurement requirement indicating at least one metric and corresponding to a first identity to identity a first session identity; means for receiving, from the third apparatus, a fifth measurement configuration for a QoE measurement at for the second apparatus, the fifth measurement requirement indicating at least one metric and corresponding to a second identity to identity a second session identity; and means for generating at least one measurement report based on the fourth and fifth measurement configuration if the at least one metric indicated by the fourth measurement configuration is at least partially overlapped with the at least one metric indicated by the fifth measurement configuration, wherein a measurement report of the at least one measurement report comprises measurement reports correspond to at least one overlapped metric.

In some example embodiments, the second apparatus further comprises: means for transmitting at least one measurement report message to the third apparatus, each measurement report message corresponding to a respective measurement report of the at least one measurement report and comprising: the respective measurement report and at least one identity, where the at least one identity corresponds to the respective measurement report and is used for identifying at least one session identity.

In some example embodiments, if the at least one metric indicated by the fourth measurement configuration is partially overlapped with the at least one metric indicated by the fifth measurement configuration, the means for generating the at least one measurement report comprises: means for performing a measurement based on the fourth and fifth measurement configurations; and means for generating the following: a first measurement report, comprising measurement results corresponding to metrics comprised in the fourth measurement configuration while not comprised in fifth measurement configuration, a second measurement report, comprising measurement results corresponding to metrics comprised in the fifth measurement configuration while not comprised in fourth configuration requirement, and a third measurement report, comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.

In some example embodiments, if the at least one metric indicated by the fourth measurement configuration is a subset of the at least one metric indicated by the fifth measurement configuration, the means for generating the at least one measurement report comprises: means for performing a measurement based on the fifth measurement configuration; and means for generating the following: a second measurement report, comprising measurement results corresponding to metrics comprised in the fifth measurement configuration while not comprised in fourth measurement configuration, and a third measurement report, comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.

In some example embodiments, if the at least one metric indicated by the fifth measurement configuration is a subset of the at least one metric indicated by the fourth measurement configuration, the means for generating the at least one measurement report comprises: means for performing a measurement based on the fourth measurement configuration; and means for generating the following: a first measurement report, comprising measurement results corresponding to metrics comprised in the fourth measurement configuration while not comprised in second measurement configuration, and a third measurement report, comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.

In some example embodiments, if the at least one metric indicated by the fifth measurement configuration is identical with the at least one metric indicated by the fifth measurement configuration, the means for generating the at least one measurement report comprises: means for performing a measurement based on either the fourth measurement configuration or the fifth measurement configuration; and means for generating a third measurement report comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.

In some example embodiments, the second apparatus further comprises: means for transmitting, to the third apparatus, a measurement report message comprising: the third measurement report, the first identity to identify the first session identity, and the second identity to identify the second session identity.

In some example embodiments, the second apparatus is a terminal apparatus and the third apparatus is an access network apparatus.

Fig. 8 shows a flowchart of an example method 800 implemented at a third device 230 in accordance with some example embodiments of the present disclosure. For the purpose of discussion, the method 800 will be described from the perspective of the third device 230 with respect to Fig. 2. It is to be understood that the method 800 may include additional blocks not shown and/or may omit some shown blocks, and the scope of the present disclosure is not limited in this regard.

At block 810, the third device 230 receives, from a second device 220, a measurement report message of a QoE measurement at the second device 220, the measurement report message comprising: a measurement report, and a plurality of identities to identify session identities.

At block 820, the third device 230 transmits, based on a correspondence or an association between the plurality of identities to identify a plurality of session identities and a plurality of identities of respective data collectors 240, the measurement report to the respective data collectors 240.

In some example embodiments, the second device 220 is a terminal device and the third device 230 is an access network device.

A third apparatus capable of performing the method 800 may comprise means for performing the respective operations of the method 800. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module. The third apparatus may be implemented as or included in the third device 230.

In some example embodiments, the third apparatus comprises means for means for receiving, at a third apparatus and from a second apparatus, a measurement report message of a QoE measurement at the second apparatus, the measurement report message comprising: a measurement report, and a plurality of session identities; and means for transmitting, based on a correspondence or an association between the plurality of identities to identity a plurality of session identities and a plurality of identities of respective data collectors, the measurement report to the respective data collectors.

Fig. 9 is a simplified block diagram of a device 900 that is suitable for implementing embodiments of the present disclosure. The device 900 may be provided to implement a communication device, for example, the first device 210, the second device 220 and the third device 230 as shown in Fig. 2. As shown, the device 900 includes one or more processors 910, one or more memories 920 coupled to the processor 910, and one or more communication modules 940 (such as, transmitters and/or receivers) coupled to the processor 910.

The communication module 940 is for bidirectional communications. The communication module 940 has at least one antenna to facilitate communication. The communication interface may represent any interface that is necessary for communication with other network elements.

The processor 910 may be of any type suitable to the local technical network and may include one or more of the following: general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs) and processors based on multicore processor architecture, as non-limiting examples. The device 900 may have multiple processors, such as an application specific integrated circuit chip that is slaved in time to a clock which synchronizes the main processor.

The memory 920 may include one or more non-volatile memories and one or more volatile memories. Examples of the non-volatile memories include, but are not limited to, a Read Only Memory (ROM) 924, an electrically programmable read only memory (EPROM) , a flash memory, a hard disk, a compact disc (CD) , a digital video disk (DVD) , and other magnetic storage and/or optical storage. Examples of the volatile memories include, but are not limited to, a random access memory (RAM) 922 and other volatile memories that will not last in the power-down duration.

A computer program 930 includes computer executable instructions that are executed by the associated processor 910. The program 930 may be stored in the ROM 920. The processor 910 may perform any suitable actions and processing by loading the program 930 into the RAM 920.

The embodiments of the present disclosure may be implemented by means of the program 930 so that the device 900 may perform any process of the disclosure as discussed with reference to Figs. 3 to 8. The embodiments of the present disclosure may also be implemented by hardware or by a combination of software and hardware.

In some embodiments, the program 930 may be tangibly contained in a computer readable medium which may be included in the device 900 (such as in the memory 920) or other storage devices that are accessible by the device 900. The device 900 may load the program 930 from the computer readable medium to the RAM 922 for execution. The computer readable medium may include any types of tangible non-volatile storage, such as ROM, EPROM, a flash memory, a hard disk, CD, DVD, and the like. Fig. 10 shows an example of the computer readable medium 1000 in form of CD or DVD. The computer readable medium has the program 930 stored thereon.

Generally, various embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device. While various aspects of embodiments of the present disclosure are illustrated and described as block diagrams, flowcharts, or using some other pictorial representations, it is to be understood that the block, apparatus, system, technique or method described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer readable storage medium. The computer program product includes computer-executable instructions, such as those included in program modules, being executed in a device on a target real or virtual processor, to carry out the methods 500 to 800 as described above with reference to Figs. 5-8. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, or the like that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or split between program modules as desired in various embodiments. Machine-executable instructions for program modules may be executed within a local or distributed device. In a distributed device, program modules may be located in both local and remote storage media.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowcharts and/or block diagrams to be implemented. The program code may execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present disclosure, the computer program codes or related data may be carried by any suitable carrier to enable the device, apparatus or processor to perform various processes and operations as described above. Examples of the carrier include a signal, computer readable medium, and the like.

The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable medium may include but not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium would include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM) , a read-only memory (ROM) , an erasable programmable read-only memory (EPROM or Flash memory) , an optical fiber, a portable compact disc read-only memory (CD-ROM) , an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are contained in the above discussions, these should not be construed as limitations on the scope of the present disclosure, but rather as descriptions of features that may be specific to particular embodiments. Certain features that are described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable sub-combination.

Although the present disclosure has been described in languages specific to structural features and/or methodological acts, it is to be understood that the present disclosure defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

A first device comprising:

at least one processor; and

at least one memory including computer program code;

wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the first device to:

obtain a first measurement requirement for a quality of experience measurement at a second device from a first data collector, the first measurement requirement comprising at least one metric;

obtain a second measurement requirement for a quality of experience measurement at the second device from a second data collector, the second measurement requirement comprising at least one metric; and

if the at least one metric comprised in the first measurement requirement is at least partially overlapped with the at least one metric comprised in the second measurement requirement, generate at least one measurement configuration based on the first and second measurement requirements, a measurement configuration of the least one measurement configuration comprising at least one overlapped metric.
The first device of claim 1, wherein the first device is further caused to:

transmit at least one first message to a third device serving the second device, each first message being used for activating a respective measurement configuration of the at least one measurement configuration and comprising:

the respective measurement configuration,

session identity assigned for a measurement session corresponding to the respective measurement configuration, and

information of at least one data collector corresponding to the respective measurement configuration.
The first device of claim 1, wherein if the at least one metric comprised in the first measurement requirement is partially overlapped with the at least one metric comprised in the second measurement requirement, the first device is caused to generate at least one measurement configuration by:

generating the following:

a first measurement configuration indicating metrics comprised in the first measurement requirement while not comprised in the second measurement requirement,

a second measurement configuration indicating metrics comprised in the second measurement requirement while not comprised in the first measurement requirement, and

a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.
The first device of claim 1, wherein if the at least one metric comprised in the first measurement requirement is a subset of the at least one metric comprised in the second measurement requirement, the first device is caused to generate at least one measurement configuration by:

generating the following:

a second measurement configuration indicating metrics comprised in the second measurement requirement while not comprised in the first measurement requirement, and

a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.
The first device of claim 1, wherein if the at least one metric comprised in the second measurement requirement is a subset of the at least one metric comprised in the first measurement requirement, the first device is caused to generate at least one measurement configuration by:

generating the following:

a first measurement configuration indicating metrics comprised in the first measurement requirement while not comprised in second measurement requirement, and

a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.
The first device of claim 1, wherein if the at least one metric comprised in the first measurement requirement is identical with the at least one metric comprised in the second measurement requirement, the first device is caused to generate at least one measurement configuration by:

generating a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.
The first device of any of claims 3 to 6, wherein,

the first measurement configuration is corresponding to the first data collector;

the second measurement configuration is corresponding to the second data collector; and

the third measurement configuration is corresponding to both the first and second data collectors.
The first device of any of claims 1 to 7, wherein the first measurement requirement is to be activated or has been activated by the first device, and the second measurement requirement is to be activated by the first device
The first device of claim 8, wherein if the first measurement requirement has been activated by the first device, the first device is further caused to:

transmit a second message to deactivate the first measurement requirement.
The first device of any of claims 1 to 9, wherein the first device is a management system device and the second device is a terminal device.
A third device comprising:

at least one processor; and

at least one memory including computer program code;

wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the third device to:

receive, from a first device, a first message for activating a measurement configuration for a quality of experience measurement at a second device, the first message indicating:

the measurement configuration,

session identity assigned for a measurement session corresponding to the measurement configuration, and

information of a plurality of data collectors corresponding to the measurement configuration;

create a correspondence between an identity used for identifying the session identity and the information of the plurality of data collectors; and

transmit, based on the received first message, a configuration message to the second device.
The third device of claim 11, wherein the third device is further caused to:

receive, from the second device, a measurement report message comprising:

a measurement report, and

an identity to identify the measurement session; and

transmit, based on the correspondence, the measurement report to the plurality of data collectors, respectively.
The third device of claim 11 or 12, wherein the first device is a management system device, the second device is a terminal device and the third device is an access network device.
A second device comprising:

at least one processor; and

at least one memory including computer program code;

wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the second device to:

receive, from a third device, a fourth measurement configuration for a quality of experience measurement at the second device, the fourth measurement requirement indicating at least one metric and corresponding to a first identity to identity a first session identity;

receive, from the third device, a fifth measurement configuration for a quality of experience measurement at for the second device, the fifth measurement requirement indicating at least one metric and corresponding to a second identity to identity a second session identity; and

if the at least one metric indicated by the fourth measurement configuration is at least partially overlapped with the at least one metric indicated by the fifth measurement configuration, generate at least one measurement report based on the fourth and fifth measurement configuration, wherein a measurement report of the at least one measurement report comprises measurement reports correspond to at least one overlapped metric.
The second device of claim 14, wherein the second device is further caused to:

transmit at least one measurement report message to the third device, each measurement report message corresponding to a respective measurement report of the at least one measurement report and comprising:

the respective measurement report, and

at least one identity, corresponding to the respective measurement report and used for identifying at least one corresponding session identity.
The second device of claim 14, wherein if the at least one metric indicated by the fourth measurement configuration is partially overlapped with the at least one metric indicated by the fifth measurement configuration, the second device is caused to generate at least one measurement report by:

performing a measurement based on the fourth and fifth measurement configurations; and

generating the following:

a first measurement report, comprising measurement results corresponding to metrics comprised in the fourth measurement configuration while not comprised in fifth measurement configuration,

a second measurement report, comprising measurement results corresponding to metrics comprised in the fifth measurement configuration while not comprised in fourth configuration requirement, and

a third measurement report, comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.
The second device of claim 14, wherein if the at least one metric indicated by the fourth measurement configuration is a subset of the at least one metric indicated by the fifth measurement configuration, the second device is caused to generate at least one measurement report by:

performing a measurement based on the fifth measurement configuration; and

generating the following:

a second measurement report, comprising measurement results corresponding to metrics comprised in the fifth measurement configuration while not comprised in fourth measurement configuration, and

a third measurement report, comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.
The second device of claim 14, wherein if the at least one metric indicated by the fifth measurement configuration is a subset of the at least one metric indicated by the fourth measurement configuration, the second device is further caused to generate at least one measurement report by:

performing a measurement based on the fourth measurement configuration; and

generating the following:

a first measurement report, comprising measurement results corresponding to metrics comprised in the fourth measurement configuration while not comprised in second measurement configuration, and

a third measurement report, comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.
The second device of claim 14, wherein if the at least one metric indicated by the fifth measurement configuration is identical with the at least one metric indicated by the fifth measurement configuration, the second device is caused to generate at least one measurement report by:

performing a measurement based on either the fourth measurement configuration or the fifth measurement configuration; and

generating a third measurement report comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.
The second device of any of claims 16 to 19, wherein the second device is further caused to:

transmit, to the third device, a measurement report message comprising:

the third measurement report,

the first identity to identify the first session identity, and

the second identity to identify the second session identity.
The second device of any of claim 14 to 20, wherein the second device is a terminal device and the third device is an access network device.
A third device comprising:

at least one processor; and

at least one memory including computer program code;

wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the third device to:

receive, from a second device, a measurement report message of a quality of experience measurement at the second device, the measurement report message comprising:

a measurement report, and

a plurality of identities to identify session identities; and

transmit, based on a correspondence between the plurality of identities to identify a plurality of session identities and a plurality of identities of respective data collectors, the measurement report to the respective data collectors.
The third device claim 22, wherein the second device is a terminal device and the third device is an access network device.
A method comprising:

obtaining, at a first device, a first measurement requirement for a quality of experience measurement at a second device from a first data collector, the first measurement requirement comprising at least one metric;

obtaining a second measurement requirement for a quality of experience measurement at the second device from a second data collector, the second measurement requirement comprising at least one metric; and

if the at least one metric comprised in the first measurement requirement is at least partially overlapped with the at least one metric comprised in the second measurement requirement, generating at least one measurement configuration based on the first and second measurement requirements, a measurement configuration of the least one measurement configuration comprising at least one overlapped metric.
The method of claim 24, further comprising:

transmitting at least one first message to a third device serving the second device, each first message being used for activating a respective measurement configuration of the at least one measurement configuration and comprising:

the respective measurement configuration,

session identity assigned for a measurement session corresponding to the respective measurement configuration, and

information of at least one data collector corresponding to the respective measurement configuration.
The method of claim 24, wherein if the at least one metric comprised in the first measurement requirement is partially overlapped with the at least one metric comprised in the second measurement requirement, the generating the at least one measurement configuration comprises:

generating the following:

a first measurement configuration indicating metrics comprised in the first measurement requirement while not comprised in the second measurement requirement,

a second measurement configuration indicating metrics comprised in the second measurement requirement while not comprised in the first measurement requirement, and

a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.
The method of claim 24, wherein if the at least one metric comprised in the first measurement requirement is a subset of the at least one metric comprised in the second measurement requirement, the generating the at least one measurement configuration comprises:

generating the following:

a second measurement configuration indicating metrics comprised in the second measurement requirement while not comprised in the first measurement requirement, and

a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.
The method of claim 24, wherein if the at least one metric comprised in the second measurement requirement is a subset of the at least one metric comprised in the first measurement requirement, the generating the at least one measurement configuration comprises:

generating the following:

a first measurement configuration indicating metrics comprised in the first measurement requirement while not comprised in second measurement requirement, and

a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.
The method of claim 24, wherein if the at least one metric comprised in the first measurement requirement is identical with the at least one metric comprised in the second measurement requirement, the generating the at least one measurement configuration comprises:

generating a third measurement configuration indicating metrics comprised in both the first and second measurement requirements.
The method of any of claims 26 to 29, wherein,

the first measurement configuration is corresponding to the first data collector;

the second measurement configuration is corresponding to the second data collector; and

the third measurement configuration is corresponding to both the first and second data collectors.
The method of any of claims 24 to 30, wherein the first measurement requirement is to be activated or has been activated by the first device, and the second measurement requirement is to be activated by the first device
The method of claim 31, wherein if the first measurement requirement has been activated by the first device, the method further comprises:

transmitting a second message to deactivate the first measurement requirement.
The method of any of claims 24 to 32, wherein the first device is a management system device and the second device is a terminal device.
A method comprising:

receiving, at a third device and from a first device, a first message for activating a measurement configuration for a quality of experience measurement at a second device, the first message indicating:

the measurement configuration,

session identity assigned for a measurement session corresponding to the measurement configuration, and

information of a plurality of data collectors corresponding to the measurement configuration;

creating a correspondence between an identity used for identifying the session identity and the information of the plurality of data collectors; and

transmitting, based on the received first message, a configuration message to the second device.
The method of claim 34, further comprising:

receiving, from the second device, a measurement report message comprising:

a measurement report, and

an identity to identify the measurement session; and

transmitting, based on the correspondence, the measurement report to the plurality of data collectors, respectively.
The method of claim 34 or 35, wherein the first device is a management system device, the second device is a terminal device and the third device is an access network device.
A method comprising:

receiving, at a second device and from a third device, a fourth measurement configuration for a quality of experience measurement at the second device, the fourth measurement requirement indicating at least one metric and corresponding to a first identity to identify a first session identity;

receiving, from the third device, a fifth measurement configuration for a quality of experience measurement at for the second device, the fifth measurement requirement indicating at least one metric and corresponding to a second identity to identify a second session identity; and

if the at least one metric indicated by the fourth measurement configuration is at least partially overlapped with the at least one metric indicated by the fifth measurement configuration, generating at least one measurement report based on the fourth and fifth measurement configuration, wherein a measurement report of the at least one measurement report comprises measurement reports correspond to at least one overlapped metric.
The method of claim 37, further comprising:

transmitting at least one measurement report message to the third device, each measurement report message corresponding to a respective measurement report of the at least one measurement report and comprising:

the respective measurement report, and

at least one identity, corresponding to the respective measurement report and used for identifying at least one corresponding session identity.
The method of claim 37, wherein if the at least one metric indicated by the fourth measurement configuration is partially overlapped with the at least one metric indicated by the fifth measurement configuration, the generating the at least one measurement report comprises:

performing a measurement based on the fourth and fifth measurement configurations; and

generating the following:

a first measurement report, comprising measurement results corresponding to metrics comprised in the fourth measurement configuration while not comprised in fifth measurement configuration,

a second measurement report, comprising measurement results corresponding to metrics comprised in the fifth measurement configuration while not comprised in fourth configuration requirement, and

a third measurement report, comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.
The method of claim 37, wherein if the at least one metric indicated by the fourth measurement configuration is a subset of the at least one metric indicated by the fifth measurement configuration, the generating the at least one measurement report comprises:

performing a measurement based on the fifth measurement configuration; and

generating the following:

a second measurement report, comprising measurement results corresponding to metrics comprised in the fifth measurement configuration while not comprised in fourth measurement configuration, and

a third measurement report, comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.
The method of claim 37, wherein if the at least one metric indicated by the fifth measurement configuration is a subset of the at least one metric indicated by the fourth measurement configuration, the generating the at least one measurement report comprises:

performing a measurement based on the fourth measurement configuration; and

generating the following:

a first measurement report, comprising measurement results corresponding to metrics comprised in the fourth measurement configuration while not comprised in second measurement configuration, and

a third measurement report, comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.
The method of claim 37, wherein if the at least one metric indicated by the fifth measurement configuration is identical with the at least one metric indicated by the fifth measurement configuration, the generating the at least one measurement report comprises:

performing a measurement based on either the fourth measurement configuration or the fifth measurement configuration; and

generating a third measurement report comprising measurement results corresponding to metrics comprised in both the fourth and fifth measurement configurations.
The method of any of claims 39 to 42, further comprises:

transmitting, to the third device, a measurement report message comprising:

the third measurement report,

the first identity to identify the first session identity, and

the second identity to identify the second session identity.
The method of any of claim 37 to 43, wherein the second device is a terminal device and the third device is an access network device.
A method comprising:

receiving, at a third device and from a second device, a measurement report message of a quality of experience measurement at the second device, the measurement report message comprising:

a measurement report, and

a plurality of identities to identify session identities; and

transmitting, based on a correspondence between the plurality of identities to identity a plurality of session identities and a plurality of identities of respective data collectors, the measurement report to the respective data collectors.
The method claim 45, wherein the second device is a terminal device and the third device is an access network device.
A first apparatus comprising:

means for obtaining, at a first apparatus, a first measurement requirement for a quality of experience measurement at a second apparatus from a first data collector, the first measurement requirement comprising at least one metric;

means for obtaining a second measurement requirement for a quality of experience measurement at the second apparatus from a second data collector, the second measurement requirement comprising at least one metric; and

means for generating at least one measurement configuration based on the first and second measurement requirements if the at least one metric comprised in the first measurement requirement is at least partially overlapped with the at least one metric comprised in the second measurement requirement, a measurement configuration of the least one measurement configuration comprising at least one overlapped metric.
A third apparatus comprising:

means for receiving, at a third apparatus and from a first apparatus, a first message for activating a measurement configuration for a quality of experience measurement at a second apparatus, the first message indicating:

the measurement configuration,

session identity assigned for a measurement session corresponding to the measurement configuration, and

information of a plurality of data collectors corresponding to the measurement configuration;

means for creating a correspondence between an identity used for identifying the session identity and the information of the plurality of data collectors; and

means for transmitting, based on the received first message, a configuration message to the second apparatus.
A second apparatus comprising:

means for receiving, at a second apparatus and from a third apparatus, a fourth measurement configuration for a quality of experience measurement at the second apparatus, the fourth measurement requirement indicating at least one metric and corresponding to a first identity to identity a first session identity;

means for receiving, from the third apparatus, a fifth measurement configuration for a quality of experience measurement at for the second apparatus, the fifth measurement requirement indicating at least one metric and corresponding to a second identity to identity a second session identity; and

means for generating at least one measurement report based on the fourth and fifth measurement configuration if the at least one metric indicated by the fourth measurement configuration is at least partially overlapped with the at least one metric indicated by the fifth measurement configuration, wherein a measurement report of the at least one measurement report comprises measurement reports correspond to at least one overlapped metric.
A third apparatus comprising:

means for receiving, at a third apparatus and from a second apparatus, a measurement report message of a quality of experience measurement at the second apparatus, the measurement report message comprising:

a measurement report, and

a plurality of identities to identify session identities; and

means for transmitting, based on a correspondence between the plurality of identities to identify a plurality of session identities and a plurality of identities of respective data collectors, the measurement report to the respective data collectors.
A computer readable medium comprising a computer program thereon, the computer program when executed, causing an apparatus to perform the method of any of claims 1-10.
A computer readable medium comprising a computer program thereon, the computer program when executed, causing an apparatus to perform the method of any of claims 11-13.
A computer readable medium comprising a computer program thereon, the computer program when executed, causing an apparatus to perform the method of any of claims 14-21.
A computer readable medium comprising a computer program thereon, the computer program when executed, causing an apparatus to perform the method of claim 22 or 23.