WO2015088405A1

WO2015088405A1 - Method and apparatus for improving quality of experience of a real-time communication service

Info

Publication number: WO2015088405A1
Application number: PCT/SE2013/051484
Authority: WO
Inventors: Mats Folkesson; Gunilla BERNDTSSON; Valentin Kulyk
Original assignee: Telefonaktiebolaget L M Ericsson (Publ)
Priority date: 2013-12-11
Filing date: 2013-12-11
Publication date: 2015-06-18

Abstract

The present invention relates to a method for improving quality of experience of a real-time communication service. The method is performed in a client configured to receive media from a communication system. The method comprises receiving (210) via a user interface of the client, a request to improve quality of experience through a trade-off between media quality and delay of received media. The method also comprises initiating (220) an action for adjusting the trade-off between media quality and delay of received media in response to the received request. The present invention also relates to the client.

Description

METHOD AND APPARATUS FOR IMPROVING QUALITY OF EXPERIENCE OF A REAL-TIME COMMUNICATION SERVICE

TECHNICAL FIELD

The disclosure relates to communication of real-time media, and more specifically to a client and a method performed in the client for improving quality of experience of a real-time communication service.

BACKGROUND

Figure 1 illustrates a real-time media communication system 100 comprising a server 120 providing a real-time media service to a receiving client 1 10 over a transport network 130. The transport network 130 of the communication system 100 may be a private network or a public network such as the Internet. The media service may e.g. be a video conference service.

There are several methods and applications described or implemented in different types of communication systems for decreasing the impact of variations in network performance. The methods may relate both to how to estimate or measure media quality, and how to improve the media quality. Furthermore, it is well-known that the end-to-end delay between the media sending side and the receiving client may have an impact on the Quality of Experience (QoE), especially in real-time communication services. QoE is a subjective measure of an end user's entire service experience. In short, QoE provides an assessment of human expectations, feelings, perceptions, cognition and satisfaction with respect to a particular service or application. It is related to but differs from Quality of Service (QoS), which attempts to objectively measure the service. QoE is not just QoS, but does also take into consideration every factor that contributes to an overall user value, such as suitableness, flexibility, cost, personalization, and choice. Apart from being user dependent, QoE will invariably be influenced by, e.g., the user's client hardware (high or low definition TV), environment (in the car or at home), expectations (wireless or not), the nature of the media content and its importance (a simple yes/no message or an orchestral concert).

Most services take the impact on the transmission delay into account, and try to keep the end-to-end delay as low as possible or at least low enough to not cause any severe impact on the QoE. However, in many real-time communication services the impact of e.g. packet loss affecting media quality on the end-user's QoE may be bigger than the impact of a delay of the media. To focus the design of a service on a low delay may give limited possibilities to avoid disturbances due to e.g. packet loss. This may be the case for services used for e.g. live streaming and low-interactive meetings with one active and many inactive parties, such as services used for education and entertainment. Such services may be optimized to have a short delay.

SUMMARY

Services optimized to have a short delay are not always the best from a QoE perspective, as a longer delay may be preferable for the receiving end-user in some situations. It is therefore an object to address at least the problem outlined above, and to provide a solution making it possible to improve the QoE of a realtime communication service. This object and others are achieved by the method and the client according to the independent claims, and by the embodiments according to the dependent claims. In accordance with a first aspect, a method for improving QoE of a real-time communication service is provided. The method is performed in a client configured to receive media from a communication system. The method comprises receiving via a user interface of the client, a request to improve QoE through a trade-off between media quality and delay of received media. The method also comprises initiating an action for adjusting the trade-off between media quality and delay of received media in response to the received request.

In accordance with a second aspect, a client is provided. The client is configured to improve QoE of a real-time communication service. The client comprises a user interface, a communication unit configured to receive media in real time from a communication system, a processor, and a memory containing instructions executable by said processor, whereby the client is operative to receive via the user interface, a request to improve QoE through a trade-off between media quality and delay of received media. The client is also operative to initiate an action for adjusting the trade-off between media quality and delay of received media in response to the received request.

An advantage of embodiments is that since different end-users have different preferences of the trade-off between video quality and delay in different situations, end-users are allowed to provide their view on whether to increase media quality or reduce delay to improve the QoE. This could be especially useful when systems designed for delay critical services are used in situations which may not be delay- critical, as in live-streaming and low-interactive meetings with one or a few active and many inactive parties.

Other objects, advantages and features of embodiments will be explained in the following detailed description when considered in conjunction with the accompanying drawings and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 is a schematic illustration of a client and a communication system.

Figures 2a-b are flowcharts illustrating the method in the client according to embodiments. Figures 3a-b are block diagrams schematically illustrating the client according to embodiments.

Figure 4 is a block diagram schematically illustrating the user interface of the client according to embodiments.

Figure 5 is a block diagram schematically illustrating a computer program product according to embodiments.

DETAILED DESCRIPTION

In the following, different aspects will be described in more detail with references to certain embodiments and to accompanying drawings. For purposes of explanation and not limitation, specific details are set forth, such as particular scenarios and techniques, in order to provide a thorough understanding of the different embodiments. However, other embodiments that depart from these specific details may also exist.

Embodiments are described in a non-limiting general context in relation to an example scenario in a real-time communication system, where the client receives video in real-time from a server in the communication system. However, it should be noted that the embodiments may be applied to any type of media, and to all types of live-streaming and other end-user services deployed over a transport network. The client hardware may be a mobile phone such as a smartphone, a tablet, a personal computer laptop or desktop, a TV set or any device with a network connection. The client software, i.e. the communication application, may either be natively implemented in or downloadable onto the client hardware. The problem of a lowered QoE for the end-user due to that a communication system most often is designed to provide a low delay, is addressed by a solution where the end-user is provided a possibility to decide whether good video quality or low delay is to be prioritized for the video service. The end-user may e.g. allow an increased end-to-end delay with the intention to increase the video quality, if improved video quality is preferred at a cost of a longer end-to-end delay to enhance the QoE. Another example is when the end-user prefers a lower video quality, e.g. a still picture or a lower resolution of the video, to get a lower end-to- end delay, as this may improve the QoE for the end-user in the situation he is currently in. The end-user may thus control the trade-off between video quality and delay by entering a choice between higher video quality and shorter delay in a user interface of the client. The choice results in a request to the client to improve QoE through a trade-off between video quality and delay in accordance with the choice made by the end-user in the user interface.

The video quality in real-time services depends to a large extent on the performance of the transmission network. Variations in network performance might cause variations of the video bit-stream transmission performance, and of the video quality in the receiving client. By allowing extra time for processing methods e.g. in non-critical situations from a delay perspective, the video quality in the receiving client might be preserved or improved. On the other hand, if a short end- to-end delay is preferred, this could be achieved by letting the user allow a lower video quality, or even no video at all.

In embodiments of the invention the end-user can allow the real-time service or application to use some extra time to handle variations or degradations of the quality due to variations in the network performance. This extra time may be used e.g. for retransmission of a video bit stream, for change of processing of the video bit stream to better handle varying network performance, or for any other precaution that may be suitable to improve or at least to keep the video quality e.g. in terms of video resolution. This extra time may cause the end-to-end delay to increase, which may however be acceptable to the end-user in some situations. If an increased delay is acceptable or not, may in many situations only be decided by the end-user of the service as it is related to an end-user perception of the service. By letting the end-user decide about the trade off between delay and video quality, the video quality could be increased in situations where this is preferred to the cost of a longer end-to-end delay from a QoE perspective.

Figure 2a is a flowchart illustrating an embodiment of a method for improving QoE of a real-time communication service. The method is performed in a client 1 10 configured to receive media from a communication system 100. In one example scenario the received media is video. The method comprises:

- 210: Receiving via a user interface of the client, a request to improve QoE through a trade-off between media quality and delay of received media. Media quality may e.g. refer to the resolution of the video. Reduced media quality may thus e.g. be a reduced resolution of a picture or a video. Another example of media quality may also be the number of pictures per time unit, where a reduced media quality may e.g. be a still picture instead of a video with moving pictures. In one embodiment, the request to improve QoE may comprise a request to increase media quality at the cost of increased delay of received media. In an alternative embodiment, the request to improve QoE may comprise a request to reduce delay of received media at the cost of reduced media quality. The request to improve QoE may be received before a session for receiving media in real time is set up. A part of the user interface in the client may make it possible for the end-user to enter a preference of higher quality or of reduced delay before any session has been started. This preference would thus be valid for any session that is subsequently set up. Alternatively, the request may be received during a session, i.e. after the session has been set up. The user interface would in this case make it possible for the end-user to choose between better quality and shorter delay during the session, the choice thus being valid for the current session only. In the latter case, i.e. when the request is received during the session, the end-user may also iterate the request, e.g. requesting that the quality is improved more than once at the cost of an even further extended delay. The end-user may also regret a first request to improve media quality and may therefore enter a new request to decrease media quality to get a shorter delay.

- 220: Initiating an action for adjusting the trade-off between media quality and delay of received media in response to the received request. The action for adjusting the trade-off between media quality and delay of received media may be initiated for a limited time period. In one exemplary embodiment, an action for reducing the delay is initiated only temporarily during a time period. When the time period has lapsed, the action may be reversed, such that the delay may be increased again. Alternatively, the action for adjusting the trade-off may be initiated and may then be ongoing e.g. until a new request is received via the user interface.

Figure 2b is a flowchart illustrating another embodiment of the method performed in the client. The method comprises:

- 200: Receiving information from the communication system 100 indicating a capability to improve QoE through a trade-off between media quality and delay of received media. - 205: Displaying the capability in the user interface. It may not always be possible to improve the media quality, so the end-user needs to be informed about what the possibilities are, i.e. what capability there is.

- 210: Receiving via the user interface of the client, a request to improve QoE through a trade-off between media quality and delay of received media. As already described above, the request to improve QoE may comprise a request to increase media quality at the cost of increased delay of received media, or a request to reduce delay of received media at the cost of reduced media quality. The other embodiments described above with reference to Figure 2a are also possible.

220: Initiating the action for adjusting the trade-off between media quality and delay, further comprising adjusting a size of a buffer in the client used at reception of media in 221 . The size of the buffer may be adjusted for a limited time period or until further notice. The buffer may e.g. be a jitter buffer where the media may be processed, e.g. delayed, dropped, or repeated. When the jitter buffer size at the receiver client is extended, the time for receiving delayed media packets is increased. Therefore, more packets may arrive in time for being decoded and displayed in the client user interface. This thus improves the media quality at the cost of an increased delay. Such a solution may be useful when the jitter on the transport channel is high. An advantage of this embodiment is that the action may be initiated in the client itself only, and therefore no signaling to any other node of the communication system is required. In analogy, a reduction of the size of the jitter buffer results in a reduced delay at the cost of reduced quality.

- 230: Displaying information in the user interface indicating whether an action to increase media quality or an action to reduce delay of the received media has been initiated.

In one embodiment, the step 220 of initiating the action for adjusting the trade-off between media quality and delay may comprise sending a request to perform the action to another node of the communication system, instead of or in addition to adjusting the size of the buffer in the client. Although some extra signaling is required to other nodes of the system, this embodiment opens up for further possibilities to affect media quality and delay. The request to perform the action may comprise at least one of the following:

• A request to initiate or cancel use of retransmissions. Re-transmission of lost media packets which also requires a longer jitter buffer may also be used to improve quality at the cost of a higher delay. Also this may be useful if the jitter on the transport channel is high. In analogy, cancelling re- transmissions results in a reduced delay at the cost of reduced quality.

• A request to adjust a time for retransmissions. If re-transmissions are already in use, the time for re-transmission may instead be increased to obtain an improved quality at a cost of a higher delay. In analogy, a decreased time for re-transmission results in a reduced delay at the cost of reduced quality.

• A request to initiate or cancel use of Forward Error Correction (FEC).

FEC may be used to improve robustness against transmission errors. However, FEC requires extra processing time and thus increases the delay. In analogy, cancelling the use of FEC results in a reduced delay at the cost of reduced quality.

• A request to reconfigure forward error correction parameters affecting media quality. If FEC is already in use, it may be used in a more capacity efficient way with longer FEC intervals. Longer FEC intervals require a higher delay but allow a more bit rate efficient protection. In analogy, shortening the FEC intervals results in a reduced delay at the cost of reduced media quality.

• A request to change processing of the media affecting media quality. A video encoder may e.g. use some extra time for encoding the video with higher quality. This would thus allow for better video quality at a cost of higher delay. One example is the possibility to use out-of-order processing. With out-of-order processing, several frames are encoded before the final bit stream is sent. By encoding several frames at the same time instead of one frame at a time, a more efficient coding may be used. The out-of-order processing delays the encoding, but the required bitrate to maintain the same quality decreases. If the bit rate is kept the same, the quality may thus be improved. In analogy, a cancelling of out-of-order processing would result in a reduced delay at the cost of reduced media quality. In one special embodiment for video, the delay may be shortened if no video is sent at all, or if just a still picture is sent now and then. End-to-end delay is in this case decreased at the cost of a lower media quality.

Figure 4 schematically illustrates an example of the user interface 1 1 1 of the client 1 10. The input to the client and service regarding the end-user's preferences for the trade-off between media quality and delay of received media can be done via any kind of user interface, such as a soft switch or a slider 10 on a touch screen device 1 10 as in Figure 4. Alternatively, any of the "hard buttons" on a client keyboard may be used. The user interface illustrated in Figure 4 also illustrates one example of how to indicate whether an action to increase media quality or an action to reduce delay of the received media has been initiated, through an indicator, 20, 30, that is highlighted when either action has been initiated. This corresponds to the displaying described in step 230 above. In Figure 4, the indicator 30 indicating that an action to reduce the delay has been initiated is highlighted. Similarly, the indicator 40 indicates that the client has the capability to improve QoE, as described in step 205 above. Alternatively, there may be different indicators for the capability to improve media quality and for the capability to decrease delay. Other information that may be possible to display in the user interface is e.g. the actual current end-to-end delay if such information is available in the client. The user interface 1 1 1 typically also comprises a display 50 for displaying the real-time media of the service, such as a video display. An embodiment of a client 1 10 configured to improve QoE of a real-time communication service is schematically illustrated in the block diagram in Figure 3a. The client 1 10 comprises a user interface 1 1 1 , and a communication unit 1 12 configured to receive media in real time from a communication system 100. The client 1 10 also comprises a processor 1 13, and a memory 1 14 containing instructions executable by the processor 1 13, whereby the client 1 10 is operative to receive via the user interface 1 1 1 , a request to improve QoE through a trade-off between media quality and delay of received media. The client is also operative to initiate an action for adjusting the trade-off between media quality and delay of received media in response to the received request.

It is to be noted that the term processor used herein includes any hardware capable of executing instructions and/or program codes, e.g., a microprocessor such as a Central Processing Unit (CPU), a digital signal processor (DSP), or any other general-purpose or application specific processors. Furthermore, the term memory used herein includes any storage medium capable of storing instructions and/or program codes, e.g., a magnetic storage medium, an optical storage medium, a semiconductor storage medium and any other volatile or non-volatile storage mediums.

In embodiments, the client 1 10 further comprises a buffer 1 15 configured to be used at reception of media. The client 1 10 may be operative to initiate the action for adjusting the trade-off between media quality and delay by adjusting a size of the buffer 1 15. The client 1 10 may be operative to initiate the action for adjusting the trade-off between media quality and delay by sending a request to perform the action via the communication unit 1 12 to another node of the communication system 100. The request to perform the action may comprise at least one of the following: a request to initiate or cancel use of retransmissions; a request to adjust a time for retransmissions; a request to initiate or cancel use of forward error correction; a request to reconfigure forward error correction parameters affecting media quality; and a request to change processing of the media affecting media quality.

In embodiments of the invention, the request to improve QoE may comprise a request to increase media quality at the cost of increased delay of received media. Alternatively, the request to improve QoE may comprise a request to reduce delay of received media at the cost of reduced media quality.

In one embodiment, the client may be operative to receive information from the communication system 100 via the communication unit 1 12, the information indicating a capability to improve QoE through a trade-off between media quality and delay of received media. The client may also be operative to display the capability in the user interface 1 1 1 . The client may be operative to receive the request to improve QoE before a session for receiving media in real time is set up, or during the session. Furthermore, the client may be operative to initiate the action for adjusting the trade-off between media quality and delay of received media for a limited time period.

In embodiments, the client may be operative to display information in the user interface 1 1 1 indicating whether an action to increase media quality or an action to reduce delay of received media has been initiated.

An alternative way to describe the embodiment in Figure 3a is illustrated in Figure 3b. The client 1 10 comprises means 310 adapted to receive via a user interface of the client, a request to improve QoE through a trade-off between media quality and delay of received media. The client also comprises means 320 adapted to initiate an action for adjusting the trade-off between media quality and delay of received media in response to the received request. In one embodiment, the means 320 adapted to initiate the action for adjusting the trade-off between media quality and delay may comprise means for adjusting a size of a buffer in the client used at reception of media. The means 310 and 320 described above are functional units which may be implemented in hardware, software, firmware or any combination thereof. In one embodiment, the means are implemented as a computer program running on a processor.

In accordance with a further aspect, a computer program product 500 comprising a computer readable medium 510 and a computer program 520 is provided. The computer program product 500 is illustrated in Figure 5. The computer program 520 comprises computer readable code which when run on the client 1 10 causes the client 1 10 to perform the method as described above with reference to Figures 2a-b.

The above mentioned and described embodiments are only given as examples and should not be limiting. Other solutions, uses, objectives, and functions within the scope of the accompanying patent claims may be possible.

Claims

1 . A method for improving quality of experience of a real-time communication service, the method being performed in a client (1 10) configured to receive media from a communication system (100), the method comprising:

- receiving (210) via a user interface of the client, a request to improve quality of experience through a trade-off between media quality and delay of received media, and

- initiating (220) an action for adjusting the trade-off between media quality and delay of received media in response to the received request.

2. The method according to claim 1 , wherein initiating (220) the action for adjusting the trade-off between media quality and delay comprises adjusting (221 ) a size of a buffer in the client used at reception of media.

3. The method according to any of claims 1 -2, wherein initiating (220) the action for adjusting the trade-off between media quality and delay comprises sending a request to perform the action to another node of the communication system.

4. The method according to claim 3, wherein the request to perform the action comprises at least one of the following:

- a request to initiate or cancel use of retransmissions;

- a request to adjust a time for retransmissions;

- a request to initiate or cancel use of forward error correction;

- a request to reconfigure forward error correction parameters affecting media quality;

- a request to change processing of the media affecting media quality.

5. The method according to any of the preceding claims, wherein the request to improve quality of experience comprises a request to increase media quality at the cost of increased delay of received media.

6. The method according to any of claims 1 -4, wherein the request to improve quality of experience comprises a request to reduce delay of received media at the cost of reduced media quality.

5 7. The method according to any of the preceding claims, the method further comprising:

- receiving (200) information from the communication system indicating a capability to improve quality of experience through a trade-off between media quality and delay of received media, and

10 - displaying (205) the capability in the user interface.

8. The method according to any of the preceding claims, wherein the request to improve quality of experience is received before a session for receiving media in real time is set up, or during the session.

15

9. The method according to any of the preceding claims, wherein the action for adjusting the trade-off between media quality and delay of received media is initiated for a limited time period.

10. The method according to any of the preceding claims, further comprising:

- displaying (230) information in the user interface indicating whether an action to increase media quality or an action to reduce delay of the received media has been initiated.

25 1 1 . A client (1 10) configured to improve quality of experience of a real-time communication service, the client (1 10) comprising a user interface (1 1 1 ), a communication unit (1 12) configured to receive media in real time from a communication system (100), a processor (1 13), and a memory (1 14) containing instructions executable by said processor (1 13), whereby the client

30 (1 10) is operative to: - receive via the user interface (1 1 1 ), a request to improve quality of experience through a trade-off between media quality and delay of received media, and

- initiate an action for adjusting the trade-off between media quality and delay j of received media in response to the received request.

12. The client (1 10) according to claim 1 1 , further comprising a buffer (1 15) configured to be used at reception of media, wherein the client (1 10) is operative to initiate the action for adjusting the trade-off between media quality and delay by adjusting a size of the buffer (1 15).

13. The client (1 10) according to any of claims 1 1 -12, wherein the client is operative to initiate the action for adjusting the trade-off between media quality and delay by sending a request to perform the action via the communication unit (1 12) to another node of the communication system (100).

14. The client (1 10) according to claim 13, wherein the request to perform the action comprises at least one of the following:

- a request to initiate or cancel use of retransmissions;

- a request to adjust a time for retransmissions;

- a request to initiate or cancel use of forward error correction;

- a request to change processing of the media affecting media quality.

15. The client (1 10) according to any of claims 1 1 -14, wherein the request to improve quality of experience comprises a request to increase media quality at the cost of increased delay of received media.

16. The client (1 10) according to any of claims 1 1 -14, wherein the request to improve quality of experience comprises a request to reduce delay of received media at the cost of reduced media quality.

17. The client (1 10) according to any of claims 1 1 -16, wherein the client is operative to:

- receive information from the communication system (100) via the 5 communication unit (1 12), the information indicating a capability to improve quality of experience through a trade-off between media quality and delay of received media, and

- display the capability in the user interface (1 1 1 ).

10 18. The client (1 10) according to any of claims 1 1 -17, wherein the client is operative to receive the request to improve quality of experience before a session for receiving media in real time is set up, or during the session.

19. The client (1 10) according to any of claims 1 1 -18, wherein the client is 15 operative to initiate the action for adjusting the trade-off between media quality and delay of received media for a limited time period.

20. The client (1 10) according to any of claims 1 1 -19, wherein the client is operative to:

20 - display information in the user interface (1 1 1 ) indicating whether an action to increase media quality or an action to reduce delay of received media has been initiated.