WO2018041680A1

WO2018041680A1 - Device and method for determining the quality of a media data stream

Info

Publication number: WO2018041680A1
Application number: PCT/EP2017/071170
Authority: WO
Inventors: Shahid Mahmood Satti; Matthias Obermann; Christian Schmidmer; Michael Keyhl
Original assignee: Opticom Dipl.-Ing. Michael Keyhl Gmbh
Priority date: 2016-08-30
Filing date: 2017-08-22
Publication date: 2018-03-08
Also published as: EP3507982A1; DE102016216346A1; DE102016216346B4

Abstract

The invention relates to a device and a method for determining the quality of a media data stream (18) during a measurement period. The media data stream (18) transmits media content of varying quality. The device comprises a media data stream analysis module (38) which is configured to determine a plurality of first quality indicators (1a-1j) from the media data stream (18); a short-term aggregation module (40) which is configured to determine a short-term quality value (q_k) using the plurality of first quality indicators (1a-1j) for a short-term period (k) that is shorter than the measurement period, wherein the short-term aggregation module (40) is configured to supply at least one respective short-term quality value (q_k) for a plurality of short-term periods (k); and a long-term aggregation module (42) which is configured to determine a plurality of statistical values (2a-2c) using a plurality of short-term quality values (q_k) for the measurement period and to supply a quality measurement (q) of the quality of the media data stream (18) using the statistical values (2a-2c).

Description

DEVICE AND METHOD FOR DETERMINING THE QUALITY OF ONE

MEDIA DATA STREAM

description

The present invention relates to determining a quality of a media data stream of varying quality. Embodiments relate to a parametric model for measuring the quality of a media data stream

For the transmission of audio and video signals, special coding methods for data reduction are available in modern transmission technology. These are used to provide the end user with the best possible quality, depending on the current capacity of the transmission channel. In the following description, the terms GOP, intra-frame, I-frame, P-frame and B-frame are used, which are used in connection with known video coding methods, such as H.264 standard or H.265 coding methods. Standard, specify the way in which individual video pictures are compressed. In order to provide efficient compression, known video coding methods do not completely re-encode each image, but exploit redundancy by only occasionally encoding a complete image. For the intermediate images, for example, an estimate of movements between the images recodes only a difference to a previous image or to a subsequent image, so that, as a consequence, in a given media data stream or bitstream, not every image is individually decodable, but rather - that dependencies exist between the individual pictures or frames. The smallest unit to be independently decoded is the so-called "Group of Pictures" (GOP), and such GOP structures usually begin with a fully encoded picture called I-frame or intra-frame, followed by P or B Frames that encode subsequent pictures depending on the previous or following pictures.

Fig. 6 shows a typical transmission link for transmitting a media content, eg a video, to an end user or client. The transmission path comprises a database 10 which stores media data representing the media content to be transmitted, a streaming server 12 which takes over the delivery of the media data to the end user 14 via a network 16, the network 16 itself and the one already - The end user or client 14 thought that the client 14 received the media data and included a media player or player to display the received media data on a screen. The typical transmission structure shown in FIG. 6 effects transmission of media content, for example using adaptive streaming techniques such as MPEG DASH (Dynamic Adaptive Streaming over HTTP) or Apple HLS (HTTP Live Streaming). Such adaptive streaming methods make the media content in multiple quality levels available to the client 14 or to the media player or player installed on the client. If the content of the media is transmitted at a low quality level, the transmission of the content will require a lower data rate than is required to transfer media content of a higher quality level. The best possible quality for displaying media content by a player will vary depending on how high the data rate is available to transfer the media content at a particular time or time. In order to efficiently serve a large number of players with different requirements at the same time, adaptive streaming methods rely on precoded data. For example, a video is divided into time segments, and for each period, pre-coded versions with different quality levels are created, the amount of data required to represent the low quality levels being significantly less than the amount of data required to represent high quality levels. A predetermined protocol allows the players to load the video with varying quality levels by switching at time limits between the individual quality levels. Such time segments may be, for example, 2 to 10 seconds long and are also referred to as chunks.

The player installed on the client 14 includes an input buffer in which a sufficient amount of data is held to decode a media content. If the input buffer of the player threatens to run dry, the player will request data or chunks that are of low quality and can therefore be transmitted faster. If, on the other hand, the player's input buffer is well filled, then the player will access data of higher quality, requesting higher quality media content from the server 12 because of the well-filled input buffer, sufficient data is available for decoding and presentation, so that the longer transmission times for the data with higher quality can be accepted. That way At the receiver side, in general, a continuous data stream, for example a video film, without disruptive interruption by refilling the input buffer, but with varying image quality, depending on the level of the input buffer and the available data rate for the transmission of the requested image data.

The just mentioned interruptions are also called stalling or rebuffering. Only in rare cases does such an approach lead to interruptions and the video quality is regulated to the best possible quality with the existing network connection, for example an Internet connection. The quality perception of a viewer is determined mainly by the coding quality and the frequency and duration of interruptions, as well as by the time required to initially, ie at the start of playback, fill the input buffer of the player sufficiently with data, which also UDP (User Datagram Protocol) -based transmissions or non-data-link transmissions can be subject to transmission interference, whereas other known streaming techniques are based on Transmission Control Protocol (TCP), which prevents transmission interference As mentioned above, it is desirable to automatically judge the quality of the media content perceived by the end user, such as the video being played, by a suitable means, for which there are various proposals and also international standards, for example those below briefly explained concepts for the automatic assessment of the quality of a video.

Known approaches or concepts for automatically assessing the quality of the video perceived by an end user can be divided into three classes. One class uses full-reference (FR-) quality measurement techniques, another class uses no-reference (NR) quality measurement techniques, and another class uses reduced-reference (RR) quality procedures.

Full-Reference (FR) quality measurement methods compare the original media content, which has not been degraded by compression, with the media content whose quality is to be determined. The disadvantage here is the need to access the original version of the media content. No-reference (NR) quality measurement methods determine the quality based solely on the received media content or received data stream representing that media content. Optionally, only transmission disturbances are detected here and quantified to determine the quality measure.

Reduced-Reference (RR) quality methods represent an interim solution between the FR and the NR methods, in that the received data stream or the received media content is used not only for the determination of the quality of the reception side, but in real time on the transmitter side intermediate results are used to determine the quality of reception. These parameters or intermediate results are usually transmitted (coded) in the transmitted media data stream.

The object of the present invention is to determine the quality of a media data stream perceived by a human viewer with high accuracy, low effort and based solely on information present on the receiver side.

This object is solved by the subject matter of the independent patent claims.

The present invention is based on the recognition that, in addition to the above-mentioned classification of existing quality measurement methods, there are further possibilities for distinguishing them, namely as to which input data are used for the analysis. Here, a distinction can be made between media content-based methods, parametric methods and hybrid methods. For example, media content-based methods analyze the individual pixels of a video image or an audio signal, whereas parametric methods of analysis use information obtained from the media data stream or bitstream, such as a bit rate or information about transmission interference. Where available, additional meta-information is included in the assessment, for example the codec type. Hybrid methods are a mixture of the media content-based methods and the parametric methods and use both types of information. Known standardized media content-based methods are either either FR methods or RR methods, whereas standardized parametric methods are all NR methods. In the field of hybrid methods, NR methods, RR methods and FR methods are standardized. Parametric NR methods have advantages in terms of usability and computation time requirements, but they represent a compromise in terms of achievable accuracy and universality compared to FR methods.

According to the present invention, it has been recognized that with parametric methods, in particular parametric NR methods, with a suitable choice of the indicators derived from the received media content, a high-quality quality assessment can be achieved without access to or access to a pixel Reference signal is necessary, whereby it is achieved that with ease of application of the method, a high accuracy is achieved. Embodiments of the present invention thus represent a parametric NR method according to the classification given above, which, however, differs from the approaches known in the prior art by the indicators or parameters used and by the manner of temporal aggregation of these parameters that due to the choice of indicators / parameters and the choice of aggregation, high accuracy is achieved without sacrificing the advantage of ease of use.

Thus, a core idea of the present invention is that it has been recognized that the quality of a media data stream perceived by a human viewer can be determined with high accuracy and little effort based solely on information provided at the receiver side, embodiments based thereon that rely on the Receiver side of a streaming server received data, such as a media data stream to analyze and aggregate the information obtained in an analysis as indicators, possibly together with further a-priori knowledge of the transmission system, to assess the quality of the received media signal.

Advantageous embodiments of the present application are the subject of the dependent claims. Preferred embodiments of the present application will be explained in more detail below with reference to the accompanying drawings. Show it: Fig. 1 is a schematic block diagram of a network environment with a

Server and a client, between which a Mediadatenstromübertra- tion of varying quality takes place, and is used in accordance with an embodiment of the inventive approach to determining quality; a schematic representation of a media data stream and the media content encoded in the media data stream, which is provided on the server side with different qualities; a block diagram of the device according to the invention according to an embodiment; 4 shows a block diagram of the device according to the invention according to a further embodiment;

5 is a flowchart of a method for determining a quality of a

Media data stream during a measurement period according to an embodiment; and

Fig. 6 shows a known server-client network environment for bit rate adaptive

Media data stream transmission.

Fig. 1 shows an environment with a server and a client, in which the concept for determining a quality of a media data stream according to an Ausführungsbeispie! used in the present application. 1 shows in particular a transmission path, as already shown with reference to FIG. 6, with a media database 10 in which media contents are stored precoded, a server 12 having access to the media database 10, and a client 14, FIG. which is communicatively connected to the server 12 via a network 16. The client 14 is, for example, a computer program or an application running on a terminal of a user. The terminal may be, for example, a mobile terminal, such as a mobile phone, a notebook or the like. The network 16 may include a wired portion and / or a wireless portion. The server 10 For example, a computer or a computer network, and the media database 14 includes, for example, one or more hard disks or other non-volatile storage. As part of the communication between server 12 and client 14, the server 12 sends, for example, a media data stream via the network 16 to the client 14. The following description assumes, by way of example, that the media data stream is a video data stream representing a video as media content , The present invention is not limited to videos, but the inventive concept can also be applied to other media content, such as audio content, 3D grid models or the like.

Fig. 2 exemplifies a media data stream 18 and the media content 20 transmitted therewith, e.g. a video. The media data stream 18 comprises a sequence of data stream sections 22 into which different media sections 24 of the media content 20 are encoded in a quality-to-bit rate varying over the media content. The video 20, which serves as an example of a time-varying media signal in FIG. 2, is subdivided, for example, into the media sections 24, in that the individual media sections 24 form contiguous consecutive time sections of the video directly adjacent one another. Each media section 24 thus comprises a respective sequence of pictures 26 of the video 20. The variation of the quality-to-bit rate with which the sections 24 are coded into the data stream sections 22 is controlled, for example, according to suitable criteria, on the client side 14 to avoid emptying a data stream buffer, a so-called "stall", in the real-time reproduction of the media content or video 20 on the client 14, and on the other hand overfilling the buffer The adaptive streaming can be, for example, http-based streaming of the media data stream 18 such as MPEG DASH, whereby the server 12 provides the client 12 with information about available downloadable quality-to-bit rate levels via so-called MPDs (Media Presentation Description) Based on these levels, an adjustment module located in the client 14 the selection among the available levels depends on estimates of the effectively too Perform and modify the available transmission data rate from server 2 to client 14 and depending on a current buffer level at the client 14.

As shown in FIG. 2, in the media database 10, for example, for each of a plurality of quality-to-bit rate levels Q, -Q, for each media portion 24, a data stream portion is stored. These are illustrated in FIG. 2 with rectangles 28. The actual data output for a client 14 media data stream 18 thus expires a sequence or stream of data stream sections 22 together, wherein each data stream section 22 is assigned to a different one of the media sections 24 and corresponds to one of the stored data stream sections 28 available for this media section 24. FIG. 2 shows by way of example that the media sections 24 are transmitted between the times t ₀ , t ₁ , t ₂ and t ₃ in the media data stream 18 with the quality levels Q ₃ and Q ₂ , but as stated, this selection is situation-dependent for the individual clients 14. In FIG. 2, the situation-dependent quality-to-bit rate setting is indicated by the arrow 30. It can, as described, for example, be made by the client 14. It should be noted that the media sections 24 may be different and include, for example, media sections coded separately in the media data stream, such as GOPs, ie, picture sequences that are independently decodable, such as IPPP ... PP picture sequences or similar. In units of such media content, the media data stream 18 may change the quality-to-bit rate in stages QQ ₄ .

In order to determine the quality of the media data stream at the client 14 or at another location along the transmission path between the server 12 and the client 14, the quality assessment concept described below provides that a device 32 (see FIG. 1) from the data stream sections 22 a plurality of first quality indicators determines, for a plurality of short-term durations, respectively, a short-term quality value determined using the plurality of first quality indicators, a plurality of statistical values determined using a plurality of short-term quality values, and a quality measure q for the quality of the media data stream using the statistical values. According to embodiments, it may be provided to further provide the quality measure q for the quality of the media data stream using a plurality of second quality indicators, e.g. provided by the client 14 (see the dashed arrow in Fig. 1).

Exemplary embodiments relating to the implementation of the inventive concept for determining a quality of a media flow during a measurement period are explained in more detail below with reference to FIGS. 3 shows a block diagram of the device according to the invention according to an exemplary embodiment. FIG. 3 shows, on the one hand, the device 32 according to the invention, which has already been explained briefly with reference to FIG. 1, and the client 14. In the exemplary embodiment shown in FIG. 3, the client comprises a media player or player 34 which stores the media data stream 18 is received and processed by the network 16. The player 34, which implements on the client 14 3, according to the embodiment shown in FIG. 3, may further comprise an input buffer 34a for buffering the received data from the media data stream, as explained above, and data depending on the filling state of the buffer 34a and the available data rate with an appropriate quality to fill the buffer. In the embodiment described with reference to FIG. 3, it is assumed that the media data stream is a video data stream, and the client 14 further comprises a display 36 for displaying the video information received, decoded and edited by the player. In other embodiments, the media data stream may include audio information, in which case the client in addition to the player 34 includes a suitable output device, such as speakers or the like. In yet other embodiments in which the video also includes audio information, eg, a sound signal, the display 36 either integrally or separately includes suitable output devices for the audio information, such as speakers.

The inventive device for determining the quality of the media data stream comprises a bit stream analysis module 38, which is also referred to as a media data stream analysis module. The device 32 further includes a short-term aggregation module 40 and a long-term aggregation module 42. The bitstream analysis module 38 is configured to determine from the media data stream 18 a plurality of first quality indicators 1 a to 1 y output by the bitstream analysis module 40 become. The short-term aggregation module is configured to determine, for a short-term duration shorter than the measurement duration, a short-term quality value q _k using the plurality of first quality indicators 1 a to 1 y, wherein the short-term aggregation module is configured to for at least one short-term quality value q _k for a plurality of short-term durations k. The long-term aggregation module 42 is configured to determine for the measuring period a plurality of statistical values 2a to 2c using a plurality of short term quality values q _k, and a Quaiitätsmaß q for the quality of Mediadatensiroms using the statistical values to Lie remote , According to embodiments, it may be provided that the long-term aggregation module 42 comprises a statistics module 44 which is connected to the short-term aggregation module 40 to receive the short-term quality values q _k from the short-term aggregation module 40 and the statistical values 2a based on these received short-term quality values to generate 2c and to provide this to an aggregation module 46 which calculates and outputs the quality measure q based on the received statistical values. According to further embodiments, it may be provided to additionally determine the quality measure q using second quality indicators 2d to 2p, wherein the second quality indicators in the exemplary embodiment illustrated in FIG. 3 are provided by the client 14, more precisely by the player 34, the aggregation module 46 of FIG Long-term aggregation module 42 are provided.

The exemplary embodiment illustrated with reference to FIG. 3 shows the media player 34 of the client 14, which controls the communication with the server 12 shown in FIG. 1 and, for example, at the request of an end user or for monitoring purposes also automatically parts (chunks) of a video or a requesting complete video from the server and providing the decoded content after the decoding of the media data stream, for example on the display device 36. As mentioned above, the measuring device according to the invention comprises the modules 38, 40 and 42, the bitstream analysis module 38 being measured Embodiment permanently communicates the communication between the player 34 and the server 12, for example by monitoring the network interface of the player 34. The analysis of the thus observed bitstream provides the mentioned indicators 1 a to 1 y, which can be averaged according to embodiments over a short period. The thus obtained indicators are passed to the short-term aggregation module 40, which generates a sequence of short-term quality values g _k provided to the statistics module 44 of the long-term aggregation module 42, the statistics module 44 processing the received indicators into the statistical values 2a to 2c, which are then combined with the indicators 2d to 2p provided by the player 34 by the aggregation module 46 to determine the final quality measure q.

The inventive approach according to the embodiment illustrated with reference to FIG. 3 is based on analyzing the media data or the media data stream 18 received on the receiver side by a streaming server, and the information obtained in this analysis as the first indicators 1 a to 1 j together with indicators 2d through 2p provided by the player 34 to evaluate the quality of the received media content.

According to embodiments, one or more of the following indicators are obtained by the analysis of the media data stream: A resolution ratio indicator 1 a (resolution ratio indicator) indicating the ratio between a resolution of the representation of the media content by the display 36 and by the output device of the client 14 to the actually transmitted in the media data stream 18 resolution, the resolution by the receiver-side display device 36 is known as a rule a-priori. The indicator 1a carries out a scaling of the media signal, for example the video signal, and takes account of the fact that the different quality levels requested by the server 16 generally also involve different image resolutions, which are all the same size on the screen 36 by the player 34 be scaled.

A frame rate indicator 1 b (frame rate indicator) indicating the number of video frames per second transmitted in the media data stream 18. A reduction in video per second is generally associated with a reduction in perceived video quality, especially in high-motion scenes.

A bit-per-pixel indicator 1 c (bit-per-pixel indicator) indicating the number of bits that each pixel in the media data stream 18 is encoded on average. The higher this number, the sharper and better the viewer appears to be.

An I2NI frames indicator 1 d (12NI frame indicator) indicating the ratio between the number of intra frames and non-intra frames in the media data stream 18. This indicator is a measure of the efficiency of video encoding.

An l2Ni Ratio Indicator 1e (12NI Ratio Indicator) that expresses the ratio between the number of bits used to transmit intra-frames to the number of bits used to transmit non-intra-frames. Frames are used. This indicator is a measure of the efficiency of video encoding.

A bit rate indicator (bit rate indicator) that displays the average bit rate used to encode a portion of the media data stream 18 in bits / sec. In general, the more bits used for coding, the higher the video quality will be. An AvqGOP2Avql indicator 1g, which indicates the ratio of the number of bits used on average to encode a complete GOP structure to the number of bits used in the means to encode only the I-frames. This indicator is used to estimate the complexity of the scenes in section k. The more complex a scene is, the more bits are needed to achieve high video quality.

An AvqGOP2AvqP indicator 1 h, which indicates the ratio of the number of bits used in the means for coding a complete GOP structure to the number of bits used in the means for coding only the P-frames.

This indicator is used to estimate the complexity of the scenes in section k. The more complex a scene is, the more bits are needed to achieve high video quality. An AvgGOP2Avq B indicator 1 / which indicates the ratio of the number of bits used in the means for coding a complete GOP structure to the number of bits used in the means for coding only the B-frames. This indicator is used to estimate the complexity of the scenes in section k. The more complex a scene is, the more bits are needed to achieve high video quality.

• An AvqGOPSize indicator 1, which indicates the number of bits used on average to encode a GOP structure. This indicator is used to estimate the complexity of the scenes in section k. The more complex a scene is, the more bits are needed to achieve high video quality.

The above-mentioned indicators can be normalized to any reference values, for example, the number of frames, the number of GOP structures, a period during which these indicators are detected, or the like. Furthermore, a scaling of the values is possible. Alternatively, these can be used without further processing as numerical values without a fixed reference value. The indicators presented above are obtained from the media data stream 18 by the bitstream analysis module 38 and passed to the short-term aggregation module 40, where the indicators may be weighted and averaged for shorter time periods, eg, for a few seconds or for the duration of a chunk. The short-term aggregation module 40 causes aggregation of the short-term indicators for each short period of time, e.g. For the duration of a GOP Structure or for the duration of a chunk, so that a short-term quality measure »q _k results for just this duration.

According to embodiments serve as methods for aggregation of the indicators, a linear combination of all or a subset of the indicators 1 a to 1; ^' , an LNorm of all or a subset of the indicators 1 a to 1 j, or a sigmoidal averaging of all or a subset of the indicators 1 a to 1 y, the sigmoidal averaging taking place according to the following formula:

where x to x _{Nik represent} the indicators 1 a to 1 /, or a subset thereof, for the period k. The constants a,., B _n and c "are set for each indicator. The values of the constants a _n , b _n and c, "of the norms and the weights depend inter alia on the codec used for the transmission and its settings, and according to embodiments it is intended to provide several sets of these coefficients in the device 32 according to the invention, for example in a suitable memory to select them dynamically as the codec knows. To determine these constants, for example, numerical optimization methods can be used during model development. These optimization methods assume a large body of subjective results, that is, results obtained by questioning subjects, and try to minimize the error between the prediction of the method of the invention and the subjective evaluation. Depending on the selected aggregation function, "least squares" methods, which minimize the mean square error, are also available here.

The following table gives example values for these constants for the 1 a to 1 g indicators.

aO -1.5986833

bO 1.0

la al 17.8024441

bl 0.18148752

cl 1.69030516

1b a2 1

b2 0.99999994

c2 0.99999999 lf a3 4.61565096

b3 5.18730877

c3 -0.82385795

1c a4 -8.43013201

b4 13.5090335

c4 9.6146457

ld a5 -0.54587484

b5 2.61358404

c5 2.3581692

lg a6 2.17889465

b6 2.94441327

C6 0.74818627

le a7 -8.33230432

a7 -0.69031332

c7 6.06174126

The constants a ₀ and b _Q serve to scale the results to a desired quality scale, so that the result q _k becomes directly comparable to results of a subject's evaluation of the same time period k of the media stream.

The aggregation methods mentioned above can be used individually or in combination, for example, a combination of the aggregation methods may include all of the mentioned methods or a subset of the methods, for example only two of the mentioned methods.

The sequence of short-term quality values generated by the short-term aggregation module 40 is provided to the long-term aggregation module 42, or more precisely the statistics module 44, which generates one or more of the following statistical values for the entire measurement duration:

An average-short-term-score (ASTS) value 2a (short-term quality value minimum value) which indicates the average value of the K short-term quality values obtained. The ASTS value is calculated under the assumption that q _{s 1,} q _Si2, describe q _s .KK short-time quality values as follows:

ASTS = ii <7s, fc • specifies an aximum-short-term-score-fMSTSj value 2b, (maximum short-term quality value) of a maximum value of the obtained short-term quality values. Assuming that q _s q _{s 2} , ■■ ·, Q _S .K specify short-term quality values, the MSTS value is calculated as follows:

MSTS = max {q _Sjl , q _Sj2 , ..., q _SjK )

• Histogram-of-short-term-scores (HSTS) values 2c, (short-term histogram values) which each indicate a relative frequency with which a short-term quality value lies within a predetermined value range. To determine the HSTS values, a scale of the possible values that q _{Sik can} assume is divided into a predetermined number of ranges, eg, into five ranges, and the relative frequency is determined by dividing each of these ranges into the values q _Si k occurs. Each of these frequency values represents its own statistical value, and assuming that the q _sk values are in the range [1; 5], an exemplary division of the value ranges is as follows:

{1 -> 1.8, 1.8-> 2.6, 2.6-> 3.4, 3.4-> 4.2, 4.2-> 5.0}

The statistical values calculated by the statistics module 44 are passed to the aggregation module 46 which, in one embodiment, numerically determines the quality value with high accuracy by applying one or more aggregation methods such that the quality value corresponds to a quality value as determined by a human observer Watching the video would be awarded at the same time. According to embodiments, a linear combination of all or a subset of the statistical values 2a to 2c, an LNorm of all or a subset of the indicators 2a to 2c or a sigmoidal averaging of all or a subset of the indicators 2a to 2c are suitable for the aggregation. Significant averaging is carried out according to the following formula:

(2) in which ! to x _{N represent} the statistical values 2a to 2c or a subset thereof, and the constants a. "b" and c "are predetermined. As has already been described above in connection with the first indicators 1 a to 1 /, values for the constants, standards or weights may depend on the codec used for the transmission and its settings, so that according to embodiments also for the aggregation in the aggregation module 46 several sets are stored by corresponding coefficients and can be dynamically selected depending on the codec used. To obtain the constants a _n , b _n and c ", the same numerical methods as described above can be used.

The constants a ₀ and b ₀ in turn serve to scale the results to a desired quality scale, so that the quality measure q is comparable to the result of a test conducted by a subject of the same media data stream. According to further embodiments of the present invention, in addition to the statistical values obtained from the indicators 1 a to 1 / can be resorted to other indicators, which are also obtained from an observation of the bitstream, and which relate to the entire measurement period and helpful for determining video quality. Alternatively, such information may also be provided by the player, who knows what quality level he has requested when from the server. These additional indicators obtained from the observation of the bitstream for the duration of the measurement comprise one or more of the following indicators:

• A Number-of-upward-switches (NUSj-2d indicator) that displays the number of jumps from a lower to a higher quality level.

• A Number-of-downward-switches (NDS) 2e indicator that shows the number of jumps from a higher to a lower quality level.

• A Total'number-of-switches (NS) Indicator 2f, which shows the total number of jumps in quality, the larger this number is, the higher the probability that a user will jump in quality perceives. • An average-upwarä-switch-amplitude (AvgUpward) -lndjkator 2g (Average-Up-Circuit-Amplitude-Indicator), which shows the average amplitude of the switch from low to high quality levels. The amplitude indicates the distance between the quality levels. In the simplest case, this is given by a sequential numbering of the quality levels on the server in ascending or descending order. Preferably, a rough estimation of the quality difference of quality levels is already made here, thus defining a kind of perceptible distance between the levels. The larger this number is, the higher the probability that a user will perceive quality leaps.

• An Ayerage-dw 2h (Average Downshift Amplitude Indicator), which shows the average amplitude of high to low quality switching.

In the exemplary embodiment illustrated with reference to FIG. 3, the further indicators 2 d to 2 h are provided by the player 34 of the client 14 and received by the aggregation module 46. These additional indicators 2d to 2h can be aggregated together with the statistical values 2a to 2c for detecting the quality measure q in the manner described above, using, for example, a linear combination, an L norm or a sigmoid averaging of all or a subset of the indicators 2a to 2h, as explained above in connection with equation (2). A combination of two or all aggregation methods is also possible. According to further embodiments of the present invention, it is provided to exploit the receiver-side detailed information for stalling and initial buffering. Such information may be provided, for example, by the player 34 of the client 14. This information on staling and initial buffering provides further indicators, which are also related to the entire duration of the measurement, which include one or more of the following indicators. For all of these indicators, the longer the more frequently and the closer to the end of the measurement period a stagnation occurs, the more annoying this is perceived by humans.

An Averaqe-rebuffering-duration (AvgRebufDur) - ndator 21 (Average refresh time indicator), which indicates the average duration of interruptions caused by staling. • An average-plavout-between-rebufferinqs (APBB) ^ ndjkator 2 / (average time-out re-buffering indicator) that specifies the playing time in seconds without breaks divided by the number of interrupts.

• A length-of-last-rebufferi 2k (longer-last-new-buffering indicator) that indicates the duration of the last interruption caused by stalling.

A position-of-page-rebuffering-reJative-to-end (PLLB) indicator 21 (last-re-buffering-relative-to-end-indicator position) indicating the duration of the media data stream since the Last interruption by a Stalling until the end of the measurement is played continuously. If no stalling is detected, this value corresponds to the total duration of the measurement or it can be set to an arbitrarily high value.

A Number-of-Equivalent (NBUFF) Indicator 2m (Refresh Count Indicator) that indicates the total number of interruptions caused by stalling during the measurement or measurement period.

A total rebuffering ^ 2n (total refresh time)

Indicator), which indicates the total duration of all interruptions caused by stalling during the measurement or measurement period.

A Tota! -Injtial-buffering-time- (1 1 B) - l_nd ikator 2o (total start buffering time indicator), which indicates the duration of the period from the beginning of the measurement to an indication of the video, this being not the preview of a first Frames means, but the first actual display of a moving image on the display 36. Excessive values for the initial buffering are often not tolerated by viewers.

• A duration-of-media-stream (Du ^ 2p (medium-duration)

Indicator) indicating the length of a media data stream in seconds. In the exemplary embodiment shown in FIG. 3, the further indicators 2 / to 2p are provided by the player 34 of the client 14 to the aggregation module 46, which aggregated in the manner described above, all or a subset of the statistic values and the indicators 2a to 2p, either by a linear combination, an L norm or a sigmoid average or a combination of these aggregation methods, as described above in connection with the equation (2 ) has been explained.

In the exemplary embodiment discussed with reference to FIG. 3, it has been assumed that the additional indicators 2 d to 2 p are provided by the player 34 of the client 14. However, the present invention is not limited to such a configuration, and in the case that the player 34 can not provide this information, for example, according to other embodiments, the additional indicators are provided by an external modeling of the player's behavior and an observation of the incoming bit stream. 4 shows a further exemplary embodiment of such an embodiment, in which, in contrast to FIG. 3, the player 34 of the client 14 does not provide the further indicators 2d to 2p. As can be seen from FIG. 4, the device 32 shown there corresponds essentially to that which was explained with reference to FIG. 3, but in contrast to FIG. 3 additionally a player simulation 48 is provided, which is designed to model the behavior of the media player 34 of the client 14 based on the incoming media data stream 18. The player simulation 48 makes it possible to obtain from the media data stream 18 the above-mentioned indicators 2 / to 2o and to provide them to the aggregation module 46 of the long-term aggregation module 42, where they are processed in the manner described above. Furthermore, in the exemplary embodiment illustrated in FIG. 4, the bitstream analysis module 38 is designed to obtain, in addition to the indicators 1 a to 1 ^", also the indicators 2 d to 2 h and 2 p from the media data stream 18, and these directly to the aggregation module 46 to determine the quality measure q It should be noted that some of these values may also be available as a-priori information, for example on the server of the streaming provider.

In the exemplary embodiments described so far, it has been explained that the device according to the invention is arranged between the network 16 and the client 14, but the approach according to the invention is not limited to this embodiment. In principle, the inventive approach can be used at any point in the transmission path, that is, for example, between the server 12 and the network 16, or between the media database 10 and the server 12. In accordance with preferred embodiments, the inventive apparatus and approach is implemented at the client 14, namely at a location where a human end-user can also view the media data stream. The inventive approach can be implemented as an independent measuring system, alternatively, the inventive approach can also be implemented as hardware or software in the player 34 or in another player, such as a set-top box.

5 shows a flowchart of a method for determining a quality of a media data stream during a measurement period according to an exemplary embodiment. In a step S1, a plurality of first quality indicators 1 a-1 j are determined from the media data stream. In step S2, for a plurality of short-term periods of time k in each case a short-term quality value q _k using the plurality determined by first quality indicators 1a-1j, the short-term period of time k is shorter than the measurement period. In step S3, a plurality of statistical values 2a-2c are determined for the measurement period using a plurality of short-term quality values q _k . In step S4, a quality measure q for the quality of the media data stream is provided using the statistical values 2a-2c.

Although some aspects have been described in the context of a device, it will be understood that these aspects also constitute a description of the corresponding method, so that a block or a component of a device is also to be understood as a corresponding method step or as a feature of a method step. Similarly, aspects described in connection with or as a method step also represent a description of a corresponding block or detail or feature of a corresponding device. Some or all of the method steps may be performed by a hardware device (or using a hardware device). Apparatus), such as a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some or more of the most important method steps may be performed by such an apparatus.

Depending on particular implementation requirements, embodiments of the invention may be implemented in hardware or in software. The implementation may be performed using a digital storage medium, such as a floppy disk, a DVD, a Blu-ray Disc, a CD, a ROM, a PROM, an EPROM, a EEPROM or a FLASH memory, a hard disk or other magnetic, optical or semiconductor memory are stored on the electronically readable control signals that can cooperate with a programmable computer system or cooperate, that the respective method is performed. Therefore, the digital storage medium can be computer readable.

Thus, some embodiments according to the invention include a data carrier having electronically readable control signals capable of interacting with a programmable computer system to perform one of the methods described herein.

In general, embodiments of the present invention may be implemented as a computer program product having a program code, wherein the program code is operable to perform one of the methods when the computer program product runs on a computer.

The program code can also be stored, for example, on a machine-readable carrier. Other embodiments include the computer program for performing any of the methods described herein, wherein the computer program is stored on a machine-readable medium.

In other words, an embodiment of the method according to the invention is thus a computer program which has a program code for performing one of the methods described herein when the computer program runs on a computer.

A further embodiment of the method according to the invention is thus a data medium (or a digital storage medium or a computer-readable medium) on which the computer program is recorded for performing one of the methods described herein.

A further exemplary embodiment of the method according to the invention is thus a data stream or a sequence of signals which represents or represents the computer program for performing one of the methods described herein. That one- The current or the sequence of signals can be configured, for example, to be transferred via a data communication connection, for example via the Internet. Another embodiment includes a processing device, such as a computer or a programmable logic device, that is configured or adapted to perform one of the methods described herein.

Another embodiment includes a computer on which the computer program is installed to perform one of the methods described herein.

Another embodiment according to the invention comprises a device or system adapted to transmit a computer program for performing at least one of the methods described herein to a receiver. The transmission can be done for example electronically or optically. The recipient may be, for example, a computer, a mobile device, a storage device or a similar device. For example, the device or system may include a file server for transmitting the computer program to the recipient. In some embodiments, a programmable logic device (eg, a programmable gate array, an FPGA) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform any of the methods described herein. In general, in some embodiments, the methods are performed by any hardware device. This may be a universal hardware such as a computer processor (CPU) or hardware specific to the process, such as an ASIC. The embodiments described above are merely illustrative of the principles of the present invention. It will be understood that modifications and variations of the arrangements and details described herein will be apparent to others of ordinary skill in the art. Therefore, it is intended that the invention be limited only by the scope of the appended claims, rather than the specific details presented in the description and explanation of the embodiments herein.

Claims

claims

1 . Apparatus for determining a quality of a media data stream (18) during a measurement period, the media data stream (18) transmitting varying quality media content, comprising: a media data stream analysis module (38) configured to select a plurality of media data streams (18) of first quality indicators (1 a-1j); a short-term aggregation module (40) configured to determine, for a short-term duration (k) shorter than the measurement duration, a short-term quality value (q _k ) using the plurality of first quality indicators (1 a-j) wherein the short-term aggregation module (40) is configured to provide at least one short-term quality value (q _k ) for a plurality of short-term durations (k), respectively; to determine and a long-term aggregation module (42) that is configured for the measurement period a plurality of statistical values (2a-2c) using a plurality of short term quality values (q _k), and a quality measure (q) for the quality of the media data stream (18) using the statistical values (2a-2c).

The apparatus of claim 1, wherein the long-term aggregation module (42) is configured to further provide the quality measure (q) for the quality of the media data stream (18) using a plurality of second quality indicators (2d-2p) the second quality indicators (2d-2p) are related to the duration of the measurement.

Apparatus according to claim 2, wherein the plurality of second quality indicators (2d-2p) comprises a plurality of first indicators (2d-2h), the first indicators (2d-2h) being responsive to a change in the quality of the media data stream (18) transmitted media content. Apparatus according to claim 3, wherein the first indicators (2d-2p) are provided by a player (34) for playing the media content or by the media data stream analysis module (38).

Apparatus according to claim 3 or 4, wherein the first indicators (2d-2p) are at least partially provided as a-priori information of the device.

Apparatus according to any one of claims 2 to 5, wherein the plurality of second quality indicators (2d-2p) comprises a plurality of second indicators (2i-2o), the second indicators (2i-2o) being based on frequency and duration interruptions in playing the media content and / or time to sufficiently fill a buffer (34a) of a player (34) at the start of playing the media content.

Apparatus according to claim 6, wherein the second indicators (2i-2o) are provided by the player (34) for playing the media content.

Apparatus according to claim 6, comprising a simulation unit (48) configured to simulate the player (34) for playing the media content in response to receiving the media data stream (18), the second indicators (2i-2o) being simulated by the simulation unit to be provided.

Apparatus according to any one of claims 2 to 8, wherein the second quality indicators comprise one or more of the following indicators: a number-of-upward-switch (NUS) indicator (2d) comprising a number of

Jumps in the media data stream (18) from a lower to a higher one

Indicates quality level; a number-of-downward-switches (NDS) indicator (2e) indicating a number of hops in the media data stream (18) from a higher to a lower quality level; a total number of switches (NS) indicator (2f) indicating a total number of quality jumps in the media data stream (18); an Average-upward-switch-amplitude (AvgUpward) indicator (2g) indicating a mean amplitude of the low to high quality switches in the media data stream (18); an average-down-switch-amplitude (AvgDownward) indicator (2h) indicating an average amplitude of the high to low quality switches in the media data stream (18); an average-rebuffering-duration (AvgRebufDur) indicator (2i) indicating an average duration of the stalls of the media data stream (18); an average-playout-between-rebuffering (APBB) indicator (2j) indicating a duration in seconds without interruptions divided by a number of interrupts; a length-of-the-last-rebuffering (LLB) indicator (2k) indicating a duration of the last interruption by a stalling; a position-of-last-rebuffering relative-to-end (PLLB) indicator (21) indicating a duration over which the media data stream (18) is continuously updated after the last interruption by a stalling until the end of the measurement period plays; a number-of-rebuffering (NBUFF) indicator (2m) indicating a total number of interruptions by stalling during the measurement period; a total rebuffering time (TBUFF) indicator (2n) indicating a total duration of all stoppage interruptions during the measurement period; a total initial buffering time (TIB) indicator (2o) indicating a duration of a period of time from the beginning of the measurement period to an indication of the video; and a duration-of-media-stream (duration) indicator (2p) indicating a length of the media data stream (18) in seconds. Apparatus according to claim 9, wherein the first indicators (2d-2h) comprise one or more of the following indicators: the number-of-upward-switches (NUS) indicator (2d), the number-of-downvvard-switches- (NDS) indicator (2e), the Total-number-of-switches (NS) indicator (2f), the Average-upward-switch-amplitude (AvgUpward) indicator (2g), and / or the Average -downward-switch-amplitude (AvgDownward) indicator (2h), and the second indicators (2i-2o) include one or more of the following indicators: the Average-rebuffering-duration (AvgRebufDur) indicator (2i) Average-playout-between-rebuffering (APBB) indicator (2j), the length-of-the-last-rebuffering (LLB) indicator (2k), the position-of-last-rebuffering relative-to- end (PLLB) indicator (21), the number-of-rebufferings (NBUFF) indicator (2m), the total rebuffering-time (TBUFF) indicator (2n), and / or the total -initial-buffering-time- (TIB) -liter (2o).

Apparatus according to any one of the preceding claims, wherein the first quality indicators comprise one or more of the following indicators: a resolution ratio indicator (1a), which is a ratio between a resolution in the presentation of the media content and that in the media data stream (18) indicates transmitted resolution; a frame rate indicator (1b) indicating a number of video subjects transmitted in the media data stream (18) for a predetermined period of time; a bits-by-pixel indicator (1 c) indicating a number of bits with which each bi-point in the media data stream (18) is encoded on average; a 12 NI frame indicator (1 d) indicating a ratio between a number of intra frames and non-intra frames in the media data stream (18); a l2NI ratio indicator (1e) indicating a ratio between a number of the bits used for transmission of intra-frames to a number of bits used for transmission of non-intra-frames; a bit rate indicator (1f) indicative of a mean data rate in bits / sec used to encode a portion of the media data stream (18); an AvgGOP2 Avg indicator (1g) indicating a ratio of a number of the bits used in the means for encoding a complete GOP structure to a number of the bits used in the means for coding only the I-frames, respectively; an AvgGOP2AvgP indicator (1 h) indicating a ratio of a number of the bits used in the means for coding a complete GOP structure to a number of the bits used in the means for coding only the P-frames; an AvgGOP2AvgB indicator (1i) indicating a ratio of a number of the bits used in the means for coding a complete GOP structure to a number of bits used in the means for coding only the B-frames; and an AvgGOPSize indicator (1j) indicating a number of bits used on average for coding a GOP structure.

Apparatus according to any one of the preceding claims, wherein the statistical values (2a-2c) comprise one or more of: an average short term score (ASTS) value (2a) which is an average of the short term quality values (q _k ) indicates; a Maximum Short Term Score (MSTS) value (2b) indicating the largest of the short term quality values (q _k ); and a plurality of histogram-of-short-term scores (HSC) values (2c) each indicating a relative frequency with which a short-term quality value (q _k ) is within a predetermined range of values.

Apparatus according to any one of the preceding claims, wherein the long-term aggregation module (42) is configured to obtain the quality measure (q) for the quality of the media data stream (18) by a linear combination, an L norm or a sigmoid average of at least a portion of the statistical values (2a-2c) and the second quality indicators (2d-2p) or by a combination of linear combination, L-norm or sigmoid-averaging.

Device according to one of the preceding claims, wherein the short-term aggregation module (40) is configured to generate a short-term quality value (q _k ) by a linear combination, an L norm or a sigmoid average of at least a part of the first quality indicators (1 a-1 j). or by a combination of linear combination, L-norm or sigmoid-averaging.

Apparatus according to any one of the preceding claims, wherein the media data stream analysis module (38) is configured to record and analyze communication between the player (34) and a server which broadcasts the media data stream (18) for a predetermined period of time to provide the first quality indicators (a-1 j).

The apparatus of claim 15, wherein the media data stream analysis module (38) determines the first quality indicators (a-1j) over a time that is less than or equal to the short-term time period (k).

Apparatus according to any one of the preceding claims, wherein the short-term period (k) comprises the duration of a GOP structure or the duration of a chunk, and wherein the measurement duration comprises a predetermined portion of the duration of the media content or the duration of the media content.

Device according to one of the preceding claims, in which the media data stream (18) comprises a sequence of data stream sections which transmit different media sections of the media content, wherein a change in quality takes place in units of data stream sections of the media data stream (18). Det, which are self-contained to be reproducible separately from other data stream sections of the media data stream (18), each data stream section comprising one or more media sections.

Apparatus according to claim 18, wherein the media sections comprise GOPs, individual files or an image of a video.

Apparatus according to any one of the preceding claims, wherein the media content transmitted by the media data stream (18) comprises a video or a video with audio information, the quality measure (q) indicating a video quality or a combined audio-visual quality.

A method of determining a quality of a media data stream (18) during a measurement period, wherein the media data stream (18) transmits varying quality media content, comprising:

Determining a plurality of first quality indicators (1 a-1 j) from the media data stream (18);

Determining, for a plurality of short-term durations (k), respectively, a short-term quality value (q _k ) using the plurality of first quality indicators (1 a-1 j), the short-term duration (k) being shorter than the measurement duration;

Determining, for the measurement time, a plurality of statistical values (2a-2c) using a plurality of short term quality values (q _k); and

Providing a quality measure (q) for the quality of the media data stream (18) using the statistical values (2a-2c).

22. Computer program with a program code for carrying out the method according to claim 21, when the program runs on a computer.