CN101356791A - System and method for providing quality feedback metrics for data transmission in rich media services - Google Patents

Abstract

An improved system and method for providing quality feedback metrics for data transmission in rich media services. Extensions are provided to quality measures concerning the transmission of rich media content to the client. Such measures are defined as new extensions in the PSS Base Vocabulary, PSS Quality of Experience and RTP/ A VPF. With the rich media client utilizing such measures, the server can assess the quality of the transmission and consider error recovery mechanisms, such as packet retransmission, to provide the client with the missing information.

Description

The system and method for quality feedback metrics is provided for the transfer of data in the rich media service

Technical field

The transmission of relate generally to rich-media content of the present invention.More specifically, the present invention relates to provide quality metric during the transfer of data in rich media applications.

Background technology

This part is intended to the invention provides background or context for what state in claims.Description at this can comprise the notion that can probe into, but notions of not necessarily having expected or probed into before those.Therefore, unless point out at this, the content of describing in this part is not a prior art for the application's description and claims, and does not just admit in this part that it is a prior art because be included in.

Rich-media content relates to the dynamically interactive type content of sending by individual interface, and these contents are enriched aspect figure, and comprises composition or multiple medium, comprises figure, literal, video and audio frequency.Telescopic video figure (SVG) is to be used for the main container that Rich Media presents.Up to date, the application of mobile device be based on text and interactivity limited.Yet, having color screen and a more advanced figure appearance of playing up the wireless device in storehouse along with more, the consumer is requiring to obtain more enriching experiences from their wireless application.Rich-media content streaming service in real time is very important for portable terminal, particularly is even more important in multimedia broadcast multicast services (MBMS), packet switch streaming service (PSS) and Multimedia Message transmission system (MMS) service field.

SVG is designed to describe the two-dimension vector graphics that is independent of resolution, and embeds other medium that have such as raster graphic, Voice ﹠ Video usually.SVG uses the event model and the animation notion of using from synchronous multimedia integrate language (SMIL) to allow interactivity.SVG also allows the unlimited scalability on the mobile device, and has strengthened the ability of user interface.Consequently, SVG is important all the more, and becomes one of core element that multimedia represents, especially for the rich media service such as the online updating of mobile TV, transport information, weather, news etc.SVG is based on XML's, thereby allows more transparent integrated with other existing web technology.Mobile scalable vector graphics (mobile SVG) by third generation partner program (3GPP) as new graphics standard, played key effect providing in improved figure and the image for mobile device.Recently, 3GPP and Open Mobile Alliance (OMA) have begun at the work based on Rich Media's activity.

In rich media applications, use the notion of scene and scene update." scene " described spatial organization, the time tissue of situation elements, the synchronizing information of situation elements, and mutual between the SVG element.Scene at first is sent to client usually and presents layout with initialization.Scene is＜svg〉＜/svg〉self-contained SVG document in the label, wherein animation and element can use＜g〉groups elements is to together.Scene can also comprise be included in＜defs＜/defs element definition in the piece, this definition can be used in current scene, perhaps can be used in follow-up scene update by element.Scene update is the incremental update to SVG DOM Document Object Model (DOM), and these upgrade and are sent to client device one at a time.These renewals comprise that the SVG element adds, element is deleted, element is replaced and element property is upgraded operation.

Although had some quality metric about packet loss, transmission quality and media repair information at medium (such as Voice ﹠ Video), the current rich media applications that is not used for is based on the scene of SVG and the quality metric of scene update.During rich media applications, client can be often to server report transmission and expression quality.This information is made for the best decision quite useful for server for adjusting transmission system.Current, be widely used for multimedia application with head it off such as the high-rise frame of PSS and MBMS.Scheme comprises: definition expression is used for the grammer of the necessary feedback data of this purpose, and follow-up these grammers is mapped to lower layer protocol, such as RTP (RTCP).But these schemes mainly are absorbed in continuous data, such as audio frequency, video and time control text, and do not have specific adaptation in the rich media applications that comprises SVG, discrete and continuous media.Along with the demand of disposing rich media applications continues to increase, that wants most is to define expansion to when front baffle, sends the relevant feedback data of quality to allow user end to server.

As mentioned above, current that in class SVG Rich Media represents, exist, only be elementary at local interaction (that is, strict) and such as a large amount of schemes of the basic remote interaction of HTTP GET/POST with incomplete at client-side.Use the real-time media stream of real-time transport protocol (rtp) can restore at packet loss to a certain extent.RTP Control Protocol (RTCP) can be used for monitor service quality and pass on about the participant information in the ongoing RTP session.RTCP is based on using the mechanism identical with the issue mechanism of packet controlling packet periodically to be transferred to all participants of session.RTP Control Protocol is carried out four functions.The first, premiere feature is to provide the feedback about the quality of data issue.Second function is to carry the persistent transmission layer identifier that is used for the RTP source, is called canonical name or CNAME.The 3rd function is to control the speed that sends bag, so that RTP is increased to a large amount of participants in proportion.At last, optional the 4th function is to pass on minimum session control information.

RTP/AVPF is based on the expansion of RTCP so that the recipient can provide feedback more timely to the sender on adding up, and therefore allows to realize short-term correction and effective repair mechanism based on feedback.This early stage feedback profile (AVPF) is safeguarded the AVP bandwidth constraints for RTCP, and has kept to the zoom capabilities of big group.A large amount of new SDP parameters define so that descriptive session in this profile.Correspondingly, the form of RTCP feedback message also is defined, and can be divided into three kinds: (1) transport layer feedback message; (2) the special-purpose feedback message of payload; And (3) application layer feedback message.Unfortunately, great majority are exclusively used in the audio-video application for the message format that RTP/AVPF advised, and are unsuitable for rich media applications.

PSS base vocabulary comprises four compositions, is called " PssCommon ", " Streaming ", " ThreeGPFileFormat " and " PssSmil ".It is to be subjected to true like this inspiration that vocabulary is divided into these compositions, that is, PSS comprises three different basic application: (1) pure Streaming Media based on RTSP/RTP, describe by the Streaming composition; (2) the 3GP file is downloaded or progressive download, is described by the ThreeGPFileFormat composition; And (3) SMIL represents, described by the PssSmil composition.SMIL represents to comprise Downloadable image, text etc., and RTSP/RTP streaming and Downloadable 3GP file.General ability is described by the PssCommon composition for all PSS use.Substantially use each other by synchronous source region branch for three.For pure Streaming Media, the source is RTP.For the 3GP file, inherit in the 3GP file format in the source.Represent for SMIL, regularly provide by the SMIL file.The PSS special ingredient comprises the attribute of a plurality of expression abilities.Yet, thereby the attribute that client end capacity provides feedback is not described.

For user, operator and ISP, the end-to-end quality of service is a high-importance.Quality of experience (QoE) has been contained large-scale service, and it comprises the wider bandwidth that is used for voice and speech, such as the new multimedia service of IPTV (IPTV), and the statistics of the bag that lose and impaired of content.Incumbently all have such pressure with emerging service provider, with by obtain the market share, activating business reduces cost simultaneously, and the loyalty of creating the client increases their operation achievement.A kind of is that QoE sends with the minimized method of client's turnover rate.QoE tolerance can help to strengthen this continuous media this contiguity and the synchronism of (for example, comprising dynamic and interactive media scene (DIMS) content).

3GPP QoE tolerance has been designated as the optional feature of PSS streaming server and client in the PSS system, and does not disturb the PSS service.Support the PSS client of this feature to carry out mass measurement, these mass measurements are gathered in the client QoE tolerance, and use content reception reporting process that this tolerance is reported to the PSS server according to measuring definition.Yet all tolerance that define in PSS all only are only applicable at least one in audio frequency, video, voice and the time control text media type, and are not suitable for other medium types, such as Composite tone, still image, bitmap graphics, vector graphics and text.The tolerance of any the unknown has all been ignored by client, and is not included in any QoE report.

Do not disturbing under the situation of MBMS service, the two is optional to MBMS QoE tolerance for MBMS streaming server and MBMS.Support the MBMS client of MBMS QoE tolerance to measure, these mass measurements are gathered in the client QoE tolerance according to measurement definition implementation quality usually, and use content reception reporting process that this tolerance is reported to the MBMS server.

U.S. Patent Application Publication No.2005/0249117 has described a kind of method of the data flow with different service quality (QoS) attribute being transmitted by the network link that is implemented in two or more channels of being used for.This method to arrive bag classify with determine their needs/the QoS attribute that is assigned with, and the bag that will classify places some logical channel queue, wherein the logical channel queue of Xuan Zeing has the suitable corresponding set of the QoS attribute that defines.Radio Link Controller is checked available channel, and at each channel, the logical channel queue that chosen content will transmit thereon.Selection to logical channel queue is carried out according to the set of QoS attribute.Therefore, each stream can have different QoS characteristics, comprises priority, reliability (ARQ, no ARQ etc.).Yet this system mainly pays close attention to and gives different logic queries with the QoS attribute assignment, does not but solve the problem of qos parameter self, especially for rich media applications.

U.S. Patent Application Publication No.2005/0169171 has described the system that is used to support end-to-end quality of service (QoS) control.Produce QoS profile identifier, it is mapped to qos parameter.This identifier transfers to the terminal station by wireless communication system, and qos parameter is determined based on the identifier that is received in this terminal station.Yet this system relates to the control qos parameter, rather than measures the real data that sends as QoS.And, utilize this system, must in end-to-end rich media applications, define the specific information of QoS tolerance.

Summary of the invention

The invention describes the system and method that is used for during the transfer of data of rich media applications, providing quality metric.Have a plurality of use-cases of interactive rich media service, wherein this quality metric can play very important effect.For example, interactive mobile TV service relates in end user interface provides the certainty of rich-media content (comprising audio-video content, text, figure, image) to play up and behavior together with TV and wireless channel.This service must be content providers navigation just in single application or service, and must allow the synchronous interaction of Local or Remote behavior, (for example: the content in the function of relevant menu or submenu, advertisement and end user's profile or service subscription) such as ballot and personalization.In this service, be useful as feedback to service from the quality metric of packet loss and impaired aspect data of description transport property.

The instant chat service may be incorporated in web camera or the video channel, perhaps is incorporated in Rich Media's blog services.The terminal use can register and preserve their surname, and exchanges messages.The rich media data that message provides with the terminal use dynamically appears in the instant chat service.This chatting service can be simultaneously secret or disclosed in one or more channels.Dynamically remind the terminal use to come from other users' new information.Dynamic message renewal meeting in this service occurs under the situation of not reloading the complete page.Describe quality metric that these in upgrading which are received improperly and can help the mistake reparation and hide, so that improve the overall customer experience of service.

In addition, video/song selects service to allow user they like from server selection/request in real time song or film.Content update can depend on which client has sent this request before, and perhaps it can be based on the priority of distributing to each client.Again in this case, QoS tolerance can be used to assess the song content of being asked and whether correctly arrives at client.

Remote subscriber interface (UI) is the mechanism of drawing user interface on other equipment that are supported in except those equipment of operation applied logic.The equipment of optimizing at some environment high is being created by manufacturer.Along with this equipment is used for various purpose scopes, their UI ability can considerably change; Screen size and ratio, color depth, have the windows system of various assembly sets, and input method makes the environment high difference.Simultaneously, application developer and UI designer are just attempting to create to playing up the user interface that podium level is optimized, thereby improve user experience by the application with various easy study and use.When this long-range UI on another equipment except that equipment of operation applied logic when playing up, need make some regulations and make the user this UI can be considered as local application and make it intuitively available.In this case, whether all UI contents are useful from long-range flow transmission correctly to another client all to QoS tolerance for assessment.

In all above-mentioned situations, application can be towards broadcasting or towards PtP.According to the present invention, each quality metric can define for transmission information, such as PSS base vocabulary is expanded, PSS Quality of experience (QoE) is expanded, RTP/AVPF is expanded and the transport layer feedback message is expanded, wherein the PSS Quality of experience is RTP packet loss, the incorrect SVG element list of enlivening the SVG element list, correctly receiving the decode that receives, impaired duration, impaired SVG group.Utilize under the situation of this measurement the next information that disappearance is provided to client of error recovery mechanisms that server can be assessed transmission quality and consider to retransmit such as bag in Rich Media's client.

Various embodiment of the present invention also provides QoE tolerance, and its analysis of statistical data and mistake that can be used for DIMS dedicated content and service is recovered and error concealment.This tolerance can be used for for example being reported in during the specific period loses quantity, impaired scene, impaired scene update and DIMS value (by the time remaining measurement of impaired scene and impaired scene update) impaired the duration at the RTP bag of each priority.

These and other advantages of the present invention and feature will become from following detailed description in conjunction with the accompanying drawings obviously with the tissue and the mode of the present invention's operation, wherein run through following accompanying drawing, and identical element has same reference numerals.

Description of drawings

Fig. 1 is the system diagram that the present invention can realize therein;

Fig. 2 is the perspective view of the mobile phone that can use in realization of the present invention;

Fig. 3 is the schematic diagram of telephone circuit of the mobile phone of Fig. 2;

Fig. 4 is the diagram that is used to be reported in during the specific period at the RTCP message of the packet loss of each priority of information;

Fig. 5 is used for the diagram that report nearest " enlivening element list " has the RTCP message of the element that how much is not correctly received the decode;

Fig. 6 be used for reporting which element is correctly received, decoding and in the diagram of current group of active RTCP message;

Fig. 7 is the diagram that is used for the RTCP message of impaired duration of indication during one group of bag; And

Fig. 8 is the diagram that is used to indicate the RTCP message of the group impaired owing to the packet loss of the contextual data that is used for each group.

Embodiment

Fig. 1 shows the system 10 that the present invention can use therein, comprises a plurality of communication equipments that can communicate by network.System 10 can comprise the combination in any of wired or wireless network, and wherein these networks include but not limited to one or more ad hoc networks between mobile telephone network, WLAN (wireless local area network) (LAN), Bluetooth personal local area network, ethernet lan, token LAN, wide area network, the Internet, the communication equipment etc.System 10 can comprise wire communication facility and Wireless Telecom Equipment.

For example, system shown in Fig. 1 10 comprises mobile telephone network 11 and the Internet 28.The connection of leading to the Internet 28 can include but not limited to that long distance wireless connects, short-distance radio connects, and various wired connection, and wired connection includes but not limited to telephone wire, cable line, power line etc.

The exemplary communication device of system 10 can include but not limited to mobile phone 12, composite type PDA and mobile phone 14, PDA 16, integrated message transmission device (IMD) 18, desktop computer 20, and notebook 22.Communication equipment can be fix or when carrying, move by the people in advancing.Communication equipment can also be in the travel pattern, includes but not limited to automobile, truck, taxi, bus, ship, aircraft, bicycle, motorcycle etc.Some or all of communication equipment can pass through to lead to wireless connections 25 transmissions and the receipt of call and the message of base station 24, and communicate with the service provider by the wireless connections 25 of leading to base station 24.Base station 24 can be connected to the webserver 26, the communication that this server 26 is supported between mobile telephone network 11 and the Internet 28.System 10 can comprise additional communication equipment and dissimilar communication equipments.Communication equipment is direct communication each other.

Communication equipment can use various transmission technologys to communicate, include but not limited to code division multiple access (CDMA), global system for mobile communications (GSM), universal mobile telecommunications system (UMTS), time division multiple access (TDMA), frequency division multiple access (FDMA), transmission control protocol/Internet protocol (TCP/IP), short message passing service (SMS), Multimedia Message passing service (MMS), Email, instant message passing service (IMS), bluetooth, IEEE 802.11 etc.Communication equipment can use various media to communicate, and includes but not limited to, wireless, infrared, laser, cable connection etc.

Fig. 2 and Fig. 3 show the representative mobile phone 12 that the present invention can realize therein.Yet should be appreciated that the mobile phone 12 or other electronic equipments that are not intended to limit the invention to a kind of particular type.The mobile phone 12 of Fig. 2 and Fig. 3 comprises display 32, keypad 34, microphone 36, earphone 38, battery 40, infrared port 42, the antenna 44 of shell 30, LCD form, smart card 46, card reader 48, wireless interface circuit 52, coding-decoding circuit 54, controller 56 and the memory 58 of general according to an embodiment of the invention UICC form.Independent circuit and element can be all types well known in the art, for example the mobile phone series in the Nokia scope.

The invention describes such system and method, be used for during the transfer of data of rich media applications, providing quality metric.The details that provides about mass measurement is below described.New expansion described here defines in PSS base vocabulary, PSS Quality of experience and RTP/AVPF.For the QoE that is exclusively used in rich media applications is provided information, some expansion of definition is necessary in PSS.Correspondingly, some expansion also defines in lower layer protocol RTP/AVPF, and these QoE data that expand to after the expansion provide the actual feedback service.PSS and RTP/AVPF are closely related to one another.The growth data that transmits in RTP/AVPF is that the expansion content from PSS derives.

Expansion to PSS base vocabulary.PSS base vocabulary comprises four compositions, is called " PssCommon ", " Streaming ", " ThreeGPFileFormat " and " PssSmil ".As described above, it is to be subjected to true like this inspiration that vocabulary is divided into these compositions, that is, PSS comprises three different basic application: (1) pure Streaming Media based on RTSP/RTP, describe by the Streaming composition; (2) the 3GP file is downloaded or progressive download, is described by the ThreeGPFileFormat composition; And (3) SMIL represents, described by the PssSmil composition.Provide the ability and the pattern of feedback in order to describe client, according to one embodiment of the invention, new attribute " FeedbackMethod " is added into the PssCommon composition.The character of this attribute is as follows:

Property Name: FeedbackMethod

Attribute definition: this attribute description client provides the ability and the pattern of feedback.

Composition: PssCommon

Type: literal

Legal value: " None ", " SMS ", " MMS ", " HTTP ", " RTSP "

Resolution rules: additional

Example:＜FeedbackMethod〉MMS＜/FeedbackMethod 〉

Expansion to the PSS Quality of experience.PSS Quality of experience tolerance feature all is optionally for the PSS server and client side, and does not disturb the PSS service.In the time of hope, support the PSS server of QoE tolerance feature to signal the activation and the collection of client QoE tolerance.Support the 3GPP PSS client of this feature to carry out mass measurement, these mass measurements are gathered in the client QoE tolerance, and when being required, use the QoE host-host protocol that this tolerance is reported to the PSS server according to measuring definition.The PSS client is measured this tolerance in transport layer, but can also measure this tolerance in application layer, to improve accuracy.For the present situation of describing sampling in the client and element (correctly receive, incorrect reception or lose), a plurality of new tolerance have been defined.

In order to report the quantity of RTP packet loss at each priority during specific period, according to one embodiment of the invention, grammer is Lost_Priority={1F, 3E, 0,21}; Range:Seq_Num=60000-60500.For the RTP bag of sequence number in 60000 to 60500 scopes, packet loss following (quantity of packet loss is expressed with hexadecimal format):

1F priority=0 packet loss

3E priority=1 packet loss

0 priority=2 packet loss

21 priority=3 packet loss

Should be noted that " Lost_Priority=... " " Range:Seq_Num=... " the position of field and corresponding value can change in other embodiments of the invention.In addition, four order of elements in the Lost_Priority field can change.

In order to report the tabulation of the element that does not correctly receive the decode in nearest " enlivening element list ", each group sends once " enlivens element list ".In response to " enlivening element list ", with undefined Lost_Element can be during this organizes transmission course or random time afterwards send repeatedly.According to one embodiment of the invention, the grammer that is used for this report is Lost_Element={element10, element22, element45}; Range:GRP=12.In having the group of GRP=12, three active elements have been lost.They are element10, element22 and element45.Shall also be noted that in other embodiments of the invention, " Lost_Element=... " " Range:GRP=... " the position of field and corresponding value can change.

In order to report which element is received, decodes and enliven by correct in current group, and no matter whether client has received " enlivening element list ", and client can send with undefined tabulation to describe the present situation that enlivens element in the client.According to one embodiment of the invention, the grammer that is used for this report is Active_Element={element1, elernent2, and element 4, element5, element8, element11}; Range:GRP=13.In this concrete example, current group GRP equals 13.In this group, correctly received six and enlivened element.They are element10, element22 and element 45.Should be noted that " Active_Element=... " " Range:GRP=... " the position of field and corresponding value can change in various embodiment of the present invention.

In order to report the Corruption_Duration by the count measurement of impaired scene update, the grammer among embodiment is Corruption_Duration_Sceneupdate={30 12345; 75 12555}; Range:GRP=16.At the RTP of the group that is used for having GRP=16 bag, total total impaired by caused two places of losing of the bag that is used for scene update.If be used for that of bag of concrete scene update has lost and not yet in effect the reparation, think that then this scene update loses.First sequence number of losing first bag of scene update is 12345.This scene update continues 30 bags.Second sequence number of losing first bag of scene update is 12555.This scene update continues 75 bags.Server determines whether to send remaining scene update based on above-mentioned information.In another example, at Corruption_Duration_Sceneupdate={30 12345; 12555}; Under the situation of Range:GRP=16, first sequence number of losing first bag of scene update is 12345.This scene update continues 30 bags.Second sequence number of losing first bag of scene update is 12555.If precise durations is unknown, then insert "? "

In order to report impaired group, the grammer among embodiment is Corrupted_Group={14}.In this example, because the packet loss of the contextual data of this group, so it is impaired to have the group of GRP=14.Server can select no longer to send any other bag relevant with these groups.Shall also be noted that " Corruption_Duration_Sceneupdate=... " " Range:GRP=... " the position of field and analog value can change in alternative embodiment of the present invention.

In order to know which group is the bag that lacks in fact belong to, the border between organizing continuously in the derivation scene is important.For this purpose, the S sign and the M that can use the RTP payload header that is used for the SVG data to define indicates.Whether the current bag of S sign (1 bit) indication comprises the starting point of current sampling, and the terminating point of the current sampling of M sign indication.The combination of M and S sign provides following information:

Work as M=1, during S=1, current bag comprises complete sampling.

Work as M=0, during S=1, current bag comprises first fragment of sampling.

Work as M=1, during S=0, current bag comprises the last fragment of sampling.

Work as M=0, during S=0, current bag comprises the intermediate segment of sampling.

In this case, the active tabulation of " sampling " expression SVG scene, scene update, SVG similitude information or SVG element.

Expansion to RTP/AVPF.At the internet draft (RTP/AVPF) (can find) that is used for based on the expansion RTP profile of RTCP feedback at www.ietf.org/internet-drafts/draft-ietf-avt-rtcp-feedbac k-11.txt, the special-purpose SDP attribute of new payload format is defined as indicating the SDP medium property of the ability of using the RTCP feedback, is defined as " a=rtcp-fb " in the document.Correspondingly, also define two expansions of RTCP bag so that transmission feedback message (transport layer/payload layer/application layer).The above-mentioned grammer that defines in PSSQoE is mapped in this SDP+RTCP profile.This SDP+RTCP profile can be used as the expansion of RTP/AVPF to be provided for the feedback message of rich media interactive.

The grammer of " rtcp-fb " attribute following (feedback kind and optional parameter all are case sensitives).The parameter of underscoring is redetermination according to the present invention:

rtcp-fb-syntax?＝“a＝rtcp-fb：”rtcp-fb-pt?SP?rtcp-fb-val?CRLF

Rtcp-fb-pt=" * "; Asterisk wildcard: be applicable to all form/fmt;

Such as in the SDP standard definition

rtcp-fb-val ＝“ack”rtcp-fb-ack-param

/“nack”rtcp-fb-nack-param

/“trr-int”SP1*DIGIT

/rtcp-fb-id?rtcp-fv-param

rtcp-fb-id ＝1*(alpha-numeric/“-”/“_”)

rtcp-fb-param ＝SP“app”[SP?byte-string]

/SP?token[SP?byte-string]

/; Empty

rtcp-fb-ack-param＝SP“rpsi”

/SP“app”[SP?byte-string]

/SP“aei”

/SP?token[SP?byte-string]

/; Empty

rtcp-fb-nack-param＝SP“pli”

/SP“sli”

/SP“rpsi”

/SP“lppi”

/SP“lei”

/SP“cdsu”

/SP“cg”

/SP“app”[SP?byte-string]

/SP?token[SP?byte-string]

/; Empty

In above-mentioned " rtcp-fb " attribute grammar, added an expansion to ACK (positive response), added four expansions to NACK (negative response).Following parameter is defined to use with " nack " with " ack "." aei " expression enlivens the element indication.The indication of priority bag is lost in " lppi " expression.The element indication is lost in " lei " expression.The impaired duration of " cdsu " expression scene update counting.The impaired group of " cg " expression.Corresponding to these SDP parameters, new feedback message is defined as follows based on RTP/AVPF.

Expansion to the transport layer feedback message: the transport layer feedback message is by the value PT=RTPFB sign of payload type (PT) field in the value RTCP header, and wherein whether PT field indication bag belongs to transport layer, payload layer or application layer.The single general transmission layer feedback message that defines in RTP/AVPF up to now is general NACK.By following FMT parameter it is carried out following sign:

0: unallocated

1: general NACK

2-30: unallocated

31: for the expansion in future of identifier code space is reserved

" lppi " NACK message defines in transport layer and defines below so that the indication of packet loss to be provided at each priority.This NACK message is identified by PT=RTPFB and FMT=30.

For losing priority bag indication (rtcp-fb-param=lppi), SVG RTP bag is divided into four priority.Priority is indicated by the GRP field in the RTP header.Losing at the bag of each priority during specific period of RTCP message report.The form that is used to lose the indication of priority bag has been shown among Fig. 4.16 SSN representative is used for the sequence number of the starting point of current measurement.It also is the sequence number that 16 ESN representative is used for the terminating point of current measurement.Each length of CP0, CP1, CP2 and CP3 all is 4.CP0 represents that priority is 0 packet loss quantity.CP1 represents that priority is 1 packet loss quantity.CP2 represents that priority is 2 packet loss quantity.CP3 represents that priority is 3 packet loss quantity.

The expansion of payload layer feedback message: payload layer feedback message is by the value PT=PSFB sign of the payload type that is used for the RTCP head (PT) field.Except the application layer feedback message, three payloads are special-purpose, and feedback message is current defines in RTP/AVPF.By following FMT parameter these message are carried out following sign:

0: unallocated

1: picture is lost indication (PLI)

2: sheet is lost indication (SLI)

3: reference picture is selected indication (RPSI)

4-14: unallocated

15: the application layer feedback message

16-30: unallocated

31: for the expansion in future of sequence number space is reserved

The definition of four new FCI forms that are used for the special-purpose feedback message of payload below has been discussed.In each form, there is SVG group (GRP) field, make the some RTCP/AVPF bags that belong to mutually on the same group comprise identical GRP id with their potential form.This bag represents to be used for the full detail of this special group (GRP) together.

Lose element indication (rtcp-fb-param=lei), FMT=27:SVG represents to comprise only effective many elements in current group.Every group once will " be enlivened element list " and be sent to client.RTCP message report shown in Fig. 5 has the element that how much does not correctly receive the decode in " enlivening element list " recently.The GRP field is 4, and the group number of indicating current bag to report.The PAD field is 4, and indicates the length of filler of this bag end, this length to be based on that byte calculates.The LLE field is variable, the expression lose element based on text list, separate mutually by comma or branch, for example " element1, element3, element5, element6 " or " element1; Element3; Element5; Element6 ".The PB field also is variable, the filler that indication is calculated based on byte.The length of PB field is indicated by PAD.It is 32 alignment that PB makes whole bag.

Enliven element indication (rtcp-fb-param=aei), FMT=28: the RTCP that defines among Fig. 6 reports which element is received, decodes and enliven by correct in current group.The GRP field is 4, and the group number of indicating current bag to report.The PAD field is 4, indicates the length of filler of this bag end, and this length is based on that byte calculates.The LAE field is variable, and the text based tabulation of element is lost in expression, is separated by comma, for example " element1, element3, element5, element6 ".The PB field also is variable, and expression is based on the filler of byte calculating.The PB field length is indicated by PAD.It is 32 alignment that the PB field makes whole bag.

The duration of by scene update counting impaired (rtcp-fb-param=cdsu), FMT=29:

The impaired duration during the group of RTCP message indication that defines among Fig. 7.Each impaired RTP from a special use unwraps the beginning, and continues a plurality of bags.The GRP field is 4, the group number of indicating current bag to report.The PAD field is 4, and indicates the length of the filler of this bag end.The CDn field also is 4, and represents the impaired duration of a particular list bag in current group.The SSNn field is 16, and indication comprises the sequence number of the corresponding scene update of at least one packet loss.This packet loss can not be repaired.(CD, SSN) pairing is listed with the incremental order of sequence number index.The PB field is variable, and indication is based on the filler of 4 bit counts.The length of this filler is indicated by PAD.It is 32 alignment that PB makes whole bag.

Impaired group (rtcp-fb-param=cg), FMT=30: the RTCP message indication shown in Fig. 8 is impaired group owing to the packet loss of the contextual data of these groups.The NCG field is 4, and the group number of indicating current bag to report.GRPn also is 4, and represents impaired group GRP.They are placed one by one.Total number is indicated by the NCG field.The PB field is variable, and represents filler.The length of filler can be derived from NCG.It is 32 alignment that the PB field makes whole bag.

Various embodiment of the present invention also relates to the basis that use is measured as introducing QoE based on the QoS tolerance of PSS and MBMS, and this QoE tolerance can be as the analysis of statistical data of DIMS dedicated content and service.This tolerance can be used for for example reporting during the specific period at the RTP packet loss quantity of each priority, impaired scene, impaired scene update and DIMS value (time remaining by impaired scene and scene update is measured) impaired the duration.In these each below is discussed.DIMS QoE tolerance all is optional feature for DIMS streaming server and client, and need not to disturb DIMS service itself.Support the DIMS client of this feature to carry out mass measurement, these mass measurements are gathered in the client QoE measurement, and use content reception reporting process that this tolerance is reported to the DIMS server according to measuring definition.Depend on based on the QoE of DIMS tolerance and to be used for the current 3GPP framework of QoE information from the client transmissions to the server.The unicast services that this 3GPP framework is included as among the PSS is used RTSP, and is multicast service use HTTP and XML object among the MBMS.

RTP packet loss quantity at each priority during specific period can be according to hereinafter reporting.Although the interruption of RTP sequence number indication disappearance bag, the importance of boolean's priority indication packet loss is low (P=0) or high (P=1).The grammer that is used for this report is Lost_Priority={X, Y}; Range:=Sy-Sz

The semanteme of above-mentioned grammer is, at the RTP bag of Sy in the Sz, loses X low priority packet and Y high-priority packets for the scope of sequence number generally.In alternative embodiment, can use the distortion of priority value.

In order to report impaired scene, if client can not construct effective DOM structure from scene content, then the DIMS scene is identified as impaired.Be used to report that the grammer of receiving impaired scene is: Corrupted_Scenes={S2Na, S7Nb}

According to the semanteme of above-mentioned grammer, scene and sequence number that sequence number is S2, continued Na bag are S7, and it is impaired to have continued Nb scene of wrapping.

In order to report impaired scene update, if client can not construct effective DOM structure after the current DOM structure that renewal is applied on the client, the DIMS scene update is identified as impaired.Be used to report that the grammer of impaired scene update is: Corrupted_SceneUpdates={S5Nc, S22Nd}

According to the semanteme of above-mentioned grammer, scene update and sequence number that sequence number is S5, continued Nc bag are S22, and it is impaired to have continued Nd scene update that wraps.

In another embodiment of the present invention, be used for the two tolerance of impaired scene and impaired scene update and can merge into the single public tolerance that is used for the DIMS medium type.

Except above-mentioned, can also report DIMS impaired duration value.This value is measured by the time remaining of impaired scene and impaired scene update.Be used to report that the grammer of this value is: DIMS_Corruption_Duration={S3Ta; S22Tb}

In this example, there are two impaired set.The sequence number of the bag of first in the first impaired set is S3, is Ta during the time remaining.The sequence number of the bag of first in the second impaired set is S22, is Tb during the time remaining.Should be noted that each impaired set can comprise the combination of impaired scene and impaired scene update.

In another example, DIMS_Corruption_Duration={S3Ta; S22?; Range:=Sy-Sz.In this example, the sequence number of the bag of first in the first impaired set is S3, is Ta during the time remaining.The sequence number of first bag of the second impaired set is S22.If precise time is unknown, then insert "? ""? " specify also and can use together in conjunction with other tolerance.In addition, should be noted that in specific embodiment of the present invention that sequence number can be used to indicate impaired the duration, and do not continue service time.

Invention has been described under the common scenarios of method step, in one embodiment, the present invention can realize that this computer program is included in the computer executable instructions of being carried out by computer in the network environment, such as program code by program product.

Usually, program module comprises routine, program, object, assembly, data structure etc., is used to carry out specific tasks or realize specific abstract data type.Computer executable instructions, related data structures and program module have been represented the example of the program code that is used to realize method disclosed herein and execution in step.The particular sequence of this executable instruction or related data structures has been represented the example of the respective action that is used for being implemented in the function that this step describes.

Software of the present invention and network are realized utilizing the standard program technology to finish, and utilize rule-based logic or other logics to come fulfillment database search step, correlation step, comparison step and steps in decision-making.Should also be noted that herein and the word that uses in claims " assembly " and " module " are intended to comprise delegation or the more realization of multirow software code and/or the equipment that manual input was realized and/or be used to receive to hardware of using.

Presented for purpose of illustration and purpose of description, provided above stated specification of the invention process.Above stated specification be not be exhaustive do not really want to limit the invention to disclosed exact form yet, also may have various changes and modifications according to above-mentioned instruction, or may obtain various changes and modifications from the practice of the present invention.Selecting and describing these embodiment is for principle of the present invention and practical application thereof are described, so that those skilled in the art can come to utilize the present invention with various execution modes and various modification with the special-purpose that is suitable for conceiving.

Claims (43)

1. one kind for the transfer of data in the rich media service provides the method for quality feedback metrics, comprising:

Receive rich-media content from remote equipment; And

In response to receiving described rich-media content, quality of service information is transferred to described remote equipment, described quality of service information comprises the expansion of at least one quality of service measures, described extended description the qualitative character of the rich-media content that receives.

2. the method for claim 1, wherein said at least one quality of service measures expansion comprises at least one expansion in the packet switch stream formula service vocabulary.

3. method as claimed in claim 2, described at least one expansion in the wherein said packet switch streaming service vocabulary comprises the ability of the feedback that description provides and " FeedbackMethod " composition of pattern.

4. the method for claim 1, wherein said at least one quality of service measures expansion comprises at least one expansion of packet switch stream formula service experience quality metric.

5. method as claimed in claim 4, described at least one expansion of wherein packet switch streaming service experience quality metric comprise during the specific period that is used to be reported in the rich-media content that receives the grammer at the quantity of each priority report RTP packet loss.

6. method as claimed in claim 4, described at least one expansion of wherein packet switch streaming service experience quality metric comprise and are used for being reported in the nearest grammer that enlivens the element list that element list correctly do not receive the decode.

7. method as claimed in claim 4, described at least one expansion of wherein packet switch streaming service experience quality metric comprise and are used for reporting correct reception, decoding and at the grammer of current group of those elements that enliven of reception rich-media content.

8. method as claimed in claim 4, described at least one expansion of wherein packet switch streaming service experience quality metric comprise the grammer that is used to report by the Coruption_Duration of the count measurement of impaired scene update.

9. method as claimed in claim 4, described at least one expansion of wherein packet switch streaming service experience quality metric comprise the grammer on border between the group of scene of the rich-media content that receives that is used for deriving.

10. the method for claim 1, wherein said at least one quality of service measures expansion comprise at least one expansion through the RTP profile of expansion that is used for based on the RTCP feedback.

11. method as claimed in claim 10 wherein is used for during the specific period that described at least one expansion based on the described RTP profile through expansion of RTCP feedback is included in the rich-media content that receives the indication at the packet loss of each priority.

12. method as claimed in claim 10 wherein is used for being included in the nearest indication that has how many elements correctly not received the decode in the element list that enlivens based on described described at least one expansion of passing through the RTP profile of expansion of RTCP feedback.

13. method as claimed in claim 10 wherein is used for not comprising the indicator element that correctly receives, decodes and enliven based on described described at least one expansion of passing through the RTP profile of expansion of RTCP feedback in current group of received data.

14. method as claimed in claim 10 wherein is used for the indication of the impaired duration during the group that described at least one expansion based on the described RTP profile through expansion of RTCP feedback is included in received data.

15. method as claimed in claim 10 wherein is used for comprising based on described described at least one expansion of passing through the RTP profile of expansion of RTCP feedback the indication of the group impaired owing to the losing of bag of reception contextual data.

16. the method for claim 1, wherein said rich-media content comprise dynamic and interactively medium scene (DIMS) dedicated content.

17. method as claimed in claim 16, wherein said at least one quality of service measures extended description be losing at a plurality of real-time transport protocol (rtp) bags of described each priority that receives rich-media content during specific period.

18. method as claimed in claim 16, wherein said at least one quality of service measures extended description receives from described remote equipment, impaired DIMS scene.

19. method as claimed in claim 16, wherein said at least one quality of service measures extended description receives from described remote equipment, impaired DIMS scene update.

20. method as claimed in claim 16, that wherein said at least one quality of service measures extended description receives from described remote equipment, impaired DIMS scene and impaired DIMS scene update both.

21. method as claimed in claim 16, wherein said at least one quality of service measures extended description receives from described remote equipment, duration of impaired DIMS scene and impaired DIMS scene update.

22. a computer program is included in the computer-readable medium, is used to the transfer of data in the rich media service that quality feedback metrics is provided, described computer product comprises:

Be used for receiving the computer code of rich-media content from remote equipment; And

Be used in response to receiving described rich-media content, quality of service information is transferred to the computer code of described remote equipment, described quality of service information comprises the expansion of at least one quality of service measures, described extended description the described qualitative character that receives rich-media content.

23. an electronic equipment comprises:

Processor; And

Memory, it can be connected to described processor communicatedly, and comprises:

Be used for receiving the computer code of rich-media content from remote equipment; And

Be used in response to receiving described rich-media content, quality of service information is transferred to the computer code of described remote equipment, described quality of service information comprises the expansion of at least one quality of service measures, described extended description the described qualitative character that receives rich-media content.

24. electronic equipment as claimed in claim 23, wherein said at least one quality of service measures expansion comprises at least one expansion in the described packet switch streaming service vocabulary

25. electronic equipment as claimed in claim 24, described at least one expansion in the wherein packet switch streaming service vocabulary comprises the described ability of the feedback that description provides and " FeedbackMethod " composition of pattern.

26. electronic equipment as claimed in claim 23, wherein said at least one quality of service measures expansion comprises at least one expansion of packet switch stream formula service experience quality metric.

27. electronic equipment as claimed in claim 26, described at least one expansion of wherein packet switch streaming service experience quality metric comprise concrete cycle of being used for being reported in the rich-media content that receives grammer at the quantity of each priority report RTP packet loss.

28. comprising, electronic equipment as claimed in claim 26, described at least one expansion of wherein packet switch streaming service experience quality metric be used for being reported in the nearest grammer that enlivens the element list that element list correctly do not receive the decode.

Be used for reporting correct reception, decoding and at the grammer of current group of those elements that enliven of reception rich-media content 29. electronic equipment as claimed in claim 26, described at least one expansion of wherein packet switch streaming service experience quality metric comprise.

30. electronic equipment as claimed in claim 26, described at least one expansion of wherein packet switch streaming service experience quality metric comprise the grammer that is used to report by the Coruption_Duration of the count measurement of impaired scene update.

31. electronic equipment as claimed in claim 26, described at least one expansion of wherein packet switch streaming service experience quality metric comprise the grammer on border between the group of scene of the rich-media content that receives that is used for deriving.

32. electronic equipment as claimed in claim 23, wherein said at least one quality of service measures expansion comprises described at least one expansion of passing through the RTP profile of expansion that is used for based on the RTCP feedback.

33. electronic equipment as claimed in claim 32 wherein is used for during the specific period that described at least one expansion based on the described RTP profile through expansion of RTCP feedback is included in the rich-media content that receives the indication at the packet loss of each priority.

34. electronic equipment as claimed in claim 32 wherein is used for being included in the nearest indication that has how many elements correctly not received the decode in the element list that enlivens based on described described at least one expansion of passing through the RTP profile of expansion of RTCP feedback.

35. electronic equipment as claimed in claim 32 wherein is used for not comprising the indicator element that correctly receives, decodes and enliven based on described described at least one expansion of passing through the RTP profile of expansion of RTCP feedback in current group of received data.

36. electronic equipment as claimed in claim 32 wherein is used for the indication of the impaired duration in the group that described at least one expansion based on the described RTP profile through expansion of RTCP feedback is included in received data.

37. electronic equipment as claimed in claim 32 wherein is used for comprising based on described described at least one expansion of passing through the RTP profile of expansion of RTCP feedback the indication of the group impaired owing to the losing of bag of reception contextual data.

38. electronic equipment as claimed in claim 23, wherein said rich-media content comprise dynamic and interactively medium scene (DIMS) dedicated content.

39. electronic equipment as claimed in claim 38, wherein said at least one quality of service measures extended description be losing at the described a large amount of real-time transport protocol (rtp) bags that receive each priority of rich-media content during specific period.

40. electronic equipment as claimed in claim 38, wherein said at least one quality of service measures extended description receives from described remote equipment, impaired DIMS scene.

41. electronic equipment as claimed in claim 38, wherein said at least one quality of service measures extended description receives from described remote equipment, impaired DIMS scene update.

42. electronic equipment as claimed in claim 38, that wherein said at least one quality of service measures extended description receives from described remote equipment, impaired DIMS scene and impaired DIMS scene update both.

43. electronic equipment as claimed in claim 38, wherein said at least one quality of service measures extended description receives from described remote equipment, duration of impaired DIMS scene and impaired DIMS scene update.

Images