Background technology
Time control text (Timed Text) is that a kind of permission is modified text, can be used for the textual form that captions, Karaoke, rolling news, hypertext link etc. are used.For example, the time control text can be associated with content of multimedia such as audio frequency, videos, can be in video content videotex information synchronously or non-synchronously.In video media, the correlating method between time control text and the video media is following:
As shown in Figure 1, in a viewing area (Display Track), (tx ty) is the coordinate that is associated with the starting point A in video area (Video Track) upper left corner to the B position coordinates in text area (Text Track) upper left corner.Be that tx is the horizontal-shift of text area with respect to the coordinate of video area starting point A, ty is the vertical shift of text area with respect to the coordinate of video area initial point A.Suppose that the text display district is positioned at the position of video viewing area bottom central, be of a size of 200*20, the video viewing area is of a size of 320*240.Tx=(320-200)/2=60 then, ty=240-(20/2)=230.Being concrete parameters such as the display position of time control text in video content, is to depend on the video parameter related with it.
In a Multimedia session, the both sides of session need accomplish media negotiation in session foundation or the process of upgrading, thus parameters such as the concrete media formats that definite both sides use, transmit port, for media flow transmission is afterwards got ready.
Offer/Answer (providing/reply) mode to use SDP is example, and concrete negotiations process is as shown in Figure 2;
At first the Offer end sends SDP Offer message to the Answer end, carries the concrete media parameter that the Offer end is supported therein, and afterwards, the Answer end returns SDP Answer message, carries the concrete media parameter that the Answer end is supported.Wherein, SDP Offer message can be by SIP or is H.323 waited message bearings such as protocol of I NVITE, ACK.After the exchange through SDP Offer message and SDP Answer message, communicating pair is recognized the information such as medium type, coded format, address and port that the other side can support, just can communicate through the medium connection according to these information afterwards.
In the SDP agreement, be exemplified below for the descriptor format of media parameter:
m=audio?3458?RTP/AVP?0?96?97?98
a=rtpmap:0?PCMU
a=rtpmap:96?G726-32/8000
a=rtpmap:97?AMR-WB
a=rtpmap:98?telephone-event
a=mid:1
m=video?3400?RTP/AVP?99?101
a=rtpmap:99?MPV
a=rtpmap:101?H.261
a=mid:2
m=video?3456?RTP/AVP?103?121
a=sendonly
a=rtpmap:103?H.261
a=rtpmap:121?MPV
a=mid:3
This example SDP message semantic definition three Media Streams, first Media Stream is an audio stream, uses the RTP/AVP protocol transmission, transmit port is UDP 3458, optional audio format be PCMU, G.726, AMR-WB or call voice; Second Media Stream is video flowing, uses the RTP/AVP protocol transmission, and transmit port is UDP 3400, and optional video format is MPV or H.261; H.261 or MPV the 3rd Media Stream is video flowing, uses the RTP/AVP protocol transmission, and transmit port is UDP 3456, and optional video format is for; And the transmission of the video flowing that the UDP 3456 that this video flowing uses only is used for does not receive the Media Stream that the other side sends.Wherein a=mid:1 presentation medium traffic identifier is 1, by that analogy.
In the SDP agreement, be similar to the description of the description of time control text and above Media Stream, it is following that it specifically describes example:
m=video?4567?RTP/AVP?98
a=rtpmap:98?3gpp-tt/1000
a=fmtp:98?tx=100;ty=100;layer=0;height=80;width=100;max-h=120;max-w=160;sver=6256,60;tx3g=81...
a=mid:2
In this example, the text display district of time control text is (100,100) with respect to the coordinate of video viewing area; Show dimensions as 100 * 80; Maximum shows dimensions as 160 * 120, and the default format of text is the value among the parametric t x3g, and the traffic identifier of this time control text flow is 2.
Because space position parameter tx, the ty of time control text are relative values; This value is with respect to video flowing, and the display position of concrete time control text depends on the display position of video flowing, therefore if think the display position of clear and definite time control text; Which video flowing just must clear and definite time control text pairing be; In the description of SDP agreement to video flowing, if having only a video flowing in the session, the pairing video flowing of time control text is confirmed so; But when having a plurality of video flowing in the session, which video flowing will be prior art can't be associated with by clear and definite time control text.
Situation for example shown in Figure 3 includes two video flowings in the session.One is main video flowing, like the key frame of news hookup; Another is the video flowing of picture-in-picture; Like the sign language interpreter picture of news, also have a time control text flow, be the captions of news.
So, the existing example of using SDP that these three kinds of Media Streams are described is following:
m=video?52886?RTP/AVP?31
a=rtpmap:31?H261/9000
a=mid:1
m=video?53334?RTP/AVP?31
a=rtpmap:31?H261/9000
a=mid:2
m=video?54444?RTP/AVP?98
a=rtpmap:98?3gpp-tt/1000
a=fmtp:98?tx=10;ty=100;layer=0;height=80;width=100;max-h=120;
max-w=160;sver=6256,60;tx3g=81...
a=mid:3
From above description, can find out; In existing media consultation process, in the description to the time control text flow, there are not the specific descriptions of the corresponding relation of time control text flow and video flowing; Therefore the display position of time control text flow is uncertain; Like this, when in the session two or more video flowing being arranged, use existing media consulation method; The both sides of media negotiation message through consultation confirm the corresponding relation of time control text flow and video flowing, so may cause the display position mistake of time control text.
Summary of the invention
The embodiment of the invention provides a kind of media consulation method, can clear and definite time control text and the corresponding relation of video flowing, thus avoid the display position mistake of time control text.
The embodiment of the invention provides a kind of media negotiation system, can clear and definite time control text and the corresponding relation of video flowing, thus avoid the display position mistake of time control text.
The embodiment of the invention provides a kind of end device that provides, can clear and definite time control text and the corresponding relation of video flowing, thus avoid the display position mistake of time control text.
The embodiment of the invention provides a kind of responder device, can clear and definite time control text and the corresponding relation of video flowing, thus avoid the display position mistake of time control text.
For achieving the above object, technical scheme of the present invention specifically is achieved in that
A kind of media consulation method, this method comprises:
The description of in the media negotiation request message that sends, carrying the corresponding relation of time control text flow and video flowing;
Reception indicates whether to support the response message of said corresponding relation.
A kind of media negotiation system, this system comprises:
End device is provided, is used to send the media negotiation request message, wherein carry the description of the corresponding relation of time control text flow and video flowing;
The responder device is used to send response message, indicates whether to support the corresponding relation of said time control text flow and video flowing.
A kind of end device that provides, this device comprises:
Time control text corresponding relation generation module is used for the corresponding relation of time control text flow and video flowing is added into the media negotiation request message;
Time control text corresponding relation sending module is used to receive that the corresponding relation generation module sends, and comprises the media negotiation request message of time control text flow and video flowing corresponding relation, and sends this message.
A kind of responder device, this device comprises:
Time control text corresponding relation receiver module, being used to receive provides end to send, and comprises the message that is used for the media negotiation request of the corresponding relation of time control text flow and video flowing;
The negotiation confirmation module; Be used to receive that time control text corresponding relation receiver module sends; The message that is used for the media negotiation request that comprises the corresponding relation of time control text flow and video flowing; And in consulting response message, indicate whether time control text flow and the corresponding relation of video flowing in the support media agreement request, and send this message.
Visible by above-mentioned technical scheme; The embodiment of the invention is added into the message that is used for media negotiation with the corresponding relation of time control text and video flowing, and the corresponding relation according to time control text and video flowing in the said message carries out media negotiation; Corresponding relation through clear and definite time control text and video flowing; Thereby the display position of clear and definite time control text has been avoided when containing a plurality of video flowing in the session, and the situation that possibly make a mistake in time control text display position takes place.
Embodiment
For making the object of the invention, technical scheme and advantage clearer, below with reference to the accompanying drawing embodiment that develops simultaneously, to further explain of the present invention.
The embodiment of the invention mainly is that the corresponding relation with time control text and video flowing is added into the message that is used for media negotiation; Corresponding relation according to time control text and video flowing in the said message; Carry out media negotiation; Thereby the display position of clear and definite time control text, when having avoided containing in the session a plurality of video flowing, the situation that possibly make a mistake in time control text display position takes place.
Carry out multimedia negotiation with the Offer/Answer mode of using the SDP agreement and be example, Fig. 4 is the Multimedia session machinery of consultation flow chart of the embodiment of the invention, and is as shown in the figure;
Step 401 is carried the corresponding relation of time control text flow and video flowing in the media negotiation request message that sends description.
Step 402 receives the response message that indicates whether to support said corresponding relation.
Wherein, the concrete grammar of description that in the media negotiation request message that sends, carries the corresponding relation of time control text flow and video flowing in the step 401 can be:
According to the said media description grammer that is used for the message of media negotiation request; Increase the media-level attributes row of the corresponding relation be used to describe time control text flow and video flowing in the message that is used for the media negotiation request, and with the traffic identifier of the pairing video flowing of the time control text flow value as said property column.
Specifically, stipulate in the SDP agreement that the grammer of media-level attributes row is: a=< attribute >: < value >, wherein < attribute>representation attribute is capable, and < value>representes the value of this property column.Particular content for a line description of describing the Media Stream specific object can not realized by agreement, so the concrete character set of a line description is selected and can not retrained by the SDP standard; According to this characteristic, can be according to this grammer at property column of the capable increase of a: a=attribute_name:TT mid, in order to the corresponding relation of expression time control text and video flowing.Wherein attribute_name is the title of property column, can use in any agreement still untapped field to describe, such as depend.TT mid is the value of this property column, and this value is the corresponding video flowing sign of time control text.
The above-described message that is used for the media negotiation request can be SDP Offer message, and specifically, SDP Offer message can use the INVITE of a Session Initiation Protocol to carry, and also can use other message bearings, looks concrete bearing protocol difference and difference.
In step 401, said new syntax is added back in the agreement of said session negotiation the description of Media Stream is exemplified below:
m=video?52886?RTP/AVP?31
a=rtpmap:31?H261/9000
a=mid:1
m=video?53334?RTP/AVP?31
a=rtpmap:31?H261/9000
a=mid:2
m=video?54444?RTP/AVP?98
a=rtpmap:98?3gpp-tt/1000
a=fmtp:98?tx=60;ty=220;layer=0;height=20;width=200;max-h=120;
max-w=160;sver=6256,60;tx3g=81...
a=mid:3
a=depend:TT?1
Wherein a=depend:TT 1 is exactly the description to the corresponding relation of time control text flow and video flowing; Explained that the time control text flow is associated with Media Stream 1; Like this; As long as the both sides of media session can both discern this property column, so just can confirm the correct display position of time control text, accomplish media negotiation.
Take an example that in media session, contains a plurality of video flowings and a plurality of time control text flows again; Two video flowings are for example arranged: video flowing 1 and video flowing 2; And two time control text flows are arranged: time control text flow 1 and time control text flow 2 are associated with video flowing 1 and video flowing 2 respectively; So concrete SDP is described below:
m=video?52886?RTP/AVP?31
a=rtpmap:31?H261/9000
a=mid:1
m=video?53334?RTP/AVP31
a=rtpmap:31?H261/9000
a=mid:2
m=video?54444?RTP/AVP?98
a=rtpmap:98?3gpp-tt/1000
a=fmtp:98?tx=60;ty=220;layer=0;height=20;width=200;max-h=120;
max-w=160;sver=6256,60;tx3g=81...
a=mid:3
a=depend:TT?1
m=video?54446?RTP/AVP?98
a=rtpmap:98?3gpp-tt/1000
a=fmtp:98?tx=20;ty=40;layer=0;height=20;width=200;max-h=120;
max-w=160;sver=6256,60;tx3g=81...
a=mid:4
a=depend:TT?2
Through above description,, the relation of time control text flow and video flowing can be described out clearly no matter relate to what video flowings and time control text flow in the media negotiation.
More than two examples be at the capable media-level attributes row that increases the corresponding relation that is used to describe time control text flow and video flowing of the medium property that is used for describing the time control text flow; Can certainly be at the capable media-level attributes row that increases the corresponding relation that is used to describe video flowing and time control text flow of the medium property that is used for describing video flowing, for example:
m=video?52886?RTP/AVP?31
a=rtpmap:31?H261/9000
a=mid:1
a=depend:TT?3
m=video?53334?RTP/AVP?31
a=rtpmap:31?H261/9000
a=mid:2
m=video?54444?RTP/AVP?98
a=rtpmap:98?3gpp-tt/1000
a=fmtp:98?tx=60;ty=220;layer=0;height=20;width=200;max-h=120;
max-w=160;sver=6256,60;tx3g=81...
a=mid:3
In addition, except new media-level attributes row of direct increase, also can realize through the parameter of expansion existing session level property column; Such as; According to the said media description grammer that is used for the message of media negotiation request, will be used for the session level property column of the message of media negotiation, increase a new parameter like the a=group of group property column; And with the traffic identifier of the corresponding video flowing of the traffic identifier of time control text flow and this time control text flow, as the value of the new parameter of said increase.
Specifically; In existing SDP agreement; The grammars of a=group property column is following: a=group:semantics (space identification-tag), and wherein semantics can be LS or FID, identific ation-tag is the Media Stream sign.
The a=group property column is represented a plurality of Media Streams in the Multimedia session are divided into groups, and 12 expressions are carried out Media Stream 1 and Media Stream 2 synchronously like a=group:LS.
So, according to this grammer, we increase a parameter TT can for the a=group property column; Detailed directions is following:
A=group:TT mid1 mid2, wherein mid1 identifies a video flowing, and mid2 identifies a time control text flow, and the video flowing of the capable parameter of the m of the time control text flow that mid2 identifies corresponding to the mid1 sign represented in such description.
In addition; Above-mentioned expansion to a=group can also represent that a plurality of time control text flows all are associated with same video flowing, that is: a=group:TT mid1 mid2 mid3..., and wherein the m of a video flowing of mid1 sign is capable; Thereafter all mid; Like mid2, mid3..., the m that all identifies the time control text flow is capable, and the number of time control text flow sign mid can be one or more.A=group:TT mid1 mid2mid3... representes that the capable parameter of the m of wherein all time control text flow sign time control text flows that mid2, mid3... identified is related with the video flowing of mid1 sign.
Therefore, if a plurality of time control text flow all is associated with same video flowing, such as time control text flow mid2, mid3 is associated with video flowing mid1, so, both can use two a=group line displays, also can be with a=group capable expression; For example:
a=group:TT?mid1?mid2
a=group:TT?mid1?mid3
Perhaps
a=group:TT?mid1?mid2?mid3
Illustrate the capable concrete method for using of a=group:TT below:
1, carry out related with a video flowing time control text flow.
a=group:TT?13
m=video?52886?RTP/AVP?31
a=rtpmap:31?H261/9000
a=mid:1
m=video?53334?RTP/AVP?31
a=rtpmap:31?H261/9000
a=mid:2
m=video?54444?RTP/AVP?98
a=rtpmap:98?3gpp-tt/1000
a=fmtp:98?tx=60;ty=220;layer=0;height=20;width=200;max-h=120;
max-w=160;sver=6256,60;tx3g=81...
a=mid:3
2, carry out related with same video flowing two time control text flows
a=group:TT?12
a=group:TT?13
m=video?52886?RTP/AVP?31
a=rtpmap:31?H261/9000
a=mid:1
m=video?54442?RTP/AVP?98
a=rtpmap:98?3gpp-tt/1000
a=fmtp:98?tx=10;ty=20;layer=0;height=20;width=200;max-h=120;
max-w=160;sver=6256,60;tx3g=81...
a=mid:2
m=video?54444?RTP/AVP?98
a=rtpmap:98?3gpp-tt/1000
a=fmtp:98?tx=20;ty=40;layer=1;height=20;width=200;max-h=120;
max-w=160;sver=6256,60;tx3g=81...
a=mid:3
Its SDP describe can also for
a=group:TT?123
m=video?52886?RTP/AVP?31
a=rtpmap:31?H261/9000
a=mid:1
m=video?54442?RTP/AVP?98
a=rtpmap:98?3gpp-tt/1000
a=fmtp:98?tx=10;ty=20;layer=0;height=20;width=200;max-h=120;
max-w=160;sver=6256,60;tx3g=81...
a=mid:2
m=video?54444?RTP/AVP?98
a=rtpmap:98?3gpp-tt/1000
a=fmtp:98?tx=20;ty=40;layer=1;height=20;width=200;max-h=120;
max-w=160;sver=6256,60;tx3g=81...
a=mid:3
Introduced the object lesson of the corresponding relation description of adding time control text flow and video flowing in the SDP Offer message above; Receive the SDP Offer message of the corresponding relation description that comprises time control text flow and video flowing at responder after; Need confirm the description of the Media Stream in the message; Confirm whether to support the corresponding relation described in this message, and return acknowledge message, be i.e. SDP Answer message.
If the responder of media negotiation is supported said corresponding relation, then in the SDP Answer message of returning, carry and the identical description of said corresponding relation that receives; If the responder of media negotiation is not supported said corresponding relation, the description of then in the SDP Answer message of returning, not carrying said corresponding relation gets final product.In addition, order that a plurality of m that carry in the SDP Answer message that responder returns are capable and mid value also must be in full accord with order and mid value in the SDP Offer message.In simple terms as long as the description of the corresponding relation of time control text flow in the SDP Offer message that guarantees and receive and video flowing is identical.
For example, in SDP Offer message, comprise the description of following corresponding relation to Media Stream and time control text flow and video flowing:
m=video?52886?RTP/AVP?31
a=rtpmap:31?H261/9000
a=mid:1
m=video?53334?RTP/AVP?31
a=rtpmap:31?H261/9000
a=mid:2
m=video?54444?RTP/AVP?98
a=rtpmap:98?3gpp-tt/1000
a=fmtp:98?tx=60;ty=220;layer=0;height=20;width=200;max-h=120;
max-w=160;sver=6256,60;tx3g=81...
a=mid:3
a=depend:TT?1
m=video?54446?RTP/AVP?98
a=rtpmap:98?3gpp-tt/1000
a=fmtp:98?tx=20;ty=40;layer=0;height=20;width=200;max-h=120;
max-w=160;sver=6256,60;tx3g=81...
a=mid:4
a=depend:TT?2
If responder is supported the Media Stream of above description and the corresponding relation of time control text flow and video flowing fully, then in the SDP Answer message of returning, will carry and the identical description of above description.If do not support the 4th Media Stream, promptly second time control text flow then deleted corresponding description in the SDP Answer message of returning, and do not carry this description and gets final product, and will carry following description in the SDP Answer message that return to this moment:
m=video?52886?RTP/AVP?31
a=rtpmap:31?H261/9000
a=mid:1
m=video?53334?RTP/AVP?31
a=rtpmap:31?H261/9000
a=mid:2
m=video?54444?RTP/AVP?98
a=rtpmap:98?3gpp-tt/1000
a=fmtp:98?tx=60;ty=220;layer=0;height=20;width=200;max-h=120;
max-w=160;sver=6256,60;tx3g=81...
a=mid:3
a=depend:TT?1
The difference of this SDP Answer message and the SDP Offer message that receives just be to lack last time control text flow with and with the description of the corresponding relation of video flowing 2.
Introduce the media consulation method of the embodiment of the invention above, will introduce the media negotiation system of this method of application below in detail.
Fig. 5 is the media negotiation system construction drawing of the embodiment of the invention, and comprising provides end device 501 and responder device 502.
End device 501 is provided, is used to send the media negotiation request message, wherein carry the description of the corresponding relation of time control text flow and video flowing.
Responder device 502 is used to send response message, indicates whether to support the corresponding relation of said time control text flow and video flowing.
Wherein, provide end device 501 to comprise: time control text corresponding relation generation module 503 and control text corresponding relation sending module 504.
Time control text corresponding relation generation module 503 is used for the corresponding relation of time control text flow and video flowing is added into the message that is used for the media negotiation request;
Time control text corresponding relation sending module 504 is used to receive that corresponding relation generation module 503 sends, and comprises the message that is used for the media negotiation request of time control text flow and video flowing corresponding relation, and sends this message.
Said time control text corresponding relation generation module 503; Specifically can be according to concrete property column grammer; In said media negotiation request message, increase the property column of the corresponding relation be used to describe time control text flow and video flowing, such as increase media-level attributes row, perhaps session level property column.
Preferably, comprise media-level attributes row generation unit 507 and session level property column generation unit 508 in the corresponding relation generation module 503.
Media-level attributes row generation unit 507; Be used for increasing the media-level attributes row of the corresponding relation that is used to describe time control text flow and video flowing, and in said media-level attributes row, indicate the corresponding relation of time control text flow and video flowing at said media negotiation request message.
Session level property column generation unit 508; Be used for increasing the session level property column of the corresponding relation that is used to describe time control text flow and video flowing, and in said session level property column, indicate the corresponding relation of time control text flow and video flowing at said media negotiation request message.
Wherein, also comprise in the media-level attributes row generation unit 507: time control text flow property column generates subelement 509 and generates subelement 510 with the video flowing property column.
Time control text flow property column generates subelement 509, and being used for increases the media-level attributes row that is used to describe the time control text flow at said media negotiation request message, comprises the traffic identifier of the pairing video flowing of time control text flow in the said property column.
The video flowing property column generates subelement 510, and being used for increases the media-level attributes row that is used to describe video flowing at said media negotiation request message, comprises the traffic identifier of the pairing time control text flow of video flowing in the said property column.
Wherein, responder device 502 comprises: time control text corresponding relation receiver module 505 and negotiation confirmation module 506.
Time control text corresponding relation receiver module 505, being used to receive provides end device 501 to send, and comprises the message that is used for the media negotiation request of the corresponding relation of time control text flow and video flowing.
Negotiation confirmation module 506; Be used to receive that time control text corresponding relation receiver module 505 sends; The message that is used for the media negotiation request that comprises the corresponding relation of time control text flow and video flowing; And in consulting response message, indicate whether time control text flow and the corresponding relation of video flowing in the support media agreement request, and send this message.
Wherein, comprise decision unit 511 in the negotiation confirmation module 506.
Decision unit 511 is used for judging responder whether the time control text flow of support media agreement request and the corresponding relation of video flowing.
If the time control text flow in the responder support media agreement request and the corresponding relation of video flowing then add said corresponding relation in the media negotiation response message of returning to.
If responder is the time control text flow in the support media agreement request and the corresponding relation of video flowing not, then in the media negotiation response message of returning, do not carry said corresponding relation.
Visible by the above embodiments; The embodiment of the invention is added into the message that is used for media negotiation with the corresponding relation of time control text and video flowing, and the corresponding relation according to time control text and video flowing in the said message carries out media negotiation; Corresponding relation through clear and definite time control text and video flowing; Thereby the display position of clear and definite time control text has been avoided when containing a plurality of video flowing in the session, and the situation that possibly make a mistake in time control text display position takes place.
Institute is understood that; The above is merely preferred embodiments of the present invention, and is not used in qualification protection scope of the present invention, and is all within spirit of the present invention and principle; Any modification of being made, be equal to replacement, improvement etc., all should be included within protection scope of the present invention.