WO2012094988A1

WO2012094988A1 - Method and device for transmitting streaming media data

Info

Publication number: WO2012094988A1
Application number: PCT/CN2012/070191
Authority: WO
Inventors: 孙健; 赵宇; 李加周; 陈光亮; 王芳
Original assignee: 中兴通讯股份有限公司
Priority date: 2011-01-11
Filing date: 2012-01-10
Publication date: 2012-07-19
Also published as: CN102594777A

Abstract

A method and device for transmitting streaming media data, wherein the method includes: a transmitting end encapsulates streaming media data into a packet and transmits the same, wherein the packet includes a packet header and a payload part, with the payload part including one or more access units (AU); wherein the transmitting end further transmits AU characteristic information about the packet when transmitting the packet, including the characteristic information about each AU in the packet and the length information about each AU; wherein the characteristic information about each AU is used for describing the attributes of the streaming media data borne thereby. The method can quickly realize personalized viewing and viewing with accurate positioning.

Description

Streaming media data transmission method and device

Technical field

The present invention relates to the field of streaming media data transmission technologies, and in particular, to a streaming media data transmission method and apparatus for identifying load data information in a multimedia real-time transmission process. Background technique

With the rapid development of communication technologies and the wide application of multimedia technologies, streaming media transmission technology for real-time transmission of multimedia content such as audio, video, subtitles or animation has emerged.

In a typical application system of streaming media, the streaming media server packs media content such as audio, video, subtitles or animation into multimedia data streams according to relevant transmission protocol specifications, and these multimedia data streams are continuously transmitted in real time on the network in the form of packet packets. . The client does not have to wait for the media content to be completely downloaded. It only takes a few seconds or ten seconds to decode the downloaded data and start playing. The media stream is played and transmitted until the media content is played or the client aborts. Applying these streaming media technologies can significantly reduce media playback latency and reduce client-side caching requirements.

However, the currently used streaming media protocol specifications are almost transparent to the transmission of content, which only provides a channel for streaming media transmission. Taking the Real-time Transport Protocol (RTP) as an example, the packet structure includes two parts, a protocol header and a payload part. The protocol header includes the payload media type, packet sequence number, timestamp, synchronization source identifier, and so on. The payload portion is typically a simple sequential storage of multimedia data. Another common streaming media transport protocol, such as the Moving Picture Experts G (MPEG) Transport Stream (TS, Transport Stream) packet, is the same for its packet structure. If there are multiple access units (AU, Access Unit) of media coding logic in a packet, and the coding characteristics of each AU are inconsistent with the attributes carrying data, the client wants to select specific data for playback or to accurately locate a packet. When an AU is in the package, the customer cannot obtain any auxiliary information from the packet header, and can only traverse all the load data. This process is often computationally intensive and inefficient, resulting in unnecessary computational complexity. Summary of the invention

The technical problem to be solved by the present invention is to provide a streaming media data transmission method and apparatus, which can quickly realize personalized viewing and precise positioning viewing.

In order to solve the above technical problems, the following technical solutions are used:

A method for transmitting streaming media data, where the sending end encapsulates the streaming media data into a packet, the packet includes a packet header and a payload portion, and the load portion includes one or more access units AU, and the method further includes :

And transmitting, by the sending end, the AU feature information of the packet, when the packet is sent, where the AU feature information of the packet is used to describe attributes of the streaming media data carried by the packet, including The feature information of each AU in the packet and the length information of each of the AUs. The AU feature information of the packet further includes:

The total length information of the AU feature information of the packet. The attribute that the AU feature information of the packet is used to describe includes: an encoding type of the streaming media data. The step of transmitting, by the sending end, the AU feature information of the packet includes: the sending end encapsulating the AU feature information of the packet in a packet header of the packet or encapsulating the payload of the packet In the section, and sent before the first AU. A method for receiving streaming media data, comprising:

After receiving the packet, the receiving end decodes the access unit AU feature information carried in the packet, that is, the AU feature information of the packet, and the AU feature information of the packet is used to describe the streaming media data carried by the packet. Attribute, including feature information of each AU in the packet, and length information of each AU;

Selecting the required AU according to the feature information of each of the AUs;

Calculating, according to length information of each of the AUs, an offset position of the required AU in the packet; The required AU is decoded from the packet according to the required offset position of the AU in the packet and the length information of the packet. The AU feature information of the packet further includes: total length information of the AU feature information of the packet;

The method further includes: when the receiving end needs to access all AUs in the packet, according to the total length information of the AU feature information of the packet, skipping the AU feature information of the packet, and directly accessing the AU feature information Each AU. The attribute that the AU feature information of the packet is used to describe includes: an encoding type of the streaming media data. Wherein: the AU feature information of the packet is encapsulated in a header of the packet or encapsulated in a payload portion of the packet, and is located before the AU.

The method includes: transmitting data according to any one of the sending methods as described above;

Data reception is performed in accordance with any of the receiving methods as described above. A device for transmitting streaming media data, comprising a first sending processing module and a second sending processing module, wherein:

The first sending processing module is configured to: package the streaming media data into a packet to be sent out; wherein, the packet includes a packet header and a payload portion, and the load portion includes one or more access units AU;

The second sending processing module is configured to: when the first sending processing module sends a packet, send the AU feature information of the packet; where the AU feature information of the packet is used to describe the carried The attribute of the media data includes feature information of each AU in the packet and length information of each of the AUs. The AU feature information of the packet further includes:

The total length information of the AU feature information of the packet. Wherein: the attributes of the AU feature information of the packet for describing are: The encoding type of streaming media data. Wherein: the second sending processing module is configured to send the AU feature information of the packet according to the following manner:

The AU feature information of the packet is encapsulated in a header of the packet or encapsulated in a payload portion of the packet, and is transmitted at a location before the first AU. A device for receiving streaming media data, comprising: a first parsing module, a selecting module, an operating module and a second parsing module, wherein:

The first parsing module is configured to: after receiving the packet, decode the access unit AU feature information carried in the packet, that is, the AU feature information of the packet, and the AU feature information of the packet is used to describe An attribute of the streaming media data that is carried, including feature information of each AU in the packet, and length information of each AU;

The selection module is configured to: select a required AU according to characteristic information of each AU;

The operation module is configured to: calculate, according to length information of each AU, an offset position of the required AU in the packet;

The second parsing module is configured to: decode the required AU from the packet according to the required offset position of the AU in the packet and the length information of the packet. The AU feature information of the packet further includes total length information of the AU feature information of the packet.

The second parsing module is further configured to: when all AUs in the packet are required to be accessed, skip the AU feature information of the packet according to the total length information of the AU feature information of the packet, and directly Analyze each AU. Wherein: the attributes of the AU feature information of the packet used for describing include:

The encoding type of streaming media data. Wherein: the first parsing module is configured to decode the AU feature information of the packet according to the following manner:

From the header of the packet or from the payload portion of the packet and before the AU The location resolves the feature information of the AU of the packet.

A streaming media data transmission system comprising: any one of the transmitting devices as described above and any one of the receiving devices as described above.

The above technical solution can quickly realize personalized viewing and precise positioning viewing. BRIEF abstract

1 is a flowchart of a method for transmitting streaming media data according to an embodiment of the present invention;

3 is a schematic diagram of encapsulating AU feature information in a payload portion of a packet according to an embodiment of the present invention;

4 is a flowchart of a method for receiving streaming media data according to an embodiment of the present invention;

FIG. 5 is a block diagram of a streaming media data transmitting apparatus according to an embodiment of the present invention; FIG.

6 is a block diagram of a streaming media data receiving apparatus according to an embodiment of the present invention;

7 is a flow chart of a method for personalized viewing of multi-view coding according to an application example of the present invention; FIG. 8 is a flow chart of a method for realizing accurate positioning of an AU according to an application example of the present invention.

Preferred embodiment of the invention

The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

When performing streaming media data transmission, the transmitting end encapsulates the streaming media data into a packet, and the packet includes a header and a payload portion, and the payload portion includes one or more access units AU. In order to enable the user to quickly locate a specific AU according to the user's needs, the embodiment of the present invention preferably provides a streaming media data sending method. As shown in FIG. 1, the method includes:

Step S101: The transmitting end further sends the AU feature information of the packet when transmitting the packet.

The AU feature information of the packet includes the feature information of each AU in the packet, and the length information of each AU. The feature information of the AU is used to describe the bearer thereof. The properties of streaming media data.

The feature information of each AU in the packet is provided for the user to quickly filter the required AU, and the length information of each AU is used for the offset calculation, so that the location of the required AU can be quickly located.

The feature information of the AU is related to the streaming media data carried in the AU, and is mainly used to distinguish the streaming media data carried in different AUs, and may be set as needed, for example, may be an encoding type of the streaming media data. Of course, other attributes of the streaming media, such as transmission priority, error flag, one or more of various hierarchical attributes, etc., may be used, and the present invention is not limited herein. Alternatively, the feature information of the AU may also be used to describe multiple attributes of the streaming media data carried by the AU, thereby further enriching the AU filtering condition, and is more suitable for personalized viewing.

For the sake of understanding, the embodiment of the present invention describes the coding type of the streaming media data carried by the AU as an example:

For the data of the adaptive multi-rate (AMR) coding type for the streaming media data carried in the AU, the feature information of the AU may be selected as the coding type (FT, Frame Type);

For the streaming media data carried in the AU is SVC (Scalable Video Coding) coding type data, the feature information of the AU may be selected as a network abstraction layer unit header ( NAL unit header, Network Abstraction Layer Unit header) NAL unit header SVC extension, Network Abstraction Layer Unit header SVC extension ;

For the streaming media data carried in the AU is multi-view video coding (MVC) coding type data, the feature information of the AU may be selected as a network abstraction layer unit header ( NAL unit header, Network Abstraction Layer Unit header) NAL unit header MVC extension, Network Abstraction Layer Unit header MVC extension .

For the data that is carried in the AU, the data is encapsulated by the TS multiplexing protocol, and the AU is a TS packet, and the feature information of the AU may be selected as a packet identifier (PID, packet identifier).

For other coding types of data, the AU's feature information can be selected according to similar principles. This is not limited.

Preferably, the AU feature information of the packet further includes: total length information of the AU feature information of the packet. It provides the total byte size occupied by the AU feature information within the packet payload. When the client wants to access all the AUs in the packet, this field can be used to skip the AU feature information and directly access the AU data. Thereby reducing computational complexity and increasing access speed.

Preferably, as shown in FIG. 2, the transmitting end may encapsulate the AU feature information of the packet in a header of the packet, for example, for RTP and MPEG-TS. The field extension is performed by the RTP header (defined according to the RTP extension standard RTP Header Extension) and the TS header (defined according to the TS header extension standard transport_private_data), and the AU feature information can be carried in the extension field. Alternatively, as shown in FIG. 3, the transmitting end may also encapsulate the AU feature information of the packet in a payload portion of the packet and at a position before the first AU. The above describes several specific ways of carrying AU feature information in the band. Other methods can also be used, and no limitation is imposed here.

Correspondingly, after receiving the packet, the receiving end (media forwarding server or client) may first parse the feature information of the AU, and then adapt and select the AU in the load according to the established policy. The specific selection policy does not belong to the content of the present invention, and details are not described herein again. The processing on the receiving side is as shown in Figure 4, including the steps:

Step S401: Decoding the access unit AU feature information carried in the packet, the AU feature information includes feature information of each AU in the packet, and length information of each AU; wherein, the feature of the AU Information is used to describe the attributes of the streaming media data it carries;

Step S402: Select a required AU according to the feature information of each AU;

Step S403: Calculate, according to the length information of each AU, an offset position of the required AU in the packet;

Step S404: Decode the required AU from the packet according to the offset position of the required AU in the packet and its length information.

With the solution of the embodiment of the present invention, the receiving end quickly filters the required AU according to the feature information of each AU in the packet, and then performs offset calculation according to the length information of each AU, so that the receiving end can be quickly determined. Positioning the desired AU enables personalized viewing to be achieved quickly and efficiently. The feature information of the AU is related to the streaming media data carried in the AU, and is mainly used to distinguish the streaming media data carried in different AUs, and may be set as needed, as described above, and details are not described herein again.

The feature information of the AU may be encapsulated in a packet header of the packet, or may be encapsulated in a payload portion of the packet and located before the AU.

Preferably, when the AU feature information of the packet further includes the total length information of the AU feature information of the packet, the receiving end may skip the AU feature by using the field when it is desired to access all the AUs in the packet. Information, direct access to AU data. This reduces computational complexity and speeds up access.

The embodiment of the present invention further provides a streaming media data transmission method. The transmitting end uses the sending method as described above, and also sends the AU feature information of the packet when transmitting the packet, and uses the receiving method as described above at the receiving end. According to the AU feature information of the packet, the required AU is filtered, and the specific location of the AU is accurately located, thereby realizing personalized playback of the specific AU.

In order to implement the foregoing sending method, the embodiment of the present invention further provides a streaming media data sending device, as shown in FIG. 5, including a first sending processing module and a second sending processing module, where:

The first sending processing module is configured to: package the streaming media data into a packet, the packet includes a packet header and a payload portion, and the load portion includes one or more access units AU;

The second sending processing module is configured to: when the first sending processing module sends a packet, send the AU feature information of the packet.

The AU feature information of the packet includes the feature information of each AU in the packet, and length information of each AU;

The feature information of the AU is used to describe attributes of the streaming media data that it carries.

Preferably, the AU feature information of the packet may further include: The total length information of the AU feature information of the packet.

The second sending processing module may encapsulate the AU feature information of the packet in a packet header of the packet, or may be encapsulated in a payload portion of the packet and located before the first AU. . Of course, it can also be sent in other ways, and the invention is not limited herein.

In order to implement the foregoing receiving method, the embodiment of the present invention further provides a receiving device, as shown in FIG. 6, including a first parsing module, a selecting module, an computing module, and a second parsing module, where: the first parsing module is configured After receiving the packet, decoding the access unit AU feature information carried in the packet, the AU feature information includes feature information of each AU in the packet, and length information of each AU; The feature information of the AU is used to describe an attribute of the streaming media data that it carries;

The operation module is configured to: calculate an offset position of the required AU in the packet according to length information of each AU;

The second parsing module is configured to: decode the required AU from the packet according to the offset position of the required AU in the packet and its length information.

Preferably, the AU feature information of the packet further includes total length information of the AU feature information of the packet. Correspondingly, the second parsing module may further skip the AU feature information and directly parse each AU according to the total length information of the AU feature information when all AUs in the packet are required to be accessed.

The feature information of the AU is related to the streaming media data carried in the AU, and is mainly used to distinguish the streaming media data carried in different AUs, and may be set as needed, for example, The encoding type for streaming media data. Of course, it may be other attributes of the streaming media, such as one or more of transmission priority, error flag, various hierarchical attributes, and the like, and the present invention is not limited herein. Alternatively, the feature information of the AU may also be used to describe multiple attributes of the streaming media data carried by the AU, thereby further enriching the AU screening condition, and is more suitable for personalized viewing.

The first parsing module may be configured to extract feature information of the AU from a header of the packet, or may be a location from a payload portion of the packet and located before the AU. The specific encapsulation location is determined by the transmitting end. When the transmitting end uses other encapsulation modes, the location of the parsing of the feature information of the AU changes accordingly. Therefore, the present invention is not limited herein.

Embodiments of the present invention also provide a system including the transmitting device and the receiving device as described above.

For further explanation, the present invention also provides the following application examples.

Application example 1:

This application example is to realize personalized viewing of multi-view coding by using the technical solution of the present invention. Assume that the MVC stream contains both viewl and view2 data. They use the view_id value of the NAL unit header MVC extension to distinguish between 1 and 2. And the transport stream packet consists of N AUs, denoted as AU1, AU2, AUn.

If the client wants to view the data of the viewl perspective, the server can select the AU in the transport stream packet, as shown in FIG. 7, the specific steps include:

Step 701: Parse the feature information of the AU.

Parsing the feature information of all AUs in the packet, and obtaining the view_id value in the NAL unit header MVC extension;

Step 702: Select a suitable AU;

Select all AUs whose view-id field is 1 and assume that there are two AU view-id fields with values of 1 , which are AU1 and AUi respectively.

Step 703: Parse all AU lengths;

Step 704: Calculate an offset of the selected AU; Based on the length of all AUs in the packet and the length of the selected AU, the offset of the selected AU in the packet can be calculated. The offset of AU1 is the size of the packet header plus the size of all the feature information of the AU. The offset of AUi is the size of the packet header plus the size of all the feature information of the AU, plus the sum of the lengths of AUI, AU2, and AUi-1.

Step 705: Obtain all selected AUs.

According to the offset and length of the selected AU, all the packets of the viewl perspective can be selected from the packet, and the client can view the program from the viewl perspective.

Application example 2:

This application example is to achieve precise positioning of the AU by using the technical solution of the present invention. Assume that the transport stream packet consists of N AUs, denoted AUI, AU2, AUn. With some strategy, the client needs to pinpoint the i-th AU of the packet. As shown in Figure 8, the specific steps include:

Step 801: Select a suitable AU;

According to the policy, select AU as AUi;

Step 802: Parse the length of all AUs;

Step 803: Calculate an offset of the selected AU;

Based on the length of all AUs in the packet and the length of the selected AU, the offset of AUi in the packet can be calculated. The offset of AUi is the size of the packet header plus the size of all the feature information of the AU, plus the sum of the lengths of AUI, AU2, and AUi-1.

Step 804: Get all selected AUs.

According to the offset of AUi, it can be accurately positioned to AUi, which meets the customer's precise positioning requirements.

The streaming media transmission method and system provided by the invention can select and accurately locate the AU in the packet in the media forwarding device and the client by using the information in the media stream, and is applicable to the streaming media on-demand and live broadcast scenarios, which can effectively improve the calculation. effectiveness.

The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention, and various modifications and changes can be made to the present invention. Where is the spirit and principle of the present invention Any modifications, equivalent substitutions, improvements, etc. made therein are intended to be included within the scope of the present invention.

Industrial applicability

The above technical solution can quickly realize personalized viewing and precise positioning viewing. Therefore, the present invention has strong industrial applicability.

Claims

Claim

A method for transmitting streaming media data, where a sending end encapsulates streaming media data into a packet, the packet includes a header and a payload, and the payload includes one or more access units AU, the method Also includes:

And transmitting, by the sending end, the AU feature information of the packet, when the packet is sent, where the AU feature information of the packet is used to describe attributes of the streaming media data carried by the packet, including The feature information of each AU in the packet and the length information of each of the AUs.

2. The transmitting method according to claim 1, wherein the AU feature information of the packet further comprises:

The total length information of the AU feature information of the packet.

3. The transmitting method according to claim 1, wherein:

The attributes for describing the AU feature information of the packet include: The encoding type of the streaming media data.

4. The transmitting method according to claim 1, wherein: the transmitting end transmitting the AU feature information of the packet includes:

The transmitting end encapsulates the AU feature information of the packet in a header of the packet or encapsulated in a payload portion of the packet, and is transmitted at a position before the first AU.

5. A method for receiving streaming media data, comprising:

Calculating, according to length information of each of the AUs, an offset position of the required AU in the packet; The required AU is decoded from the packet according to the required offset position of the AU in the packet and the length information of the packet.

6. The receiving method according to claim 5, wherein:

The AU feature information of the packet further includes: total length information of the AU feature information of the packet;

The method further includes: when the receiving end needs to access all AUs in the packet, according to the total length information of the AU feature information of the packet, skipping the AU feature information of the packet, and directly accessing the AU feature information Each AU.

7. The receiving method according to claim 5, wherein:

8. The receiving method according to claim 5, wherein:

The AU feature information of the packet is encapsulated in the header of the packet or encapsulated in the payload portion of the packet, and is located before the AU.

9. A streaming media data transmission method, comprising:

Data transmission according to the transmission method according to any one of claims 1 to 4;

Data reception is performed according to the receiving method according to any one of claims 5-8.

A device for transmitting streaming media data, comprising: a first sending processing module and a second sending processing module, wherein: the first sending processing module is configured to: package the streaming media data into a packet to send out; The packet includes a header and a payload portion, and the load portion includes one or more access units AU;

The second sending processing module is configured to: when the first sending processing module sends a packet, send the AU feature information of the packet; where the AU feature information of the packet is used to describe the carried The attribute of the media data includes feature information of each AU in the packet and length information of each of the AUs.

The transmitting device according to claim 10, wherein the AU feature information of the packet further comprises:

The total length information of the AU feature information of the packet.

12. The transmitting device according to claim 10, wherein: the AU feature information of the packet is used to describe attributes including:

The encoding type of streaming media data.

13. The transmitting apparatus according to claim 10, wherein: the second transmission processing module is configured to transmit AU feature information of the packet in the following manner:

The AU feature information of the packet is encapsulated in a header of the packet or encapsulated in a payload portion of the packet, and is transmitted at a location before the first AU.

14. A device for receiving streaming media data, comprising: a first parsing module, a selecting module, an arithmetic module, and a second parsing module, wherein:

The second parsing module is configured to: decode the required AU from the packet according to the required offset position of the AU in the packet and the length information of the packet.

15. The receiving device of claim 14, wherein:

The AU feature information of the packet further includes total length information of the AU feature information of the packet,

The second parsing module is further configured to: when all AUs in the packet are required to be accessed, skip the AU feature of the packet according to the total length information of the AU feature information of the packet Information, directly parsing each AU in turn.

The receiving device according to claim 14, wherein: the AU feature information of the packet is used to describe attributes including:

The encoding type of streaming media data.

17. The receiving device of claim 14, wherein: the first parsing module is configured to decode AU feature information of the packet as follows:

The feature information of the AU of the packet is parsed from the header of the packet or from the payload portion of the packet and before the AU.

A streaming media data transmission system, comprising: the transmitting device according to any one of claims 10-13, and the receiving device according to any one of claims 14-17.