WO2012094980A1

WO2012094980A1 - Method and system for transmitting streaming media, and media forwarding server

Info

Publication number: WO2012094980A1
Application number: PCT/CN2012/070174
Authority: WO
Inventors: 陈光亮; 王芳; 刘继年; 孙健
Original assignee: 中兴通讯股份有限公司
Priority date: 2011-01-11
Filing date: 2012-01-10
Publication date: 2012-07-19
Also published as: CN102594775A

Abstract

The present invention relates to a method and system for transmitting streaming media, and a media forwarding server. The method includes: a streaming media server sends a media stream carrying media packet size adaption information; a media forwarding server receives the media stream, and performs packet size adaption processing on the media stream according to the media packet size adaption information in the received media stream; and the media forwarding server forwards the packet size adaption processed media stream. The method and the system for transmitting the streaming media in the present invention can improve the transmission efficiency of the network.

Description

Streaming media transmission method, system and media forwarding server

Technical field

The present invention relates to the field of multimedia data transmission technologies, and in particular, to a multimedia data transmission method, system and media forwarding server for packet size adaptation in a multimedia real-time transmission process.

Background technique

With the popularization of network technologies and the wide application of multimedia technologies, streaming media transmission technology for real-time transmission of multimedia content such as audio, video, subtitles or animation has emerged.

The streaming media server packs media content such as audio, video, subtitles or animation into a media stream, and the media stream is continuously transmitted in real time on the network in groups. The client does not have to wait for the media content to be completely downloaded, and only needs to delay a few seconds or ten. Playback can be started in a few seconds, while the media stream is played and played until the media content is played or the client aborts. Application of streaming media transmission technology can significantly shorten media playback latency and reduce client cache requirements.

Streaming media is facing new application scenarios across networks or heterogeneous networks. The same media streams need to support services on different networks. In order to improve the end user experience, it is necessary to improve the transmission quality of the media stream. In the process of media stream transmission, the packet adaptation (PacketSize) adaptation is a transmission adaptation measure. When the packet size adapts to the underlying transmission link condition, the optimal transmission efficiency can be obtained, and the packet loss rate can be controlled to a medium and low level, and the bandwidth utilization rate can be increased without wasting limited bandwidth resources, thereby providing better quality of service.

At present, the Internet Protocol (Internet Protocal, IP) uses a method of segmentation and assembly for packet size adaptation, and assigns a unique identifier to each IP data packet, that is, an IP data packet. The header portion also has segmentation and assembly. Related segmentation marks and displacements. When an IP packet is segmented, each segment needs to contain the original identifier. However, in order to improve efficiency and reduce the burden on the router, segmentation and reassembly are usually provided by the IP source and destination, and the router does not perform IP layer segmentation and reassembly.

The Internet Protocol Version 6, IPv6, includes a segment offset value, a "more segment" flag, and an identification field through the segmentation extension header. The length of the source node pair exceeds the source and destination. Maximum Transmission Unit (Maximum Transmission Unit, The MTU) packet is segmented. IPv6 only allows the source node to segment the packet, which simplifies the processing of the packet by the intermediate node.

In summary, when the media stream is output from the source, the packet size has been determined. When transmitting across the network or in a heterogeneous network, the router cannot perform packet size adaptation, so the optimal transmission efficiency cannot be achieved.

This patent is directed to the above-mentioned streaming media transmission requirements and problems. Summary of the invention

The technical problem to be solved by the present invention is to provide a streaming media transmission method and system to improve network transmission efficiency.

To solve the above technical problem, the present invention provides a streaming media transmission method, the method comprising: a streaming media server sending a media stream, where the media packet size adaptation information is carried;

Receiving, by the media forwarding server, the media stream, and performing packet size adaptation processing on the media stream according to the media packet size adaptation information in the received media stream;

The media forwarding server forwards the media stream after the packet size adaptation process.

The above method can also have the following characteristics:

The media packet size adaptation information includes adaptation range information, load layout information, and adaptation load information; the adaptation range information includes a maximum transmission unit (MTU) corresponding to an available packet size; the load layout information. The number of load packets corresponding to each packet size is included, and load header and load data positioning information of each load packet; and the adapted load information includes load header data and data length.

The above method can also have the following characteristics:

When the media stream is scrambled data, the media packet size adaptation information further includes descrambling information, which is used to provide descrambling required information for the packet size-adapted scrambled data.

The above method can also have the following characteristics:

The media packet size adaptation information is carried by a transport header or data portion.

The above method can also have the following characteristics: Performing the packet size adaptation process according to the media packet size adaptation information includes: segmenting or reorganizing the media stream to be forwarded according to the media packet size adaptation information and combining the underlying transmission link state, and completing the adaptation process. .

The above method can also have the following characteristics:

The media stream forwarded by the media forwarding server carries the original media packet size adaptation information or the updated media packet size adaptation information; or the media packet size adaptation information is filtered out in the media stream.

To solve the above technical problem, the present invention also provides a streaming media transmission system, the system comprising a streaming media server and a media forwarding server, wherein:

The streaming server is configured to: send a media stream, where the media packet size adaptation information is carried; the media forwarding server includes:

a receiving module, configured to: receive the media stream;

And an adaptation processing module, configured to: perform packet size adaptation processing on the media stream according to media packet size adaptation information in the received media stream;

The forwarding module is configured to: forward the packet size to process the processed media stream.

The above method can also have the following characteristics:

The present invention also provides a media forwarding server, which includes:

a receiving module, configured to receive a media stream sent by the streaming media server;

And an adaptation processing module, configured to perform packet size adaptation processing on the media stream according to the received media packet size adaptation information in the media stream; and

A forwarding module configured to forward the media stream after the packet size adaptation process. In the streaming media transmission method and system of the present invention, the media stream sent by the streaming media server carries the media packet size adaptation information; in the network transmission process of the media stream, the media forwarding server adapts the information and the link status according to the media packet size in the media stream. Provide packet size adaptation for optimal network transmission efficiency. BRIEF abstract

1 is a schematic diagram of a streaming media transmission method according to an embodiment of the present invention;

2 is a schematic diagram of a streaming media transmission system according to an embodiment of the present invention;

3 is a schematic diagram of a specific example of a communication system;

4 depicts an example of media packet size adaptation information of the present invention being transmitted in a separate stream; FIG. 5 depicts an example of media packet size adaptation information for media packets of the present invention to be placed in separate streams for transmission;

Figure 6 is a diagram showing an example of media packet size adaptation information of the present invention placed in a media transport header for transmission;

Figure 7 illustrates the processing flow of the media forwarding server in the packet size adaptation embodiment;

Figure 8 depicts a media client endpoint broadcast procedure in a packet size adaptation embodiment.

Preferred embodiment of the invention

In the embodiment of the present invention, a logical or physical device that provides a media transport layer forwarding function in a media stream transmission process is referred to as a media forwarding server. The real-time segmentation or assembly of the media stream by the media forwarding server is an application layer solution for realizing media packet size adaptation. However, in the existing media transmission protocol, the Moving Pictures Experts Group (MPEG) Transport Stream (TS) packet length is fixed at 188 bytes; Real-time transmission protocol (Real-time)

Transport protocol, RTP) supports variable length packets and specifies various load formats. The media forwarding server needs to parse the RTP payload and perform real-time segmentation or assembly according to the payload format. On the one hand, the media forwarding server needs to support various encoding formats with high complexity; on the other hand, real-time segmentation and assembly can affect the efficiency of the media forwarding server. In the case where the media stream is scrambled, the media forwarding server cannot perform arbitrary on the media stream. Segmentation or assembly, otherwise it may affect the descrambling of the code stream at the terminal.

In view of this, an embodiment of the present invention provides a streaming media transmission method and system, where a streaming media server sends a media stream, which carries media packet size adaptation information; in a network transmission process of the media stream, the media forwarding server is based on the media stream. Media packet size adaptation information and link conditions provide packet size adaptation for optimal network transmission efficiency.

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict.

As shown in FIG. 1, the streaming media transmission method of the embodiment of the present invention includes:

Step 101: The media server sends a media stream, where the media packet size adaptation information is carried. The media packet size adaptation information in the embodiment of the present invention includes, but is not limited to, adaptation range information, load layout information, and adaptation load information. When the media stream is scrambled data, the media packet size adaptation information further includes adding descrambling information.

The media packet is large, and the adaptation information includes but is not limited to the following four types of information:

First, the adaptation range information

The adaptation range information provides a description of the packet size adaptation range, the total number of choices available for packet size adaptation, and the MTU for each selection.

Second, the load layout information

The load layout information is selected according to each packet size, and the number of load packets is provided, as well as the load header and load data location information of each load packet.

Provides packet size information according to alignment points, and supports multiple alignment methods, such as frame alignment, video GOP alignment, or other methods.

When aligned according to the frame, a media frame may be split into multiple packets for transmission. The load layout information indicates how many load packets the media frame is split into, the load header and the corresponding load data start position. When aligned according to the video GOP, the load layout information indicates how many load packets, the load headers, and the corresponding load data start positions that the GOP is split into. The starting position is a byte offset based on the alignment point, and the specific description corresponds to the specific transmission format of the media packet size information, which is not limited herein. Third, adapt the load information

Provide pre-made load header data and data length. The payload header uses the agreed load transfer format, such as the load format defined by the RTP series transmission standard defined by the standard organization IETF, or other restrictions.

Fourth, add descrambling information

Applies only to scenarios where the media load is descrambled. The descrambling information is transmitted, and the information required for descrambling is provided for the packet data after the packet size is adapted.

According to the existing streaming media transmission protocol, a streaming media packet carried on the underlying network usually consists of a transmission header, a payload header, and data.

The media packet size adaptation information may be placed in a transport header to carry. For example, extensions are made for RTP and MPEG-TS. The existing standard media transport protocols RTP and MPEG-TS represent two different transport ideas, respectively. RTP supports variable length. MPEG-2 TS is defined as a fixed byte of 188 due to the early impact of early transmission network technology. With the development of network technology, it is evolving. Hereinafter, in order to facilitate the description of the present invention, a variable length transmission method based on MPEG-2 TS is referred to as TS'. By extending the field of the RTP header (defined according to the RTP extension standard RTP Header Extension) and the TS' header (defined according to the TS header extension standard transport specific data (transport_private_data)), the media group can be carried in the extension field. Size adaptation information, as shown in Figure 5 below.

The media packet size adaptation information may also be placed in the data part, for example, for the TS' transmission mode, a PID (Program Identification) is added to the PMT (Program Map Table) to transmit the media packet size. The information flow is adapted, and the media packet size adaptation information is delivered through a custom format, as shown in FIG. 4 subsequently. The redundant content in the media packet size adaptation information, such as the adaptation range, may also be carried in a user-defined descriptor ( Private descriptors) in the PMT that describes the transport size adaptation information stream.

The above describes several specific ways of carrying the media packet size adaptation information in the inband, and may also use other methods without limitation.

Step 102: The media forwarding server receives the media stream according to the media in the received media stream. The packet size adaptation information performs packet size adaptation processing on the media stream;

In this step, the media forwarding server applies media packet size adaptation information to provide fast packet and assembly adaptation functions.

Step 103: The media forwarding server forwards the media stream after the packet size adaptation process. The media layer adaptation is performed by the media forwarding server in the transport network according to the media packet size adaptation information in the media stream. A media forwarding server, such as a media forwarding server located at the edge of the network, identifies the adaptation requirements for the packet size. Then, according to the adaptation of the underlying transmission link status, the media stream carries the media packet size adaptation information, and the media stream is segmented or reorganized to complete the adaptation process.

The media forwarding server may choose to transparently forward the media packet size adaptation information during the forwarding process, or modify the media packet size adaptation information and then forward the packet, or filter the media packet size adaptation information.

Corresponding to the above method, the embodiment of the present invention further provides a streaming media transmission system. As shown in FIG. 2, the system includes a streaming media server and a media forwarding server, where:

The streaming media server is configured to send a media stream, where the media packet size adaptation information is carried; the media forwarding server includes:

a receiving module, configured to receive the media stream;

The adaptation processing module is configured to perform packet size adaptation processing on the media stream according to media packet size adaptation information in the received media stream;

The forwarding module is configured to forward the media stream after the packet size adaptation process.

Optionally, the media packet size adaptation information includes adaptation range information, load layout information, and adaptation load information; the adaptation range information includes a maximum transmission unit (MTU) corresponding to an available packet size. The load layout information includes the number of load packets corresponding to each packet size, and load header and load data positioning information of each load packet; the adapted load information includes load header data and data length.

Optionally, when the media stream is scrambled data, the media packet size adaptation information further includes descrambling information, and is used to provide descrambling required information for the packet size-adapted scrambled data. The media packet size adaptation information may be, but is not limited to, being carried by a transport header or a data part. Belt.

Performing the packet size adaptation process according to the media packet size adaptation information includes: segmenting or reorganizing the media stream to be forwarded according to the media packet size adaptation information and combining the underlying transmission link state, and completing the adaptation process. .

Fig. 3 shows a specific example of a communication system in which the network 1 includes a streaming server 10, a router 11, a router 12a and a media forwarding server 12b, the network 2 includes a router 13, and the network 3 includes a router 14. The communication system is used for network transmission and forwarding of packet data such as audio, video, subtitle, data, and the like. The network layer of the communication system uses the IP protocol, and the data link layer uses Dense Wavelength Division Multiplexing (DWDM) or Asynchronous Transfer Mode (ATM).

The forwarding steps of the media stream in the communication system are as follows:

Step A: In the network 1, the streaming server 10 outputs various media streams in real time. The link of the streaming server 10 is configured to transmit at least 1500 bytes of network packets.

Figures 4 and 5 show an example of media packet size adaptation information being transmitted in a separate stream. The example uses a variable length TS' media transport format based on MPEG-2 TS.

Figure 4 depicts the media streaming situation:

1. TS, Program Association Table (PAT) contains programs with program number (program_number) of 0x0001 and PID of 0x0100.

2. In TS', the stream transmission program with 0x0100 PID is 0x0001, and ΡΜΤ contains two streams:

2-1. The H.264 standard video stream with stream type (stream_type) of 0xlB and PID of 0x0111; and:

2-2. User-defined type with stream type 0x80 and PID 0x0112, used here for media packet size adaptation information type. 3. The TS' packet with a PID of 0x0111 transmits a video stream;

4. The TS' packet transmission medium packet size adaptation information stream with a PID of 0x0112;

5. Insert the corresponding media packet size adaptation information with PID 0x0112 in front of the alignment point with PID 0x0111. If the alignment point uses the frame boundary, one or several media packet size adaptation information corresponding to the video frame is inserted in the first bit transmission of each video frame of the OxlB type.

Figure 5 shows an example of the adaptation information format. The specific structure is defined as shown in Table 1:

Table 1

The above structure definition contains the following information:

Adaptation range information: MTU_num and MTU fields;

Load layout information: Fields in the second For Loop;

Adaptation load information: header-len and header-data-byte fields point to the payload header; the bytes between payload-start-offset and payload-end-offset are payloads; Scrambling information: is—conditional access field. If the value 1 indicates that the media is scrambled, no segmentation or combination is allowed.

Figure 6 shows an example of media packet size adaptation information placed in the media transport header for transmission. The difference from Figure 4 is:

1. TS, ΡΜΤ only one media stream with stream type 0xlB and PID 0x0111, and media packet size adaptation information is placed in the TS' header of the video stream.

2. The media packet size adaptation information is carried at the alignment point where the PID is 0x0111 (not shown in this figure. In TS, the method of header extension is standard, and the information field is embodied in Figure 5 and Table 1, no longer here. Brief description). If the alignment point is a frame boundary, the packet size of the TS' packet containing the first bit of each video frame carries the media packet size adaptation information. Figure 5 shows the adaptation information format PacketSizeAdaptation—Private—data is also applicable, and it needs to be optimized in terms of byte length.

Step B: The media stream flows through the router 11 to the router 12a, and the router 12a receives the media stream and forwards it to the 12b media forwarding server of the same node;

Step C: The media forwarding server 12b performs packet size adaptation on the media stream according to the MTU or link status of the network 2 and the network 3. Assume that the terminals facing network 2 are mobile handheld devices such as 15a and 15b, and the MTU is configured to be 500 bytes. Network 3 is for PCs and TV devices such as 15d and 15e, and the MTU is configured to be 1000 bytes. 15c is a high-definition television device, and the MTU is configured as 1500 bytes. If the media packet size adaptation information provides four MTU options of 250 bytes, 500 bytes, 1000 bytes, and 1500 bytes, the MTU of the 1500-byte original media stream after being adapted by the media forwarding server 12b is 500 words. Section, 1000 bytes. For 15c, the media forwarding server 12b does not need to perform packet size adaptation.

Step D, the adapted media stream is transmitted to the router 13 of the network 2 and the router 14 of the network 3 through the router 12a;

Step E, the router 13 transmits the media stream QoS to the network nodes 15a and 15b, and the router 14 transmits the media stream QoS to the network nodes 15d and 15e for media playback. Figure 7 depicts the processing flow of the streaming media forwarding server. The process is as follows:

Step 701: The streaming media server 10 sends the foregoing example media stream.

Step 702: The media forwarding server 12b receives the media stream, and the media forwarding server 12b identifies the media packet size adaptation information PacketSizeAdaptation_Private_data;

Step 703: The media forwarding server 12b performs adaptation according to PacketSizeAdaptation-Private-data. Specifically, the step includes:

703a, filtering program information of the media stream and buffering the local information;

703b, according to the link MTU, select an appropriate MTU from the adaptation range information (such as the MTU_num and MTU fields in Table 1);

703c, determining load layout information according to a suitable MTU (targeting to the second For loop in Table 1);

703d: Determine an adaptation load according to the determined load layout information: sequentially execute a third For loop in the second For loop that has been located. Read header-len and header-data to get the payload header, read bytepayload_start_offset and payload-end-offset to get the load from the media data;

703e: segment or assemble the adapted load according to the load layout information into a complete transport packet and forward it to the underlying network for forwarding;

Step 704, the media forwarding server 12b detects whether the program information changes, if there is no change, step 707 is performed, otherwise step 705 is performed;

Step 705: Cache the changed program information, and send the updated program information.

Step 705: The media forwarding server 12b forwards the adapted media stream, and filters the media packet size adaptation information when the transmission efficiency is forwarded.

Figure 8 illustrates the client endpoint broadcast process in the IPTV multicast live broadcast scenario. The process is as follows: Step 801: The client 15a selects an IPTV multicast live broadcast;

Step 802: The client 15c joins the multicast group.

Step 803, the router 12a notifies the media forwarding server 12b; Step 804: The media forwarding server 12b sends the buffered program information that changes according to the packet size adaptation.

Step 805: The client 15c initializes the decoding resource according to the program information.

Step 806, the media forwarding server 12b forwards the packet size-adapted code stream to the router 12a;

Step 807: The router 12a forwards the media stream to the 15c client.

Step 808: The client 15c decodes and presents the received multicast media stream.

The streaming media transmission method and system provided by the embodiments of the present invention perform media layer packet size adaptation on the media forwarding device by means of media stream inband information, and are applicable to scenarios in which streaming media is transmitted across networks and heterogeneous networks, and can be effective. Improve transmission efficiency.

It should be noted that the specific embodiments in the present invention are merely enumerating individual examples of the present invention, and the streaming media transmission method and system using the design idea of the present invention and any equivalent transformation thereof are all within the scope of protection of the present invention. .

One of ordinary skill in the art will appreciate that all or a portion of the above steps may be accomplished by a program instructing the associated hardware, such as a read-only memory, a magnetic disk, or an optical disk. Alternatively, all or part of the steps of the above embodiments may also be implemented using one or more integrated circuits. Correspondingly, each module/unit in the above embodiment may be implemented in the form of hardware or in the form of a software function module. The invention is not limited to any specific form of combination of hardware and software.

Industrial applicability

The media stream transmission method and the media stream sent by the streaming media server in the system carry the media packet size adaptation information; in the network transmission process of the media stream, the media forwarding server provides the media packet size adaptation information and the link status according to the media stream. The packet size is adapted to achieve optimal network transmission efficiency.

Claims

Claim

1. A streaming media transmission method, the method comprising:

The streaming media server sends a media stream, where the media packet size adaptation information is carried;

Receiving, by the media forwarding server, the media stream, and performing packet size adaptation processing on the media stream according to media packet size adaptation information in the received media stream;

2. The method of claim 1 wherein:

The media packet size adaptation information includes adaptation range information, load layout information, and adaptation load information;

The adaptation range information includes a maximum transmission unit corresponding to a selectable packet size

( MTU ) ;

The load layout information includes a number of load packets corresponding to each packet size, and load header and load data positioning information of each load packet;

The adapted load information includes load header data and data length.

3. The method according to claim 1, wherein: when the media stream is scrambled data, the media packet size adaptation information further includes descrambling information, which is used for scrambling after packet size adaptation. The data provides the information needed to descramble.

4. The method of claim 1 wherein: the media packet size adaptation information is carried using a transport header or data portion.

5. The method according to claim 1, wherein: the step of performing packet size adaptation processing according to the media packet size adaptation information comprises: combining the media packet size adaptation information according to the underlying transmission link state, The media stream to be forwarded is segmented or reorganized to complete the adaptation process.

The method of claim 1, wherein the media stream forwarded by the media forwarding server carries original media packet size adaptation information or updated media packet size adaptation information; or the media stream is filtered out. The media packet size adaptation information.

7. A streaming media transmission system, the system comprising a streaming media server and a media forwarding server, wherein:

The streaming media server is configured to: send a media stream, where the media packet size adaptation information is carried;

The media forwarding server includes:

a receiving module, configured to receive the media stream;

And an adaptation processing module configured to perform packet size adaptation processing on the media stream according to media packet size adaptation information in the received media stream;

A forwarding module configured to forward the media stream after the packet size adaptation process.

8. The system of claim 7 wherein:

The adaptation range information includes a maximum transmission unit (MTU) corresponding to an available packet size;

The adapted load information includes load header data and data length.

9. The system according to claim 7, wherein: when the media stream is scrambled data, the media packet size adaptation information further includes descrambling information, which is used for scrambling after packet size adaptation. The data provides the information needed to descramble.

10. The system of claim 7, wherein: the media packet size adaptation information is carried by a transport header or data portion.

11. The system of claim 7 wherein:

The adaptation processing module is configured to: segment or reassemble the media stream to be forwarded according to the media packet size adaptation information and the underlying transmission link state, and complete the adaptation process.

The system of claim 7, wherein the media stream forwarded by the media forwarding server carries original media packet size adaptation information or updated media packet size adaptation information; or the media stream is filtered out The media packet size adaptation information.

13. A media forwarding server, comprising:

14. The media forwarding server of claim 13, wherein:

15. The media forwarding server of claim 13 wherein:

The adaptation processing module is configured to: segment or reassemble the media stream to be forwarded according to the media packet size adaptation information and the state of the underlying transmission link, and complete the adaptation process.