WO2007045140A1 - Methode en temps reel pour transferer des donnees multimedia - Google Patents

Methode en temps reel pour transferer des donnees multimedia Download PDF

Info

Publication number
WO2007045140A1
WO2007045140A1 PCT/CN2006/001845 CN2006001845W WO2007045140A1 WO 2007045140 A1 WO2007045140 A1 WO 2007045140A1 CN 2006001845 W CN2006001845 W CN 2006001845W WO 2007045140 A1 WO2007045140 A1 WO 2007045140A1
Authority
WO
WIPO (PCT)
Prior art keywords
real
header information
transport protocol
rtp
time transport
Prior art date
Application number
PCT/CN2006/001845
Other languages
English (en)
Chinese (zh)
Inventor
Bin Song
Zhong Luo
Original Assignee
Huawei Technologies Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co., Ltd. filed Critical Huawei Technologies Co., Ltd.
Publication of WO2007045140A1 publication Critical patent/WO2007045140A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/643—Communication protocols
    • H04N21/6437—Real-time Transport Protocol [RTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/65—Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8451—Structuring of content, e.g. decomposing content into time segments using Advanced Video Coding [AVC]

Definitions

  • the present invention relates to the field of multimedia communication technologies, and in particular, to a method for real-time transmission of multimedia data. Background technique
  • Real-time streaming is real-time delivery, especially for live events. Real-time streaming must match the connection bandwidth, which means that image quality will degrade due to reduced network speed to reduce the need for transmission bandwidth.
  • the concept of "real time" means that the delivery of data in an application must be kept in precise time relationship with the generation of the data.
  • video communication is gradually becoming one of the main services of communication.
  • Two-way or multi-party video communication services such as video telephony, video conferencing, and mobile terminal multimedia services, impose strict requirements on the transmission of multimedia data streams and the quality of services. Not only does network transmission require better real-time performance, but equivalently requires video data compression coding to be more efficient.
  • the ITU-T Telecommunication Standardization Sector In view of the current demand for media communication, the ITU-T Telecommunication Standardization Sector officially released H.264 in 2003 after the development of video compression standards such as H.261, H.263, and H.263+. standard.
  • This is a highly efficient compression coding standard that is jointly developed by the ITU-T and the Moving Picture Experts Group (MPEG) of the International Standardization Organization (ISO) to adapt to the new phase of network media delivery and communication requirements. It is also the main content of Part 10 of the MPEG-4 standard.
  • MPEG Moving Picture Experts Group
  • ISO International Standardization Organization
  • the purpose of the H.264 standard is to improve video coding efficiency and its network The suitability of the network.
  • the H.264 video compression coding standard has gradually become the mainstream standard in multimedia communication.
  • a large number of ⁇ .264 multimedia real-time communication products such as conference TV, videophone, 3CJ mobile communication terminal
  • network streaming products have been introduced.
  • Whether or not to support ⁇ .264 has become a key factor in determining the competitiveness of products in this market segment. It can be predicted that with the official promulgation and widespread use of ⁇ .264, multimedia communication based on IP networks and 3G and 3G wireless networks will inevitably enter a new stage of rapid development.
  • multimedia communication not only requires high efficiency of media compression coding, but also requires real-time transmission of the network.
  • multimedia streaming is basically based on the RTP Real-time Transport Protocol and its Real-time Transport Control Protocol (RTCP).
  • RTP is a transport protocol for multimedia data streams over the Internet, published by the Internet Engineering Task Force (IETF).
  • IETF Internet Engineering Task Force
  • RTP is defined to work in one-to-one or one-to-many transmissions with the goal of providing time information and stream synchronization.
  • the typical application of RTP is based on the User Datagram Protocol (UDP), but it can also work on other protocols such as TCP (Transport Control Protocol) or Asynchronous Transfer Mode (ATM). .
  • UDP User Datagram Protocol
  • ATM Asynchronous Transfer Mode
  • RTP itself only guarantees the transmission of real-time data, and does not provide a reliable transmission mechanism for transmitting packets in sequence, nor does it provide flow control or congestion control. It relies on RTCP to provide these services.
  • RTCP is responsible for managing the transmission quality to exchange control information between current application processes.
  • each participant periodically transmits RTCP packets, which contain statistics such as the number of transmitted packets and the number of lost packets. Therefore, the server can use this information to dynamically change the transmission rate, even Change the payload type. Used in conjunction with RTCP, RTP optimizes transmission efficiency with efficient feedback and minimal overhead, making it ideal for real-time data on the delivery network.
  • H.264 multimedia data is transmitted over IP networks, also based on UDP and its upper layers.
  • RTP protocol itself is structurally applicable to different media data types, but different high-level protocols or media compression coding standards in multimedia communication (eg H.261, H.263, .MPEG-1/-2/-4, MP3, etc.), the IETF will formulate a specification file for the RTP payload (Payload) packaging method of the protocol, and specify the method of RTP encapsulation and packaging, which is optimized for the specific protocol.
  • the corresponding IETF standard for H.264 is PC 3984: RTP Payload Format for H.264 Videowhi This standard is currently H.264 The main standard for video stream transmission over IP networks is widely used. In the field of video communication, the products of major manufacturers are based on RFC 3984, and it is currently the only H.264/RTP transmission method.
  • H.264 defines a new layer called NAL (Network Abstract Layer), which is a standard that makes it standard.
  • NAL Network Abstract Layer
  • the interface opens up the underlying business capabilities and shields the underlying network from the differences and abstracts the business capability layer.
  • VCL Video Coding Layer
  • H.264 brings greater application flexibility and defines a new layer of NAL.
  • the early ITU-T video compression coding protocols such as H.261, H.263/H.263+/H.263++ were not available.
  • RTP better for H.264, practical, and worthy of study.
  • the scheme encapsulates the NAL layer data in the RTP payload for bearer based on the RTP protocol (RFC 3550).
  • the NAL layer is located between the VCL and the RTP, and is configured to divide the video stream into a series of network abstraction layer data units (NALUs, NAL Units) according to defined rules and structures.
  • NALUs, NAL Units network abstraction layer data units
  • the encapsulation format of the RTP payload for NALU is defined in RFC3984. The following briefly introduces the RTP frame format and the existing NALU encapsulation method.
  • RTP Real-time multimedia conferencing and continuous data storage, interactive distributed simulation, control and measurement applications.
  • RTP is typically carried over the UDP protocol to take advantage of its multiplexing and parity functions. If the underlying provides multipoint distribution, RTP supports multi-address delivery.
  • the functions provided by RTP include: payload type identification, sequence number, time stamp, and transmission monitoring.
  • RTP packages the NALU package of H.264 into an RTP packet stream.
  • the NALU is mainly defined in the RFC 3984 file, and the H.264 layer is given based on this.
  • FIG. 1 shows the encapsulation structure of a NALU in the payload of the RTP.
  • the first byte in the previous byte is the NALU header information, and then the data content of the NALU, and the payloads of the multiple NALUs that are filled in the end to the RTP packet.
  • there is an optional RTP padding which is specified in the RTP packet format, in order to make the length of the RTP packet meet certain requirements (such as Fixed length), optional RTP fill data is generally zero filled.
  • the NALU header information is the first byte, also known as the Octet, which has three fields. The meaning and full name are described as follows:
  • the F field is defined as a forbidden bit (forbidden_zero-bit), which is 1 bit, used to identify grammatical errors, etc., and is set to 1 if there is a syntax conflict.
  • a forbidden bit forbidden_zero-bit
  • the NI field is defined as a NAL reference identifier (nal_ref_idc), which is 2 bits, used to indicate the importance of the NALU data.
  • a value of 00 indicates that the content of the NALU is not used to reconstruct the inter-predicted reference image, instead of 00.
  • Indicates that the current NALU is important data such as a slice or a sequence parameter set (SPS, Sequence Parameter Set) and a picture parameter set (PPS, Picture Parameter Set) belonging to a reference frame. The larger the value, the more important the current NAL is;
  • the Type field is defined as NALU type (Nal_ unit_type), which is 5 bits in total, and can have 32 types of NALU. The correspondence between the value and the specific type is given in Table 1.
  • the information given in one byte of the NALU header information mainly includes the validity and importance level of the NALU, and based on the information, the importance of the data carried by the RTP can be determined.
  • the prior art scheme can be summarized as follows: First, the video bit stream of H.264 is segmented according to a certain rule to form a NALU stream in the NAL layer, and this step actually belongs to the category of H.264 implementation, for example, Taking a frame of image as a NALU, you can also use a slice as a NALU, and then package the NALU stream into an RTP packet stream according to the package encapsulation strategy associated with the application.
  • the RTP data packet after the header information is the NALU data area. If an RTP data packet encapsulates multiple NACHs of H.264, then these NALUs are arranged end to end, and each NALU occupies a continuous bit, and each NALU is the same.
  • One byte is the NALU header information byte, and at the end of the RTP packet, there may be some optional padding data bits as needed.
  • the underlying RTP protocol does not process the specific information of the NALU.
  • the NALU header information of H.264 is not reflected in the header information of the RTP packet.
  • the NALU header information byte contains a lot of important information.
  • N I indicates whether the corresponding NALU contains H.264 non-reference frames or reference frames or other important image data such as parameter sets.
  • the RTP protocol itself does not provide any QoS (Quality of Service) mechanism, and does not provide information about the priority of the bearer data, etc., the RTP itself does not have any errors such as network packet loss on the IP and wireless networks. Fault tolerance, or Error Resilience.
  • QoS Quality of Service
  • the information about the H.264 NALU attribute can be obtained by directly scanning the RTP header information. Based on this information, it is possible for the network device to handle the RTP packets differently, so as to ensure the priority of important data in the transmission process.
  • an RTP packet encapsulates multiple H.264 NALUs, the types of these NALUs are the same, then the header bytes of these NALUs are exactly the same. If there are N NALUs in an RTP packet, then the N NALU header information bytes can be replaced by one byte without any loss of information, but the efficiency is improved because N-1 redundant bytes can be reduced. .
  • the header information of the NALU is completely encapsulated in the payload, so that the RTP
  • the protocol cannot directly know the attributes, levels, importance, etc. of the payload, so that the QoS mechanism based on this cannot be implemented.
  • such an encapsulation format also causes the NALU header information to occupy the payload resources, because each NALU has header information, which results in many cases, because the header information of multiple NALUs of the same type in an RTP are the same. , which wastes RTP transmission bandwidth resources.
  • the main purpose of the present invention is to provide a real-time transmission method of H.264 multimedia data, so that the RTP protocol can efficiently carry multimedia video data, and the service is added under the premise of being compatible with existing RTP protocol devices and transmission methods. Quality assurance mechanism.
  • the multimedia data is divided into a network abstraction layer unit stream in a network abstraction layer, and the network abstraction layer unit includes header information, and the method includes:
  • the sender encapsulates at least one network abstraction layer unit with the same header information in the same improved real-time transport protocol packet according to the improved real-time transport protocol encapsulation format, and sets an identifier in the improved real-time transport protocol header information to distinguish it from real-time transmission.
  • Agreement package
  • the receiving party determines whether the packet is an improved real-time transport protocol packet according to the identifier, and if so, processes the packet according to the improved real-time transport protocol encapsulation format, and acquires the carried network abstraction layer unit;
  • the same header information possessed by the network abstraction layer unit carried by the network is included in the header information of the improved real-time transport protocol packet, and the header of the network abstraction layer unit carried After the information is removed, it is populated into the payload of the improved real-time transport protocol packet.
  • the network abstraction layer unit header information includes:
  • a disable bit field configured to indicate whether the network abstraction layer unit is in error
  • a network abstraction layer reference identifier field configured to indicate an importance of the network abstraction layer unit
  • a type field configured to indicate a type of the network abstraction layer unit
  • the network abstraction layer reference identification field and type field are populated in a payload type field of the improved real-time transport protocol header information.
  • the improved real-time transport protocol identifier is the version of the improved real-time transport protocol header information
  • the information field, the version information field is set in the improved real-time transport protocol header information.
  • the forbidden bit field is populated in a tag field of the improved real-time transport protocol header information;
  • the receiver judges whether the network abstraction layer unit carried by the real-time transport protocol packet is in error according to the marked field of the improved real-time transport protocol packet.
  • the receiving party includes a communication terminal and a network intermediate device.
  • the improved real-time transport protocol identifier is in the marked field of the improved real-time transport protocol header information.
  • the sender first determines whether the forbidden bit field in the header information of at least one of the network abstraction layer units is valid, and accordingly divides the barred data field into a normal network abstraction layer unit and an error network abstraction layer unit;
  • the receiver first divides the modified real-time transport protocol identifier into the improved real-time transport protocol packet and the real-time transport protocol packet according to the received header information of the packet; according to the improved real-time transport protocol encapsulation format Processing the improved real-time transport protocol packet, processing the real-time transport protocol packet according to the real-time transport protocol packet encapsulation format.
  • the type of the network abstraction layer unit is less than 16 types, only the lower 4 bits of the type field are used, and the highest bit of the type field is reserved as an extension. Bit.
  • the multimedia transmission device learns related information of the network abstraction layer unit carried by the real-time transmission protocol header information according to the improved real-time transmission protocol header information, and implements the service quality policy for real-time transmission of the multimedia data according to the implementation.
  • the technical solution of the present invention provides an improved RTP protocol (MRTP, Modified RTP) for carrying NALU data by using all NALUs in the same RTP packet.
  • Header information bytes are combined into its header information
  • a combination method is adopted to prevent the operation of the existing RTP protocol and the device, and the attribute of the NALU payload can be directly reflected in the MRTP header information, so that the encapsulation efficiency of the MRTP to the NALU is greatly improved.
  • the implementation of the NAL mechanism by the MRTP on the payload NALU attribute provides the basis for the implementation of the QoS mechanism;
  • the identification method of distinguishing the existing RTP and MRTP is given, which makes it possible for the existing network media equipment to support both RTP and MRTP, and improve The compatibility of MRTP; and the corresponding conventional RTP single transmission method for NALU with syntactical errors or errors, and the scheme of alternate processing of MRTP and RTP data packets are given.
  • the header information of the improved MRTP protocol of the present invention carries the NALU header information of the H.264, so that the importance of the MRTP data packet carrying the NALU can be determined by performing a fast scan of the MRTP header information without decoding the NALU. Therefore, corresponding measures are taken to implement QoS policies, etc., to further improve service quality;
  • the purpose of reducing redundancy and improving transmission efficiency is achieved, thereby improving the video transmission quality of the multimedia video communication of the IP network and further satisfying the requirements of the user.
  • FIG. 1 is a schematic diagram of a format of encapsulation of NALU data in an RTP packet payload in the prior art
  • FIG. 2 is a schematic diagram of a header information structure of an RTP packet
  • FIG. 3 is a schematic diagram showing the structure of a header information of an MRTP data packet according to an embodiment of the present invention. detailed description
  • the present invention aims to provide a multimedia data transmission scheme capable of embodying H.264 NALU header information in the header information of the RTP.
  • the basic principle is to use some or some bytes or bits in the current RTP header information to represent the NALU header information, and the purpose of these bytes or bits in the RTP is to combine with the specific media protocol carried. Extended space of.
  • the improved RTP protocol will not affect the interoperability with devices supporting the original RTP protocol, that is, some terminals in the communication adopt the improved RTP protocol according to the present invention, and the other terminals adopt the unmodified RTP protocol, and the terminals use It is fully communicable.
  • the terminal can also adopt different processing measures for different situations to implement the alternate transmission processing scheme of MRTP and RTP. This includes a solution to the syntax error NALU, which is transmitted using traditional RTP.
  • the improvement of the existing RTP by MRTP mainly involves the redefinition of the payload type FT field and the version information V field in the RTP packet header information.
  • the two potential values of the scheme are: Provide a certain QoS mechanism for H.264 data transmission; Improve the efficiency of RTP encapsulation H.264 NALU.
  • the format of the RTP packet is briefly introduced:
  • the basic option of the RTP header information occupies 12 bytes (minimum case), and the header information of the IP protocol and the UDP protocol respectively occupy 20 bytes and 8 words. Therefore, the RTP packet is encapsulated in the UDP packet and then encapsulated in the IP packet.
  • the detailed structure of the header information of the RTP packet is shown in Figure 2.
  • the front-to-back RTP header information shown in Figure 2 is:
  • the first byte (byte 0) is some field about the header information structure itself
  • the second byte (byte 1) is the defined payload type
  • the third 4 bytes (bytes 2, 3) are the sequence number (Sequence Number)
  • the 5th-8th byte is the timestamp (timestamp)
  • the 9th-12th byte is the synchronous contribution source identifier (SSRC ID, Synchronous Source) Identifier )
  • SSRC ID Synchronous Source
  • CSRC Ids Contributing Source Identifiers
  • the first 12 bytes appear in all different types of RTP packets, while other data in the header information, such as the contribution source identifier, is only available when the mixer is inserted. Therefore, CSRC is generally used when there is media mixing. For example, in a multi-party conference, the audio needs to be mixed, and the video can also provide multi-screen functions in this way.
  • the synchronization source identifier SSRC is actually the identifier of the carried media stream.
  • the P field is a padding flag (Padding), which is 1 bit. If P is set, it indicates that the packet contains one or more padding bytes (Padding) at the end, and the padding does not belong to a part of the payload;
  • the X field is an extension flag bit (Extension) ), occupying lbit, if X is set, the last part of the RTP header must be followed by a variable-length header extension (if there is a CSRC list, the header extension is followed), mainly for the header information in some application environments.
  • the header extension includes a 16-bit length field to count how many 32-bit words are in the extension, and the first 16 bits of the header extension are left-opened to distinguish between identifiers and parameters.
  • the format of the bit is defined by a specific level specification, which is described in detail in Section 5.3.1 of RFC 3550, which is not given here;
  • the CC field is the number of contributing sources (CSRC Count), which is 4 bits, indicating the number of CSRC identifiers at the end of the header information.
  • the receiver can determine the length of the CSRC IDs list following the header information according to the CC field.
  • the M field is a marker bit (Marker), which occupies 1 bit.
  • the interpretation of the identifier bit is defined in a specific profile. It allows identification of important events in the packet stream.
  • One layer can define additional identification bits or specify no Identification bit, the so-called level refers to the specific application environment setting, which is specifically agreed by the communication parties and is not limited by the agreement;
  • the PT Payload Type
  • the PT indicates the payload type, a total of 7 bits, identifies the format of the RTP payload and confirms his interpretation in the application; the flag bit and the payload type share a specified number of bytes, and this byte may be specified.
  • the level is redefined to suit different needs.
  • the so-called profile can be defined in a specific application. In fact, it is a set of static (that is, agreed by the communication parties), and the different values of the PT bits are associated with different media formats.
  • dynamic negotiation can also be used to define the relationship between the PT value and the media format through signaling other than RTP.
  • the RTP source can change the PT.
  • serial number 16 bits. Each time an RTP data packet is sent, the serial number value is incremented by one, so that the receiver can use it to detect the data packet loss and recover the data packet sequence.
  • the initial value of the serial number in one communication can be given randomly. , does not affect communication.
  • the timestamp occupies 32 bits, which reflects the sampling time of the first byte in the RTP packet.
  • the sampling time here must be derived from a monotonically increasing clock, and the receiver adjusts the media playback time or synchronizes according to it.
  • the synchronization source SSRC ID occupies 32 bits, and its specific value can be randomly selected. However, to ensure the uniqueness in the same RTP session, it can uniquely identify a media source. If a source changes the source transmission address, a new SSRC must be selected. The identifier.
  • the source CSRC list can be 0-15 items as required, each item occupying 32 bits.
  • the length of the list ie the number of CSRC IDs, is exactly indicated by the 4 bits of the CC field.
  • the CSRC identifier used to identify a media source is identical to the SSRC identifier of its corresponding contribution source, except that the role of the different receivers is different and is set to SSRC or CSRC.
  • the CSRC ID is inserted by the mixer.
  • NALU data is encapsulated in a modified RTP (MRTP, Modified RTP) format, which is described below by way of specific implementation. All the descriptions of MRTP given are only different from the existing RTP. The most basic difference between MRTP and RTP is that the header information of the NALU packet with the same header information is integrated into the header information of the MRTP during the MRTP encapsulation process.
  • MRTP Modified RTP
  • the NALU header information structure has been mentioned above, and the NALU information includes: a 1-bit F field for indicating whether the NALU is in error;
  • a 5-bit Type field indicating the type of the NALU.
  • the sender encapsulates multiple NALUs with the same header information in the same MRTP packet according to the MRTP encapsulation format, and sets an MRTP identifier in the MRTP header information to distinguish the RTP packets.
  • only the same type of H.264 NALU is stored in the same MRTP data packet, that is, it has the same header information.
  • the same type of NALU can be accumulated until a certain number is satisfied and then encapsulated into MRTP.
  • the RTP padding method may be adopted.
  • the receiver determines that the packet is an MRTP packet according to the MRTP identifier, and is encapsulated according to the MRTP.
  • the format processes the MRTP packet to obtain the NALU carried.
  • the receiver can identify the MRTP packet according to the MRTP identifier, which is mainly different from the RTP packet, so that the terminal using the MRTP protocol does not affect the normal communication of the existing RTP protocol, and has backward compatibility.
  • the above mentioned MRTP encapsulation format integrates the same header information of the NALU carried by the NATU in the header information of the MRTP packet, and removes the header information of the carried NALU and fills the MRTP packet. In the payload. This is the main difference between MRTP and RTP. As mentioned earlier, this facilitates function expansion and saves bandwidth.
  • the focus here is on integrating the NALU header into the MRTP header and identifying the MRTP packet.
  • the NRI field and the Type field in the NALU header information are filled in the PT field of the MRTP header information.
  • the PT field is located in the second byte of the MRTP header information. The last 7 bits.
  • the format of this MRTP header is shown in Figure 3, where the difference from RTP has been shown in bold, and in addition, some places in the figure are explained later.
  • the V field in the MRTP header is used as the MRTP identifier, and if it is the MRTP packet, the V field is taken as 3 (binary value 11), and the V field is located before the first byte of the MRTP header information.
  • the M field of the MRTP packet determines whether the NALU carried by the MRTP is in error, and the F bit disable function is also implemented.
  • the current version V of the MRTP is set to 3, which is equivalent to the new version of RTP, and the current RTP version V has a value of 2.
  • the RTP protocol is an improved version of MRTP, so that the subsequent processing is performed according to the processing flow for improving the RTP protocol.
  • An alternative will be described later, and the purpose of representing the difference between MRTP and RTP can be achieved without modifying V.
  • the NALU header information byte (8 bits) is replaced with the identification M field 1 bit in the original RTP header information and the PT field 7 bits total 8 bits.
  • the specific replacement order can be like this:
  • F bits replace M bits; NRI 2 bits replace the highest 2 bits of the PT 7 bits;
  • Type 5 bits replace the lowest 5 bits of the PT 7 bits
  • the PT 7 bits are free to use.
  • the purpose of the M field is specified in RTP (RFC 3550) as follows:
  • a specific profile (Profile) can specify not to use M bits, but to incorporate it into the PT, so that ⁇ can have up to 8 bits, distinguishing 256 different type. Therefore, replacing the ⁇ bits with F bits is completely RTP-compliant and does not affect the interworking between MRTP and the original RTP.
  • the MRTP encapsulation format of the present invention has three obvious advantages: First, the overhead is small, especially when there are multiple NALUs in one RTP, the number of transmission bits is obviously saved; Second, there is no need for the RTP packets. .264 NALU data decoding can discriminate the relative importance of these NALUs. Third, without decoding the ⁇ .264 NALU data in the RTP data packet, it can identify whether the RTP packet can be correctly decoded due to other bit loss. .
  • the following describes the process of MRTP encapsulation and decapsulation.
  • multiple H.264 NALU types in the same MRTP data packet are identical, that is, their header information bytes are the same, and when they are encapsulated into the MRTP data packet, the original information can be stripped off.
  • the header information byte so if there are N NALUs, you can reduce N bytes.
  • the NALU When decapsulating, the NALU is extracted from the MRTP packet and restored to the original form, that is, the N NALUs are extracted from the MRTP packets they are in, and then the 7 bits of the PT in the MRTP header information are copied to The lowest 7 bits of one byte H (8 bits) are removed, and the highest bit of H is set to 0 as the F bit. The generated H bytes are then appended to the top of each extracted NALU, thus restoring each NALU.
  • the F field in the MRTP header is 1, it indicates that the NALU in the MRTP packet is in error, so it can be directly discarded, which saves processing time.
  • a second solution is given on the basis of the first embodiment, which is similar to the second embodiment in that: the NI and Type fields in the NALU header are filled into 7 bits of the PT field of the MRTP header. .
  • two types of F are set. NALU treats separately. For the error NALU set by F, the original RTP transmission is used, and for normal, MRTP is adopted. Transmit, but ignore the F bit. The specific details are as follows.
  • the M field is set to 1 to identify the MRTP packet, which is located in the first 1 byte of the 2nd byte of the MRTP header information.
  • it is specified in the H.264 protocol: 1 if there is a syntax conflict or an error.
  • the network recognizes that there is a bit error in this unit, it can be set to 1 so that the receiver drops the unit. It is mainly used to adapt to different kinds of network environments, such as wired and wireless combined environments.
  • the specific usage principle is: Generally, when the sender and receiver of the communication perform H.264 encoding and decoding on the video, the bit is not "written, operated, and the decoder performs a "read" operation on the bit.
  • the receiver will discard the NALU during the decoding process.
  • the "write" operation for the F bit is mainly the gateway between two different networks. Performed on, for example, the case of encoding conversion (MPEG-4 to H.264, H.263 to H.264, etc.).
  • the F bit is ignored and is not used for the purpose of the original H.264 definition.
  • the third embodiment of the present invention performs the following processing for this case: In the MRTP encapsulation format, the above is ignored. The F field in the NALU header information; but on the sender side, the error NALU that is valid for the F field is still encapsulated in the RTP packet, and only the normal NALU is used in the MRTP wrapper; on the receiving side, the receiver is judged to be the MRTP or the RTP packet. The package is processed in the corresponding package format.
  • the F bit when used in some special cases, it is used for the purpose of the original H.264 definition, that is, to indicate the possible H.264 NALU syntax error, if an intermediate device such as a gateway is in the When the video is video-encoded according to the H.264 protocol, it is found that a certain NALU has a syntax error, and then the NALU is separately packaged.
  • the sender first determines whether the F field in the header information of at least one NALU is valid, and accordingly divides it into a normal NALU and an error NALU;
  • the receiver first determines whether the header information of the received packet is an MRTP identifier, and divides it into an MRTP packet and an RTP packet;
  • the MRTP packet is then processed according to the MRTP encapsulation format, and the RTP packet is processed according to the RTP packet encapsulation format.
  • the gateway is in accordance with the foregoing method for the normal NALU, according to a certain rule of the same type of H.264 NALU (determined by the specific application, mainly specified in each MRTP data packet) How many similar NALUs are encapsulated for MRTP encapsulation.
  • a regular RTP encapsulation is required for the NALU.
  • the regular RTP packet may contain only one H.264 NALU.
  • the premise of the above method is that in the continuous H.264 NALU stream, a separate syntax error NALU occasionally appears. At this time, the wrong NALU is taken out separately and encapsulated in RTP.
  • the H.264 NALU is decapsulated according to the MRTP rule; if the RTP packet is received, the H.264 NALU is decapsulated according to the RTP rule.
  • the M NALUs can still be encapsulated by the traditional RTP.
  • the error NALUs can be accumulated until they reach the length of one RTP packet and then packed with RTP, which can save bandwidth without affecting the receiver, because the receiver It is possible to know which NALUs are missing based on the sequence number.
  • the fourth embodiment of the present invention in the MRTP encapsulation format, when all types of NALUs are less than 16 types, only the lower 4 bits of the Type field are used, and The highest bit of Type is used as an extended reserved bit. Called the C field, as shown in Figure 3. Leave the C bit for later use and continue with the function expansion. After the bit C is reserved, the NALU type given in Table 1 should be modified accordingly: A total of 16 values, the values 0-12 are the same as Table 1, and the values 13-15 are reserved.
  • the multimedia transmission device can directly learn the relevant information of the NALU carried by the multimedia transmission device according to the MRTP header information, and implement the QoS policy for real-time transmission of the H.264 multimedia data according to the same.
  • the NALU layer information is not concerned, and the head information of each NALU in the payload cannot be known, so that the QoS policy cannot be implemented.

Abstract

L'invention concerne une méthode pour transférer efficacement, en temps réel, des données vidéo multimédia H264, en faisant appel au protocole RTP. Cette méthode concerne un mécanisme perfectionné de qualité de service garantissant la compatibilité avec un appareil et des méthodes de transfert connus faisant appel au protocole RTP. Un protocole RTP modifié de l'invention est destiné à transférer des données H264 par la combinaison de tous les octets d'information d'en-tête de H264 NALU et des informations d'en-tête du paquet RTP lui-même, de sorte à ne pas affecter l'action du protocole RTP connu et de l'appareil connu. Ceci permet d'indiquer directement l'attribut de la charge H264 NALU dans les informations d'en-tête MRTP. L'invention concerne également une méthode d'identification permettant de distinguer le RTP connu du MRTP, par la modification des champs pertinents, notamment le champ M et F des informations d'en-tête RTP connues, ceci permet à l'appareil média de réseau connu de prendre en charge à la fois RTP et MRTP, et d'améliorer la compatibilité de MRTP et la souplesse de l'application.
PCT/CN2006/001845 2005-10-17 2006-07-25 Methode en temps reel pour transferer des donnees multimedia WO2007045140A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN200510113942.1 2005-10-17
CN2005101139421A CN100407726C (zh) 2005-10-17 2005-10-17 H.264多媒体数据实时传送方法

Publications (1)

Publication Number Publication Date
WO2007045140A1 true WO2007045140A1 (fr) 2007-04-26

Family

ID=37390618

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2006/001845 WO2007045140A1 (fr) 2005-10-17 2006-07-25 Methode en temps reel pour transferer des donnees multimedia

Country Status (2)

Country Link
CN (1) CN100407726C (fr)
WO (1) WO2007045140A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112839242A (zh) * 2020-12-31 2021-05-25 四川长虹网络科技有限责任公司 音视频媒体文件封装实现方法
CN114979092A (zh) * 2022-05-13 2022-08-30 深圳智慧林网络科技有限公司 一种基于rtp的数据传输方法、装置、设备和介质

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1972166B (zh) * 2006-11-30 2011-05-25 中兴通讯股份有限公司 一种移动多媒体广播系统的音频流传送方法
CN101355571B (zh) * 2007-07-26 2011-09-28 中国移动通信集团公司 多媒体消息的处理方法、装置及系统
CN101488947B (zh) * 2008-01-16 2012-07-25 联想(北京)有限公司 一种数据传输方法及装置
CN101873549B (zh) * 2010-05-26 2013-08-21 姜红志 一种采用实时流传输协议的移动视频信息点对点传输方法
CN103002353B (zh) * 2011-09-16 2015-09-02 杭州海康威视数字技术股份有限公司 对多媒体文件进行封装的方法及装置
KR101947000B1 (ko) * 2012-07-17 2019-02-13 삼성전자주식회사 방송 시스템에서 멀티미디어 데이터의 전송 특징 정보 전달 방법 및 장치
CN103313045A (zh) * 2013-05-31 2013-09-18 哈尔滨工业大学 宽带多媒体集群系统调度台h.264视频分包方法
EP2961176B1 (fr) * 2014-06-23 2017-01-11 Harman Becker Automotive Systems GmbH Correction d'erreurs dans un flux de transport multimédia numérique
CN105407351B (zh) * 2014-09-15 2019-03-12 杭州海康威视数字技术股份有限公司 一种从实时传输协议数据包中重建编码方式的方法和装置
CN110719495A (zh) * 2018-07-13 2020-01-21 视联动力信息技术股份有限公司 一种视频数据的处理方法和系统
CN112073822B (zh) * 2019-06-10 2022-10-18 成都鼎桥通信技术有限公司 一种宽带集群通信中的媒体变更方法和系统
CN114449200B (zh) * 2020-10-30 2023-06-06 华为技术有限公司 音视频通话方法、装置及终端设备

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1402492A (zh) * 2002-09-29 2003-03-12 清华大学 基于实时传输协议和传输控制协议的流媒体传输实现方法
EP1494425A1 (fr) * 2003-07-03 2005-01-05 Microsoft Corporation Format de charge utile de RTP
US20050007263A1 (en) * 2003-07-09 2005-01-13 Minhua Zhou Video coding

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1402492A (zh) * 2002-09-29 2003-03-12 清华大学 基于实时传输协议和传输控制协议的流媒体传输实现方法
EP1494425A1 (fr) * 2003-07-03 2005-01-05 Microsoft Corporation Format de charge utile de RTP
US20050007263A1 (en) * 2003-07-09 2005-01-13 Minhua Zhou Video coding

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112839242A (zh) * 2020-12-31 2021-05-25 四川长虹网络科技有限责任公司 音视频媒体文件封装实现方法
CN114979092A (zh) * 2022-05-13 2022-08-30 深圳智慧林网络科技有限公司 一种基于rtp的数据传输方法、装置、设备和介质
CN114979092B (zh) * 2022-05-13 2024-04-02 深圳智慧林网络科技有限公司 一种基于rtp的数据传输方法、装置、设备和介质

Also Published As

Publication number Publication date
CN100407726C (zh) 2008-07-30
CN1863314A (zh) 2006-11-15

Similar Documents

Publication Publication Date Title
WO2007045140A1 (fr) Methode en temps reel pour transferer des donnees multimedia
US10728591B2 (en) Method of configuring and transmitting an MMT transport packet
US10939149B2 (en) Apparatus and method for transmitting/receiving processes of a broadcast signal
Schierl et al. System layer integration of high efficiency video coding
CN101517553B (zh) 用于对内容进行包化以经由网络传输的方法和设备
Wenger et al. RTP payload format for H. 264 video
US20200029130A1 (en) Method and apparatus for configuring content in a broadcast system
KR101951650B1 (ko) 단일 포트 또는 다중 포트에서 미디어 콘텐츠 전송 방법 및 장치
TWI432035B (zh) 可縮放視訊編碼之圖像反向相容聚合技術
EP1936868B1 (fr) Méthode pour surveiller une qualité de service dans une communication multimédia
KR20190085899A (ko) 미디어 데이터를 송수신하기 위한 인터페이스 장치 및 방법
KR20190045117A (ko) 오버헤드를 최소화한 헤더를 가지는 패킷 기반의 미디어 데이터 전송 방법
WO2007045141A1 (fr) Procede de prise en charge de transmission de donnees multimedias avec tolerance aux erreurs
KR20140002026A (ko) 파일 전달 방법들을 이용한 ｉｐ 브로드캐스트 스트리밍 서비스 배포
US20200021867A1 (en) Broadcast signal transmitting and receiving method and device
CN109194982A (zh) 一种传输大文件流的方法和装置
Park et al. Delivery of ATSC 3.0 services with MPEG media transport standard considering redistribution in MPEG-2 TS format
KR20190018142A (ko) Mmt 전송 패킷의 설정 방법 및 전송 방법
Basso et al. Transport of MPEG—4 over IP/RTP
Wenger et al. RFC 3984: RTP payload format for H. 264 video
Standard Transport of high bit rate media signals over IP networks (HBRMT)
Pourmohammadi et al. Streaming MPEG-4 over IP and Broadcast Networks: DMIF based architectures
KR20130040148A (ko) Mmt 페이로드 설정 방법 및 전송 방법
Wang et al. RFC 6184: RTP Payload Format for H. 264 Video
WO2014061925A1 (fr) Procédé pour transmettre de manière adaptative des données de parité fec au moyen de l'optimisation inter-couches

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 06761575

Country of ref document: EP

Kind code of ref document: A1