WO2012094975A1

WO2012094975A1 - Method and device for transmitting and receiving multimedia data

Info

Publication number: WO2012094975A1
Application number: PCT/CN2012/070161
Authority: WO
Inventors: 赵宇; 王芳; 孙健; 刘继年; 李加周
Original assignee: 中兴通讯股份有限公司
Priority date: 2011-01-11
Filing date: 2012-01-10
Publication date: 2012-07-19
Also published as: CN102595203A

Abstract

Provided are a method and device for transmitting and receiving multimedia data, relating to the multimedia field. The method for transmitting multimedia data disclosed in the present invention includes: a media forwarding server generates descriptive information about each rendering code stream included in a Scalable Video Coding (SVC) code stream to be transmitted and the correlation identified by the spatial time quality (DTQ) according to the Supplemental Enhancement Information (SEI) in the SVC code stream; and the media forwarding server transmits the generated descriptive information about each rendering code stream and the correlation identified by DTQ while or before transmitting the SVC code stream to a multimedia client. The method enables the receiver to simplify the multimedia data processing after receiving the multimedia data.

Description

Method for transmitting and receiving multimedia data and transmission and receiving device thereof

Technical field

The invention relates to the field of multimedia, in particular to a method for transmitting and receiving multimedia data and a transmission and receiving device thereof. Background technique

With the development of coding technology, in the typical application system of streaming media, more and more video formats are supported. Among them, the code stream of H264/AVC (Advanced Video Coding) format is widely used. And based on the H264/AVC code stream, SVC (Scalable Video Coding) has been developed. SVC code streams have spatial domain layering, time domain layering and quality layering. Due to the layering nature of the code stream, SVC brings a new user experience to the user. However, the currently used streaming media protocol protocol specification is almost transparent to the transmission content, and it only provides a channel for streaming media transmission.

For example, the Real-time Transport Protocol (RTP) is an example. The packet structure includes two parts, a protocol header and a payload part. The protocol header includes the payload media type, packet sequence number, timestamp, synchronization source identifier, and so on. The payload portion is typically a simple sequential storage of multimedia data. Another common streaming media transfer protocol, such as the MPEG (Moving Pictures Experts Group) TS (Transport Stream) packet, is the same for its packet structure. Summary of the invention

For SVC and MVC streams, the client cannot obtain any auxiliary information from the packet header when extracting a subset of them in the transport stream. In the current standard specification, the current MPEG-2 TS bears the SVC stream. Only one substream corresponding to one airspace hierarchy is determined, as shown in Figure 1. There is no further subdivision of the quality and time domain grading in the same airspace grading, which makes it easy to find the code stream corresponding to a certain airspace grading by PID (program identification), but in the airspace grading, I want to find a certain Quality grading or time domain grading is difficult, and can only be obtained by traversing the load and comparing the NALU (Network Abstraction Layer Unit) header. This problem also occurs when MPEG2-TS is used as a storage file. When the client frequently switches the attributes of the viewing stream, for example, frequently switches the size of the viewing stream (spatial grading), frame rate (time domain grading). When MPEG2-TS is used as a file, it is not possible to efficiently extract the required code stream from the file. And when the user extracts the transmitted code stream, it must parse to SEI (Supplemental Enhancement Information) to determine the DTQ (Space Time Quality) corresponding to the code stream to be extracted. The SEI does not necessarily exist in the transport stream.

The invention provides a method for transmitting and receiving multimedia data and a transmission and receiving device thereof. One of the objectives of the present invention is to provide information of each presentation stream in a multimedia encoded stream and its corresponding identification to simplify processing of multimedia data by the receiving end.

To this end, the present invention discloses a method for transmitting multimedia data, including:

And the media forwarding server generates, according to the additional information (SEI) in the layerable video coding (SVC) code stream to be sent, description information of each presentation code stream included in the SVC code stream, and a spatial time quality (DTQ) identifier. Correspondence relationship

And sending, by the media forwarding server, the generated description information of each presentation code stream and the correspondence between the DTQ identifiers before sending the SVC code stream to the multimedia client or before transmitting the SVC code stream.

Preferably, the method further includes: after the multimedia client receives the SVC code stream, determining, according to the received description information of each of the presentation code streams and the correspondence between the DTQ identifiers, that the client supports playing Deriving the DTQ identifier corresponding to the description information of the code stream, traversing the SVC code stream to obtain the display code stream corresponding to the determined DTQ identifier for decoding display.

Preferably, the above method further includes:

Before the media forwarding server sends the SVC code stream to the multimedia client, the original code stream of the SVC code stream is divided into one or more spatial grading code stream segments according to spatial grading, according to time domain grading and quality grading. In a hierarchical manner, each spatial grading code stream segment is subjected to secondary grading, and each secondary grading code stream segment is hierarchically classified three times according to another hierarchical manner, to be divided into different presentation code streams, and each presentation is performed. The code streams correspond to a DTQ identifier.

Preferably, the multimedia client receives the description information of each presentation code stream and the correspondence between the DTQ identifiers, and determines a DTQ label corresponding to the description information of the presentation code stream supported by the client. Obtaining, the selected code stream corresponding to the determined DTQ identifier is selected from the received SVC code stream for decoding display.

It is yet another object of the present invention to achieve an ordered layering of a multimedia code stream.

The media forwarding server divides the original code stream of the received SVC code stream into one or more spatial grading code stream segments according to spatial grading, and respectively classifies each space according to a hierarchical manner in time domain grading and quality grading. The flow segment is subjected to secondary grading, and the code segments of each secondary grading are respectively classified into three times according to another grading manner, and are divided into different presentation code streams, and each of the presentation code streams respectively corresponds to one DTQ identifier; the media forwarding The server sends the SVC code stream that has been hierarchically graded as needed.

Preferably, in the above method, the multimedia client receives the SVC code stream, according to the

The SEI in the SVC code stream generates description information of each presentation code stream included in the SVC code stream and a correspondence relationship between the DTQ identifiers, and determines a DTQ identifier corresponding to the description information of the presentation code stream supported by the client, from the The display code stream corresponding to the determined DTQ identifier is selected and decoded in the SVC code stream.

Preferably, the method further includes: before the sending, by the media forwarding server, the description information of each presentation stream included in the SVC code stream according to the SEI in the SVC code stream before sending the SVC code stream. And the corresponding relationship between the DTQ identifiers, and the corresponding description information of the generated presentation code streams and the spatial temporal quality DTQ identifiers are sent before the SVC code stream is sent or before the SVC code stream is sent.

Preferably, the multimedia client receives the description information of each of the presentation code streams and the correspondence between the DTQ identifiers, and determines a DTQ identifier corresponding to the description information of the presentation code stream supported by the client, from the received SVC. The code stream corresponding to the determined DTQ identifier is selected and decoded for display.

Still another object of the present invention is to simplify the data processing operation when the multimedia client receives the information of each of the presentation streams in the multimedia encoded stream and its corresponding identifier.

To this end, the present invention discloses a method for receiving multimedia data, including:

When receiving the SVC code stream, the multimedia client determines, from the received description information of each code stream in the SVC code stream and the correspondence between the DTQ identifiers, the presentation code supported by the client. The DTQ identifier corresponding to the description information of the stream is traversed to the SVC code stream to obtain the display code stream corresponding to the determined DTQ identifier for decoding display.

Another object of the present invention is to improve the efficiency of a multimedia client in extracting a multimedia code stream.

To this end, the present invention discloses a method for receiving multimedia data, including: when a multimedia client receives an SVC code stream, determining the present information from the description information of each code stream and the correspondence between the DTQ identifiers in the SVC code stream. The client supports the DTQ identifier corresponding to the description information of the displayed presentation stream, and selects the presentation stream corresponding to the determined DTQ identifier from the SVC stream to perform decoding and display.

Preferably, in the foregoing method, the multimedia client determines, according to the description information of each code stream in the SVC code stream and the correspondence between the DTQ identifiers, the DTQ corresponding to the description information of the presentation code stream supported by the client. The process of identification is as follows:

The multimedia client generates, according to the additional information SEI in the received SVC code stream, the description information of each presentation code stream included in the SVC code stream and the correspondence between the DTQ identifiers, and determines that the client supports the playback. The DTQ identifier corresponding to the description information of the code stream is displayed.

Preferably, the multimedia client receives the description information of each presentation stream in the SVC stream sent by the multimedia forwarding server and the correspondence between the DTQ identifiers, and determines the description information of the presentation stream supported by the client. Corresponding DTQ identifier.

Still another object of the present invention is to provide a transmission apparatus that can provide information of each presentation stream in a multimedia encoded stream and its corresponding identification.

To this end, the present invention discloses a multimedia data transmission device, including:

a first module, configured to: generate, according to an SEI in the SVC code stream to be sent, description information of each presentation code stream included in the SVC code stream and a correspondence relationship between the DTQ identifiers;

And a second module, configured to: after sending the SVC code stream to the multimedia client or before sending the SVC code stream, send the generated description information of each presentation code stream and a correspondence relationship of the DTQ identifier.

Preferably, the foregoing transmission device further includes: a third module, configured to: divide the original code stream of the SVC code stream into one or more spatial grading code stream segments according to spatial grading, according to time domain grading and quality grading a grading method to separately classify each spatial grading code stream segment, according to another The grading mode divides the code streams of each of the two gradings into three times, and divides them into different presentation streams, and sends them to the multimedia client, where each of the presentation streams respectively corresponds to one DTQ identifier. Still another object of the present invention is to provide a transmission device that can implement ordered layering of a multimedia code stream.

a first module, configured to: divide the original code stream of the SVC code stream into one or more spatial grading code stream segments according to a spatial grading;

The second module is configured to: separately classify each spatial grading code stream segment according to a grading manner in time domain grading and quality grading, and separately code each code segment of the second grading according to another grading manner Perform three times of grading to divide into different presentation streams and send them to the multimedia client, where each presentation stream corresponds to a DTQ identifier.

Preferably, the foregoing transmission device further includes: a third module, configured to: generate, according to the additional addition information SEI in the SVC code stream, description information of each presentation code stream included in the SVC code stream, and correspondence of the DTQ identifier And the relationship between the description information of each generated presentation stream and the DTQ identifier is sent before the SVC code stream is sent to the multimedia client or before the SVC code stream is sent.

Still another object of the present invention is to provide a receiving device for multimedia data which simplifies data processing operations.

To this end, the present invention discloses a receiving device for multimedia data, including:

The first module is configured to: receive the SVC code stream, and determine, from the received description information of each code stream in the SVC code stream and the correspondence between the DTQ identifiers, the presentation code stream supported by the client to play Describe the DTQ identifier corresponding to the information;

And a second module, configured to: traverse the SVC code stream to obtain a decoded display stream corresponding to the determined DTQ identifier.

Another object of the present invention is to provide a receiving device for extracting multimedia data with high efficiency of a multimedia code stream.

To this end, the present invention discloses a receiving device for multimedia data, including: a first module, configured to: receive an SVC code stream, and determine, according to the description information of each code stream in the SVC code stream and the correspondence between the DTQ identifiers, a DTQ corresponding to the description information of the presentation code stream supported by the client. Identification

And a second module, configured to: select a presentation code stream corresponding to the determined DTQ identifier from the SVC code stream for decoding display.

Preferably, in the foregoing device, the first module is further configured to: generate, according to the SEI in the received SVC code stream, description information of each presentation code stream included in the SVC code stream, and a correspondence between DTQ identifiers And determining, from the DTQ identifier corresponding to the description information of the presentation stream supported by the client.

Preferably, in the foregoing device, the first module is further configured to: receive the description information of each presentation code stream in the SVC code stream sent by the multimedia forwarding server, and the correspondence between the DTQ identifiers, and determine the client support The DTQ identifier corresponding to the description information of the displayed presentation stream.

One embodiment of the present invention reorders existing SVC code streams to provide regular storage, i.e., provides an ordered hierarchical multimedia code stream. Therefore, the server or the client can conveniently find the required code stream when the code stream is extracted, switched, etc., and the response speed of the server or the client is improved. Still another embodiment of the present invention provides information of each presentation stream in the SVC code stream and its corresponding DTQ identifier, thereby simplifying processing of the multimedia data by the receiving end side. BRIEF abstract

1 is a schematic structural diagram of a prior art MPEG2-TS bearer SVC;

2 is a flowchart of transmitting multimedia data in Embodiment 3 of the present invention;

3 is a schematic structural diagram of an MPEG2-TS bearer SVC according to Embodiment 3 of the present invention; FIG. 4 is a schematic diagram of a SVC code stream multicast networking in Embodiment 4 of the present invention;

5 is a schematic diagram of MPEG2-TS bearer SVC code stream substream division according to Embodiment 4 of the present invention; FIG. 6 is a flowchart of a client extracting a substream according to Embodiment 4 of the present invention. Preferred embodiment of the invention The technical solutions of the present invention are further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, in the case of no conflict, the features in the embodiments and the embodiments in the present application may be arbitrarily combined with each other.

Example 1

The applicant of the present invention considers that when the streaming media server sends the media stream, the media stream carries

SVC loads are divided according to airspace, quality, time domain or space, time domain, and quality layer. That is, the outermost level is the airspace rating. Quality and time domain grading are not in order. When the media forwarding server or the client parses the SVC payload, the code stream is selected according to these ratings, thereby achieving personalized viewing. If the SVC load in the SVC stream sent by the media forwarding server is directly sorted according to the airspace, quality, time domain or space, time domain, and quality layer, the client of the SVC does not need to traverse the entire load. Determine the DTQ corresponding to the code stream to be extracted.

Therefore, based on the above idea, the embodiment provides a multimedia data transmission process, in which the media forwarding server divides the original code stream of the existing SVC code stream into one or more spatial grading code streams according to spatial grading. Segments, according to a hierarchical manner in time domain grading and quality grading, respectively categorize each spatial grading code stream segment, and classify each second grading code stream segment three times according to another grading manner. For different presentation streams, each presentation stream corresponds to one DTQ identifier; the media forwarding server sends the three-stage SVC stream as needed.

The media forwarding server sends the SVC code stream after three times of classification according to the requirement: When the media forwarding server sends the multicast mode, the three-level SVC code stream is sent as a load; the media forwarding server uses the unicast mode. When transmitting, the D (space class grading) value, the T (time domain grading) value, and the Q (quality grading) value of the presentation code stream required for the unicast transmission are selected from the three-stage SVC code stream. The corresponding spatial domain grading code stream and its dependent code stream are found according to the D value. In the air stream of the air domain hierarchy and its dependent code stream, the corresponding time domain hierarchical code stream and its dependent code stream are found according to the T value. In the found time domain grading code stream and its dependent code stream, the corresponding quality grading code stream and its dependent code stream are found according to the Q value. Finally, the media forwarding server sends the selected presentation stream.

At this time, when receiving the SVC code stream, the multimedia client first determines the DTQ identifier corresponding to the description information of the presentation code stream supported by the client in the SVC code stream, and selects the determined DTQ identifier from the SVC code stream. The corresponding presentation code stream is decoded and displayed. The multimedia client may generate, according to the SEI in the SVC code stream, description information of each presentation code stream included in the SVC code stream and a correspondence relationship between the DTQ identifiers, thereby determining a description of the presentation code stream supported by the client. The DTQ identifier corresponding to the information.

Example 2

The applicant of the present invention further proposes a method for transmitting multimedia data, including:

Before the SVC code stream is sent, the media forwarding server generates, according to the SEI in the SVC code stream, the description information of each presentation code stream included in the SVC code stream and the correspondence between the DTQ identifiers; the media forwarding server needs to send the SVC code. At the time of streaming or before transmitting the SVC code stream, the generated description information of each presentation code stream and the correspondence relationship of the DTQ identifier are transmitted.

In this way, when the multimedia client can receive the description information of each of the foregoing presentation streams and the correspondence between the DTQ identifiers, the DTQ identifier corresponding to the description information of the presentation stream supported by the client is determined, and the received SVC code is received. The stream of the presentation code corresponding to the determined DTQ identifier is found in the stream for decoding and display.

Example 3

This embodiment provides a preferred solution, which combines the technical means of the foregoing Embodiments 1 and 2 to perform multimedia data transmission. The specific transmission process is as shown in FIG. 2, and includes the following process:

Step 200: The multimedia forwarding server generates a code stream description that is displayed and a correspondence relationship with the DTQ, that is, an overall description information of the SVC code stream.

Specifically, the operation process of this step is as follows:

The multimedia forwarding server traverses the SVC code stream to find the SEI, parses the SEI, and obtains the total number of presentation streams that the SVC code stream can provide; parses the SEI to generate description information for each presentation stream; parses the SEI, and obtains the DTQ corresponding to each presentation stream. That is, the multimedia forwarding server generates an overall description information of the SVC code stream.

In this embodiment, the multimedia data of the MPEG2-TS is taken as an example, and the SVC code stream overall description information svc_total_descriptor is added to the PMT, which can be defined as a program level information, which can be defined as follows:

Table 1 is the svc_total_descriptor definition table

Syntax ( Bytes) Mnemonic Svc— total— descriptorO

{ uimsbf

Descriptor — tag 8 uimsbf

Descriptor — length 8 uimsbf

PresentationNum 32 uimsbf

For(i=0; i< PresentationNum; i++)

/

\

DID, 32 uimsbf

TID, 128 bslbf

QID, 32 uimsbf

PresentationDescription

}

32 uimsbf

} In the above table, the meaning of each field is as follows:

PresentationNum: indicates the number of views (presentation streams) in the SVC stream; DID: the identifier of the airspace hierarchy;

TID: the identifier of the time domain hierarchy;

QID: the identification of the quality rating;

PresentationDescription: A description of the presentation stream that is provided to the user for reading and selects the stream to watch based on the description.

Step 201: The multimedia forwarding server generates a layer-level hierarchical SVC code stream for storage. Specifically, the operation process of the step is as follows:

The multimedia forwarding server traverses the SVC code stream generated by the current technology, accesses the extended header of the SVC code stream NALU unit, and divides the original code stream into one or more according to the DID of the spatial hierarchy based on the original code stream of the SVC code stream. DID stream segment. The length and DID of the code stream segment are added before the code stream segment. Performing a second grading on the code streams of different DIDs respectively, traversing the code segment of the spatial domain grading, and dividing the spatially classified code stream into one or more TID codes according to the time domain grading corresponding TID Flow segment. The length and TID of the code stream segment are added before the code stream segment. Performing a third grading on the code stream segments of different TIDs respectively, traversing the code stream segments of the time domain grading, and dividing the code stream of the time domain grading into one or more QID code stream segments according to the QID corresponding to the quality grading. The length and QID of the code stream segment are added to the code stream segment.

Of course, in other scenarios, after the multimedia forwarding server divides the original code stream into one or more DID code stream segments according to the DID corresponding to the spatial hierarchy, the multimedia forwarding server may first perform secondary classification according to the QID corresponding to the quality classification, and the spatially graded code stream. Dividing into one or more QID code stream segments; thereafter, traversing the quality-graded code stream segments, performing a third grading according to the TID corresponding to the time domain grading, and dividing the quality grading code stream into one or more TID code stream segments .

Step 202: When the multimedia forwarding server uses the multicast sending mode (the multimedia forwarding server may also be referred to as a multicast server), the sending operation is performed according to the large and complete SVC code stream of the good level; the multimedia forwarding server uses the unicast In the case of the transmission mode (the multimedia forwarding server may also be referred to as a unicast server), the corresponding code stream segment is selected from the well-classed code streams and then transmitted. The multimedia forwarding server sends the SVC code stream overall description information generated in step 200 at the same time as or before the SVC code stream is sent.

In this step, the SVC code stream generated in step 201 is sent as a load, including the newly added length and other fields, according to the large and complete code stream of the good level. For example, when this step is implemented to implement MPEG2-TS bearer SVC, the structure of the SVC is as shown in FIG. 3, which is consistent with the current SVC standard. On the basis of the substream, it is divided into one or more mass substreams, and based on the mass substream, it is divided into one or more time substreams.

After selecting the corresponding code stream segment from the code stream that is classified into a good level (that is, the presentation code stream of the DTQ identifier after the classification), the sending further includes:

The media forwarding server determines the D (space domain grading) value, the T (time domain grading) value, and the Q (quality grading) value of the presentation code stream to be selected from the SVC code stream overall description information generated by the step 200 as needed. The corresponding spatial domain-classified code stream and its dependent code stream are found according to the D value. In the code stream of the spatial domain classification and its dependent code stream, the corresponding time domain hierarchical code stream and its dependent code stream are found according to the T value. In the found time domain grading code stream and its dependent code stream, the corresponding quality grading code stream and its dependent code stream are found according to the Q value. Finally, the media forwarding server sends the selected presentation stream. Step 203: The multimedia client receives the foregoing SVC code stream, and performs decoding and displaying.

When the multimedia client receives the SVC code stream sent by the multicast mode, it selects and displays the corresponding presentation code stream for decoding and display, and the process is as follows:

The multimedia client determines the D (space domain grading) value, the T (time domain grading) value, and the Q (quality grading) value of the presentation code stream to be selected according to the received SVC code stream overall description information. The corresponding airspace-classified code stream and its dependent code stream are found according to the Q value. In the code stream of the spatial domain grading and the code stream it depends on, the corresponding time domain grading code stream and its dependent code stream are found according to the T value. In the found time domain grading code stream and its dependent code stream, the corresponding quality grading code stream and its dependent code stream are found according to the Q value. The selected code streams are reordered and passed to the decoder for decoding.

In the above process, the SVC code stream overall description information generated by step 200 and the layered SVC code stream generated by step 201 are generated. Just do it once. For the stored hierarchical SVC streams, it does not need to be regenerated.

Example 4

In this embodiment, in the network shown in FIG. 4, the mobile phone accesses the SVC code stream multicast as an example to describe the method for receiving the multimedia data. As shown in FIG. 4, when the multimedia forwarding server multicasts the SVC code stream, the terminal access devices (also referred to as multimedia clients) can be classified into three types, one is a mobile phone with a small screen, and the other is a moderate screen. PDA devices, one type of laptop or television device with a large screen. These three types of devices receive the multicast stream sent by the server. Each device extracts the code streams supported by each of the multicast streams. For example, the mobile phone device extracts the image size from the code stream to QQVGA (1/4 of the size of QVGA), the frame rate is 15 frames/s, and the code stream of high definition image quality. The PDA device extracts an image size of QVGA from the code stream, a frame rate of 15 frames/s, and a high-definition picture quality stream. The computer device extracts the image size from the code stream to VGA, the frame rate is 30 frames/s, and the high-definition picture quality stream. Specifically, as shown in FIG. 5, the multimedia forwarding server sends a stream of PID 201 in the MPEG2-TS code stream carrying the SVC to carry the substream of the QQVGA. A code stream with a PID of 202 carries a substream of QVGA, and a code stream with a PID of 203 carries a substream of VGA. In the code stream of each airspace classification, it is further divided into quality classification and time domain classification. At this point, the process of accessing the SVC multicast by the mobile phone is as shown in FIG. 6, and includes the following steps:

Step 601: The client (that is, the mobile phone) receives the TS data.

Step 602: Determine whether the received TS data is completed. If the receiving is complete, go to step 610. Otherwise skip to step 603.

Step 603: Determine whether the received data is PSI data, and if yes, skip to step 604, otherwise, go to step 605.

Step 604: Parse the PSI data, and obtain a PID corresponding to the QQVGA code stream. Obtain the t value corresponding to 15 frames/second to obtain the q value corresponding to HD.

Step 605, determining whether the PSI data has been received. If yes, then go to step 606, otherwise skip to step 607.

Step 606: Determine whether the PID of the received code stream is the PID of the QQVGA code stream. If yes, go to step 608. Otherwise, go to step 607.

Step 607, discarding the data.

Step 608, parsing the load, skipping the length field, and determining whether the Q field is less than or equal to the q value corresponding to the high definition. If yes, it is judged whether the T field is equal to the t value corresponding to 15 frames/s. If yes, select the codestream. Otherwise, the stream segment is not selected. After that, the code stream segment is skipped, and the code stream is selected by the above judgment method. Until the end of the code stream, then step 609 is performed.

Step 609: Send the selected code stream segment to the decoder.

Step 610, ending.

Example 5

The embodiment provides a multimedia data transmission device, which is built in a media forwarding server or other multimedia network element. Specifically, the device includes:

The first module generates, according to the SEI in the SVC code stream to be sent, the description information of each presentation code stream included in the SVC code stream and the correspondence between the DTQ identifiers;

The second module sends the generated description information of each presentation code stream and the correspondence relationship of the DTQ identifiers before sending the SVC code stream to the multimedia client or before transmitting the SVC code stream.

In a preferred solution, the transmission device may further include a third module, and the original code stream of the SVC code stream is divided into one or more spatial grading code stream segments according to a spatial grading, according to a hierarchical manner in time domain grading and quality grading. The spatial grading code stream segments are separately graded separately, and the code segments of each secondary grading are respectively graded three times according to another grading manner, and are divided into different presentation code streams, and sent to The multimedia client, wherein each of the presentation streams respectively corresponds to a DTQ identifier.

Example 6

The embodiment further provides a multimedia data transmission device, which may also be built in a media forwarding server, or other multimedia network element. The device includes:

The first module divides the original code stream of the layerable video coding SVC code stream into one or more spatial grading code stream segments according to a spatial grading;

In the second module, each spatial grading code stream segment is separately categorized according to one of the time domain grading and the quality grading, and the second grading code stream segments are respectively categorized three times according to another grading manner. Divided into different presentation streams, and sent to the multimedia client, where each presentation stream corresponds to a spatial time quality DTQ identifier.

Preferably, the transmission device further includes: a third module, generating, according to the additional information SEI in the SVC code stream, description information of each presentation code stream included in the SVC code stream, and a correspondence relationship between the DTQ identifiers, And before the multimedia client sends the SVC code stream or sends the SVC code stream, sending the generated description information of each presentation code stream and the correspondence relationship of the DTQ identifier.

Example 7

The embodiment provides a multimedia data receiving device, which can be a multimedia client such as a mobile phone, a PDA device, a computer, and a television. The device can include:

The first module receives the SVC code stream, and determines, from the received description information of each code stream in the SVC code stream and the correspondence between the DTQ identifiers, the description information corresponding to the presentation code stream supported by the client. DTQ logo;

The second module traverses the received SVC code stream to obtain the decoded code stream corresponding to the determined DTQ identifier for decoding display.

In this embodiment, the SVC code stream received by the first module is an SVC code stream that has not been hierarchically sorted. Therefore, the second module needs to traverse the received SVC code stream to obtain the corresponding DTQ identifier corresponding to the display. Code stream.

Example 8

The embodiment provides a receiving device for multimedia data, which may be a mobile phone, a PDA device, or an electric Brain and TV and other multimedia clients. The device can include:

The first module, the SVC code stream is received, and the DTQ identifier corresponding to the description information of the presentation code stream supported by the client is determined from the description information of each code stream in the SVC code stream and the correspondence between the DTQ identifiers;

The first module generates, according to the SEI in the received SVC code stream, the description information of each presentation code stream included in the SVC code stream and the correspondence between the DTQ identifiers, and determines the presentation code stream supported by the client. The description information corresponds to the DTQ identifier. Alternatively, the description information of each presentation stream in the SVC stream sent by the multimedia forwarding server and the correspondence between the DTQ identifiers are received, and the DTQ identifier corresponding to the description information of the presentation stream supported by the client is determined.

The second module selects the presentation code stream corresponding to the determined DTQ identifier from the SVC code stream for decoding and display.

In this embodiment, the SVC code stream received by the first module is the SVC code stream that is hierarchically sorted. Therefore, the second module selects the presentation code stream corresponding to the determined DTQ identifier from the hierarchically sorted SVC code streams. Just fine.

The above description is only the preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes can be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Industrial applicability

One embodiment of the present invention reorders existing SVC code streams to provide regular storage, i.e., provides an ordered hierarchical multimedia code stream. Therefore, the server or the client can conveniently find the required code stream when the code stream is extracted, switched, etc., and the response speed of the server or the client is improved. Still another embodiment of the present invention provides information of each presentation stream in the SVC code stream and its corresponding DTQ identifier, thereby simplifying processing of the multimedia data by the receiving end side.

Claims

Claim

A method for transmitting multimedia data, the method comprising:

The media forwarding server generates, according to the additional enhanced information SEI of the layered video coding SVC code stream to be sent, description information of each presentation code stream included in the SVC code stream and a correspondence relationship between the spatial time quality DTQ identifiers;

2. The method of claim 1, wherein the method further comprises:

After receiving the SVC code stream, the multimedia client determines, according to the received description information of each of the presentation code streams and the DTQ identifier, the description information corresponding to the presentation code stream supported by the client. The DTQ identifier traverses the SVC code stream to obtain a display code stream corresponding to the determined DTQ identifier for decoding display.

3. The method of claim 1, wherein the method further comprises:

Before the media forwarding server sends the SVC code stream to the multimedia client, the

The original code stream of the SVC code stream is divided into one or more spatial grading code stream segments according to spatial grading, and each spatial grading code stream segment is separately graded according to a hierarchical manner in time domain grading and quality grading, according to another A hierarchical manner separately divides the code segments of each secondary grading into three different gradations to be divided into different presentation code streams, and each of the presentation code streams respectively corresponds to one DTQ identifier.

4. The method of claim 3, wherein

Receiving, by the multimedia client, the description information of each of the presentation code streams and the correspondence between the DTQ identifiers, determining a DTQ identifier corresponding to the description information of the presentation code stream supported by the client, and selecting from the received SVC code streams. The display code stream corresponding to the determined DTQ identifier is decoded and displayed.

5. A method for transmitting multimedia data, the method comprising:

The media forwarding server presses the original code stream of the layered video coded SVC code stream received Dividing into one or more spatial grading code stream segments according to spatial grading, respectively classifying each spatial grading code stream segment according to a grading manner in time domain grading and quality grading, respectively, according to another grading manner The second-stage code stream segment is divided into three times to be divided into different presentation code streams, and each of the presentation code streams respectively corresponds to a spatial time quality DTQ identifier;

The media forwarding server sends the SVC code stream that has been hierarchically graded as needed.

6. The method of claim 5, wherein

The multimedia client receives the SVC code stream, and generates, according to the additional information SEI in the SVC code stream, the description information of each presentation code stream included in the SVC code stream and the correspondence between the DTQ identifiers, and determines the client. The DTQ identifier corresponding to the description information of the displayed presentation stream is selected, and the presentation code stream corresponding to the determined DTQ identifier is selected from the SVC code stream for decoding display.

7. The method of claim 5, wherein the method further comprises:

Before the SVC code stream is sent, the media forwarding server generates, according to the SEI in the SVC code stream, the description information of each presentation code stream included in the SVC code stream and the correspondence between the DTQ identifiers, and sends the Before the SVC code stream is sent or before the SVC code stream is sent, the generated description information of each presentation code stream and the correspondence relationship of the DTQ identifier are transmitted.

8. The method of claim 7, wherein

9. A method of receiving multimedia data, the method comprising:

When receiving the splicable video coding SVC code stream, the multimedia client determines, according to the description information of each presentation code stream in the SVC code stream and the correspondence between the spatial time quality DTQ identifiers, determining that the client supports playing. The DTQ identifier corresponding to the description information of the code stream is traversed, and the SVC code stream is traversed to obtain the display code stream corresponding to the determined DTQ identifier for decoding display.

10. A method of receiving multimedia data, the method comprising:

When the multimedia client receives the layered video coded SVC code stream, determining, from the description information of each code stream in the SVC code stream and the correspondence between the spatial time quality DTQ identifiers, determining the presentation code stream supported by the client. Decoding the DTQ identifier corresponding to the information, and selecting a presentation code stream corresponding to the determined DTQ identifier from the SVC code stream for decoding display.

11. The method of claim 10, wherein

The process of determining, by the multimedia client, the DTQ identifier corresponding to the description information of the presentation stream supported by the client from the description information of the presentation stream and the correspondence between the DTQ identifiers in the SVC code stream is as follows:

12. The method of claim 10, wherein

The multimedia client determines, from the description information of each code stream in the SVC code stream and the correspondence between the DTQ identifiers, the DTQ identifier corresponding to the description information of the presentation code stream supported by the client:

Receiving, by the multimedia client, the description information of each presentation stream in the SVC code stream and the correspondence between the DTQ identifiers, and determining the DTQ identifier corresponding to the description information of the presentation stream supported by the client. .

13. A multimedia data transmission device, the transmission device comprising:

a first module, configured to: generate, according to the additional addition information SEI in the layered video coded SVC code stream to be sent, description information of each presentation code stream included in the SVC code stream and a correspondence of a spatial time quality DTQ identifier Relationship

And a second module, configured to: before sending the SVC code stream to the multimedia client or before sending the SVC code stream, send the generated description information of each presentation code stream and a correspondence relationship of the DTQ identifier.

The transmission device according to claim 13, wherein the transmission device further comprises: a third module, configured to: divide the original code stream of the SVC code stream into one or more spatial classification code streams according to a spatial classification Segments, according to a hierarchical manner in time domain grading and quality grading, respectively, each spatial grading code stream segment is subjected to secondary grading, and according to another grading manner, each secondary grading code stream segment is hierarchically divided into three. The different presentation streams are sent to the multimedia client, where each presentation stream corresponds to a DTQ identifier.

15. A multimedia data transmission device, the transmission device comprising:

a first module, configured to: divide the original code stream of the layerable video coding SVC code stream into one or more spatial grading code stream segments according to a spatial grading;

The second module is configured to: separately classify each spatial grading code stream segment according to a grading manner in time domain grading and quality grading, and separately code each code segment of the second grading according to another grading manner The three-level grading is divided into different presentation code streams and sent to the multimedia client, where each presentation code stream corresponds to a spatial time quality DTQ identifier.

The transmission device according to claim 15, wherein the transmission device further comprises: a third module, configured to: generate the information according to the additional added information SEI in the SVC code stream

Transmitting the description information of each presentation code stream included in the SVC code stream and the correspondence between the DTQ identifiers, and transmitting the generated presentation codes before transmitting the SVC code stream to the multimedia client or before transmitting the SVC code stream. The description of the flow and the correspondence between the DTQ identifiers.

17. A receiving device for multimedia data, the device comprising:

a first module, configured to: receive a layerable video encoded SVC code stream, from the received

Determining, in the SVC code stream, the description information of each code stream and the correspondence between the spatial time quality DTQ identifiers, determining a DTQ identifier corresponding to the description information of the presentation code stream supported by the client;

18. A receiving device for multimedia data, the device comprising:

a first module, configured to: receive a layered video encoded SVC code stream, from the SVC code stream Determining, in the correspondence between the description information of the code stream and the spatial time quality DTQ identifier, determining a DTQ identifier corresponding to the description information of the presentation code stream supported by the client;

19. The apparatus according to claim 18, wherein

The first module is further configured to: generate, according to the added additional information SEI in the received SVC code stream, description information of each presentation code stream included in the SVC code stream and a correspondence relationship of the DTQ identifier, and determine the customer from the The DTQ identifier corresponding to the description information of the presentation code stream supported by the terminal is supported.

20. The apparatus according to claim 18, wherein

The first module is further configured to: receive the description information of each presentation code stream in the SVC code stream sent by the multimedia forwarding server, and the correspondence between the DTQ identifiers, and determine the description information of the presentation code stream that the client supports to play. Corresponding DTQ identifier.