WO2008069503A1

WO2008069503A1 - Apparatus and method for dynamically processing scalable information in scalable video coding

Info

Publication number: WO2008069503A1
Application number: PCT/KR2007/006149
Authority: WO
Inventors: Se-Yoon Jeong; Kyung-Ae Moon; Jin-Woo Hong
Original assignee: Electronics And Telecommunications Research Institute
Priority date: 2006-12-04
Filing date: 2007-11-30
Publication date: 2008-06-12

Abstract

Provided are an apparatus and method for dynamically processing scalable information in scalable video coding. The apparatus includes an input unit configured to receive a message and a scalable video coding (SVC) stream; a level shifting unit configured to shift a scalability level of the received SVC stream according to an extraction parameter; a message updating unit configured to update the received message based on information on the shifted scalability level; and a transmitting unit configured to transmit the level-shifted SVC stream and the updated message.

Description

APPARATUS AND METHOD FOR DYNAMICALLY PROCESSING SCALABLE INFORMATION IN SCALABLE

VIDEO CODING

Technical Field

[1] The present invention relates to an apparatus and method for dynamically processing scalable information in scalable video coding; and, more particularly, to an apparatus and method for dynamically processing scalable information in scalable video coding, which can provide a streaming service more efficiently by updating scalable information of a stream dynamically changing in a bitstream extracting process during a scalable video streaming service.

[2] This work was supported by the IT R&D program for MIC/IITA [2005-S-403-02,

"Development of Super-intelligent Multimedia Anytime- anywhere Realistic TV (SmarTV) Technology"].

[3]

Background Art

[4] According to a scalable video coding (SVC) standard based on H.264/advanced video coding (AVC), scalability information of a coding stream is generated only in a coding process and is used in a bitstream extractor. That is, the scalability information is recorded in a scalable information supplemental enhancement information (SI-SEI) message and is used by the bitstream extractor that selectively extracts/processes the bitstream according to the scalability required by a terminal or network.

[5] Generally, the SI-SEI message is generated in a coding process and is used only in an extracting process of the bitstream extractor. Thus, the bitstream extractor does not update the scalability information changed after the extracting/processing process. That is, the scalability information changed after the extracting/processing process is not recorded in the SI-SEI message.

[6] A further description will be made below with reference to Fig. 1.

[7] Fig. 1 is a block diagram illustrating a conventional method for dynamically processing scalable information of a scalable video stream in a single network environment.

[8] Referring to Fig. 1, an SVC extractor 12 of an SVC streaming service system receives an SVC stream and an SI-SEI message from an SVC server 11 in a single network environment. Thereafter, the SVC extractor 12 changes scalability of the SVC stream according to extraction parameters based on the network environment, and transmits the changed scalability information to a client via a network. [9] The SVC extractor 12 does not record the changed scalability information in the SI-

SEI message.

[10] Therefore, the streaming service cannot be efficiently provided in an environment such as a heterogeneous network, which requires a different extracting process according to a network status or traffic.

[11] In other words, in the heterogeneous network where a variety of networks are located between the SVC server 11 and the client, the extracting process needs to be performed several times because the extraction parameters are different according to each network status. In this case, the SVC streaming service cannot be efficiently provided because the scalability information changed in each extracting process is not recorded.

[12] A further description will be made below with reference to Fig. 2.

[13] Fig. 2 is a block diagram illustrating a conventional method for dynamically processing scalable information of a scalable video stream in a heterogeneous network.

[14] Referring to Fig. 2, when no network error exists, an SVC stream outputted from a first SVC extractor 21 and passing through a network is identical to an output of the first SVC extractor 21, but SVC stream has scalability different from an SVC stream of an SVC server 20. This is because the first SVC extractor 21 excludes streams higher than the scalability according to extraction parameters.

[15] However, since the information is not updated in an SI-SEI message during the extracting process, the information generated in an initial coding process is maintained intact. Further, the information that is not updated is used in a second SVC extractor 23 for a second network 24 and also used in following SVC extractors for other networks.

[16] Therefore, the SVC streaming service is provided in a very inefficient way.

[17] As one example, when an SVC stream is received from a server through a streaming in a P2P-like environment and then is provided to another terminal through a streaming service, the received stream and the scalability information of an SI-SEI message may not be completely identical to each other due to network problems. In this case, incorrect stream information may be provided to the terminal even though there is no error in the streaming service itself.

[18] As another example, since network status varies with time, the SVC extractor applies the scalability that dynamically changes according to the variation of the network status in the SVC streaming service. That is, since different extraction parameters are applied according to the network status, the scalability level of the SVC stream is dynamically changed.

[19] Since the scalability information of the actual stream is not contained in the SI-SEI message and the actual stream contains unnecessary information, the conventional apparatus and method are inefficient in specific applications. Disclosure of Invention

Technical Problem

[21] An embodiment of the present invention is directed to providing an apparatus and method for dynamically processing scalable information in scalable video coding, which can provide a streaming service more efficiently by updating scalable information of a stream dynamically changing in a bitstream extracting process during a scalable video streaming service.

[22] Other objects and advantages of the present invention can be understood by the following description, and become apparent with reference to the embodiments of the present invention. Also, it is obvious to those skilled in the art of the present invention that the objects and advantages of the present invention can be realized by the means as claimed and combinations thereof.

[23]

Technical Solution

[24] In accordance with an aspect of the present invention, there is provided an apparatus for dynamically processing scalable information of a scalable video stream, including: an input unit configured to receive a message and a scalable video coding (SVC) stream; a level shifting unit configured to shift a scalability level of the received SVC stream according to an extraction parameter; a message updating unit configured to update the received message based on information on the shifted scalability level; and a transmitting unit configured to transmit the level-shifted SVC stream and the updated message.

[25] In accordance with an aspect of the present invention, there is provided a method for dynamically processing scalable information of a scalable video stream, including: receiving a message and an SVC stream; shifting a scalability level of the received SVC stream according to an extraction parameter; updating the received message based on information on the shifted scalability level; and transmitting the level-shifted SVC stream and the updated message.

Advantageous Effects

[26] In accordance with embodiments of the present invention, a streaming service can be provided more efficiently by updating scalable information of a stream dynamically changing in a bitstream extracting process during a scalable video streaming service.

[27]

Brief Description of the Drawings

[28] Fig. 1 is a block diagram illustrating a conventional method for dynamically processing scalable information of a scalable video stream in a single network environment. [29] Fig. 2 is a block diagram illustrating a conventional method for dynamically processing scalable information of a scalable video stream in a heterogeneous network.

[30] Fig. 3 is a block diagram illustrating a method for dynamically processing scalable information of a scalable video stream in accordance with an embodiment of the present invention.

[31] Fig. 4 is a block diagram illustrating an apparatus for dynamically processing scalable information of a scalable video stream in accordance with an embodiment of the present invention.

[32] Fig. 5 is a flowchart illustrating a method for dynamically processing scalable information of a scalable video stream in accordance with an embodiment of the present invention.

[33] Fig. 6 is a block diagram illustrating a streaming service supporting a multipath in accordance with an embodiment of the present invention.

[34]

Best Mode for Carrying Out the Invention

[35] The advantages, features and aspects of the invention will become apparent from the following description of the embodiments with reference to the accompanying drawings, which is set forth hereinafter. Therefore, those skilled in the field of this art of the present invention can embody the technological concept and scope of the invention easily. In addition, if it is considered that detailed description on a related art may obscure the points of the present invention, the detailed description will not be provided herein. The preferred embodiments of the present invention will be described in detail hereinafter with reference to the attached drawings.

[36] Fig. 3 is a block diagram illustrating a method for dynamically processing scalable information of a scalable video stream in accordance with an embodiment of the present invention.

[37] Referring to Fig. 3, an SVC extractor 32 receives an SI-SEI message and an SVC stream from an SVC server 31.

[38] The SVC extractor 32 changes a scalability level of the SVC stream according to extraction parameters and updates the SI-SEI message using the changed scalability information.

[39] An input unit of the SVC stream is an access unit (AU). The SVC stream is formed of layers. The AU includes data of the layers having the same time. Data of each layer are expressed as a network abstraction layer (NAL) header, and layer information is recorded in the NAL header.

[40] In addition, the scalability information of the SVC stream can be exactly expressed when the SI-SEI message is updated on the basis of the AU. However, this results in the increase of data.

[41] Therefore, it is preferable to update the SI-SEI message only for instantaneous decoding refresh (IDR) AUs rather than for all received AUs. The IDR AUs represent AUs with changed scalability level. Since the IDR is a portion where a random access is possible, a portion ranging from the IDR to the front of a next IDR is a minimum unit of a decodable stream.

[42] As a result, it is preferable to output the SI-SEI message in each decodable IDR period.

[43] Fig. 4 is a block diagram illustrating an apparatus for dynamically processing scalable information of a scalable video stream in accordance with an embodiment of the present invention.

[44] Referring to Fig. 4, the apparatus, e.g., an SVC extractor, includes an input unit 41, a level shifting unit 42, a message updating unit 43, and a transmitting unit 44. The input unit 41 receives an SI-SEI message and an AU-based SVC stream from an SVC server. The level shifting unit 42 shifts a scalability level of the received SVC stream according to extraction parameters. The message updating unit 43 updates the SI-SEI message by using the information about the changed scalability level. The transmitting unit 44 transmits the level-shifted SVC stream and the updated SI-SEI message to a terminal or another SVC extractor.

[45] The message updating unit 43 updates the SI-SEI message on the basis of IDR unit.

[46] Further, the message updating unit 43 uses a "valid_flag[i]" field to record whether the SI-SEI message is updated or not.

[47] The transmitting unit 44 transmits the SVC stream, whose scalability level is shifted by the level shifting unit 42, to the terminal or another SVC extractor, and transmits the SI-SEI message, which is updated by the message updating unit 43, to the terminal or another SVC extractor on the basis of IDR unit.

[48] Fig. 5 is a flowchart illustrating a method for dynamically processing scalable information of a scalable video stream in accordance with an embodiment of the present invention.

[49] Referring to Fig. 5, an SI-SEI message and an AU-based SVC stream are received from an SVC server in step S501.

[50] In step S502, a scalability level of the SVC stream is shifted according to extraction parameters. The process of shifting the scalability level of the SVC stream is identical to a process that is performed in an SVC reference software.

[51] In step S503, the SI-SEI message is updated using scalability information that is changed according the shifted scalability level.

[52] In step S504, the level-shifted SVC stream and the updated SI-SEI message are transmitted to a terminal or another SVC extractor. [53] The SI-SEI message is given as the following Table 1.

[54]

[55] Table 1

[Table 1] [Table ]

[56]

[57] The SI-SEI message is generated one time in the SVC coding process of Joint

Scalable Video coding Model (JSVM) software, and is recorded one time in front of the SVC stream.

[58] In accordance with the embodiment of the present invention, the SI-SEI message is updated on the basis of IDR unit. The first SVC extractor updates the received SI-SEI message on the basis of IDR unit and maintains the updated information. The second SVC extractor receives the SI-SEI message on the basis of IDR unit because the first SVC extractor transmits the SI-SEI message on the basis of IDR unit.

[59] That is, the SI-SEI information is used if it exists in the IDR AU. If the SI-SEI information does not exist in the IDR AU, previous SI-SEI information is updated and attached in front of the IDR AU.

[60] In the process of updating the SI-SEI message, the determination whether each layer is extracted or discarded is updated by a "valid_flag" value.

[61] That is, an i-th layer in which valid_flag[i] is 1 means that the SI-SEI message passes through the SVC extractor and transmits to a next stage of a network. An i-th layer in which valid_flag[i] is 0 means that the SI-SEI message does not pass through the SVC extractor and thus does not transmit to a next stage.

[62] The received extraction parameters determine whether the SVC extractor outputs each layer. The process of extracting NAL data of each layer of the AU is performed in the same way as JSVM Extractor software.

[63] Fig. 6 is a block diagram illustrating a streaming service supporting a multipath in accordance with an embodiment of the present invention.

[64] Referring to Fig. 6, a server 61 can provide a stream having three type of spatial scalability in picture size: 4CIF, CIF and QCIF. Thus, a router A 62 can receive QCIF+CIF data from the server 61 and transmit the QCIF+CIF data to the terminal 64 and a router C 65. At this point, a router B 63 receives only 4CIF data from the server 61.

[65] This is an example in which paid data are transmitted to another path when 4CIF is transmitted to only an authorized user in a charged service.

[66] The router B 63 transmits only 4CIF data to a terminal 64 of a charged service user, and transmits only 4CIF data to the router C 65, which is a private router of a group charged service subscriber. The private subscriber terminal 64 uses the service by receiving free-of-charge data from the router A 62 and receiving paid 4CIF data from the router B 63. The group subscriber terminal 66 receives all data from the private router C 65.

[67] The service of Fig. 6 can be easily and conveniently provided by updating the SI-SEI message. That is, the SI-SEI message exactly describes the case that data are outputted based on specific streams as illustrated in Fig. 6, as well as the case that the SVC extractor outputs the scalability lower than the condition. Thus, the present invention can be easily applied to the service supporting the path of Fig. 6 or more complex multipath.

[68] As described above, the technology of the present invention can be realized as a program and stored in a computer-readable recording medium, such as CD-ROM, RAM, ROM, floppy disk, hard disk and magneto-optical disk. Since the process can be easily implemented by those skilled in the art of the present invention, further description will not be provided herein.

[69] The present application contains subject matter related to Korean Patent Application

Nos. 2006-0121252 and 2007-0040510, filed in the Korean Intellectual Property Office on December 4, 2006 and April 25, 2007, respectively, the entire contents of which is incorporated herein by reference.

[70] While the present invention has been described with respect to certain preferred embodiments, it will be apparent to those skilled in the art that various changes and modifications may be made without departing from the scope of the invention as defined in the following claims.

Claims

[1] An apparatus for dynamically processing scalable information of a scalable video stream, comprising: an input unit configured to receive a message and a scalable video coding (SVC) stream; a level shifting unit configured to shift a scalability level of the received SVC stream according to an extraction parameter; a message updating unit configured to update the received message based on information on the shifted scalability level; and a transmitting unit configured to transmit the level-shifted SVC stream and the updated message.

[2] The apparatus of claim 1, wherein the input unit receives a scalable information- supplemental enhancement information (SI-SEI) message from an external SVC server, and receives the SVC stream based on access unit (AU). [3] The apparatus of claim 2, wherein the message updating unit updates the SI-SEI message on the basis of instantaneous decoding refresh (IDR) unit. [4] The apparatus of claim 3, wherein the message updating unit records whether the

SI-SEI message is updated or not in a valid_flag[i] field. [5] The apparatus of claim 3, wherein the transmitting unit transmits the SI-SEI message on the basis of the IDR unit. [6] A method for dynamically processing scalable information of a scalable video stream, comprising: receiving a message and an SVC stream; shifting a scalability level of the received SVC stream according to an extraction parameter; updating the received message based on information on the shifted scalability level; and transmitting the level-shifted SVC stream and the updated message. [7] The method of claim 6, wherein the step of receiving the message and the SVC stream includes the steps of: receiving a scalable information- supplemental enhancement information

(SI-SEI) message from an external SVC server; and receiving the SVC stream on the basis of access unit (AU). [8] The method of claim 7, wherein the step of updating the receiving message includes the step of: updating the SI-SEI message on the basis of instantaneous decoding refresh

(IDR) unit, and the step of transmitting the level-shifted SVC stream includes the step of: transmitting the SI-SEI message on the basis of the IDR unit.