JP2014509114A - Bitstream subset instructions - Google Patents

Abstract

  A method is provided for indicating a bitstream subset within a video bitstream (210). The method includes receiving a bitstream, dividing the bitstream into video packets (211 to 216) each containing either video data or supplemental information, and each with a single subset identifier (stream_id). Marking the packet. Each subset identifier is associated with a corresponding bitstream subset (221-223). Further provided is a method for extracting video packets from a video bitstream. The method comprises providing a relevant subset identifier, receiving a video packet from the bitstream, and examining the subset identifier of the packet for each received video packet. A packet is extracted if the subset identifier matches one of the relevant subset identifiers. This can summarize the properties of the bitstream subset into a single identifier, simplifying the processing of video packets within the network and on the client side. Furthermore, an apparatus corresponding to the above-described method is provided.

Description

  The present invention relates to a method and apparatus for indicating a bitstream subset in a compressed video bitstream, and a method and apparatus for extracting video packets from a compressed video bitstream. The invention further relates to a computer program and a computer program product.

  H. also known as MPEG-4 / Advanced Video Coding (AVC). H.264 is the latest video coding standard. This is a composite codec that utilizes the removal of both redundancy within each video frame and redundancy between frames. The output of the encoding process is video coding layer (VCL) data, which is further encapsulated in a network abstraction layer (NAL) unit before transmission or storage. In addition to the video data, other data that can be carried by the NAL unit includes a sequence parameter set (SPS) and a picture parameter set (SPS) that carry data necessary for decoding VCL data such as video resolution or required decoder performance. Parameter set such as PPS) or supplemental extended information (SEI) that carries information useful to the decoder or network element but not essential for decoding the VCL data.

  NAL is simple and effective for a wide variety of systems to transfer and store video data such as transmission via Real Time Transfer Protocol (RTP) or Hypertext Transfer Protocol (HTTP), or storage in ISO file format. And it is designed to be able to use VCL flexibly. The concept of a NAL unit is thought to provide a means for the network, ie the transmission and storage system, to access the bitstream and group and manipulate them by dividing the compressed bitstream into logical units. For example, a unit corresponding to one compressed picture is supplemented with advanced information that indicates to the network whether the encoded picture can be used as a random access point for initiating decoding of the compressed video.

  NAL is an H.264 standard. It is a functional unit of the minimum size of H.264 / AVC video. A NAL unit can be subdivided into a NAL unit header and a NAL unit payload. The NAL unit header consists of a set of identifiers used by the network to manage the compressed bitstream. Based on the information carried in the NAL unit header to minimize quality degradation due to discarding video data, for example to reduce the video transmission bit rate when bandwidth is limited, The NAL unit is discarded. This process is called “bitstream thinning”.

  Conventional video services provide video in a single representation, i.e., using a fixed camera position and spatial resolution, but recently, multi-resolution and multi-view video representations have gained importance. Multi-resolution representations represent multiple different spatial resolution videos for use with target devices having different display resolutions. Multi-viewpoint expression represents content from different camera viewpoints. A specific example is stereoscopic video, where the scene is captured by two cameras that have a distance similar to the human eye. Using appropriate display technology, depth perception is provided to the viewer.

  In many cases, multi-resolution representations and multi-view video representations are called hierarchical representation layers. In this case, the base layer represents the basic quality of the video, and subsequent enhancement layers modify the representation for higher quality.

  Scalable video coding (SVC) and multi-view video coding (MVC) are video coding standards used to compress multi-resolution video representations and multi-view video representations, respectively. By removing, high compression efficiency is achieved. SVC and MVC are based on the AVC standard and are included as appendices G and H of the new version of AVC. As such, they share most of the AVC structure.

  Due to the hierarchical dependencies inherent in the SVC and MVC bitstreams, additional information fields such as decoding dependencies and view identifiers are required in the NAL unit header. However, the basic AVC NAL unit header was not changed in order to maintain compatibility with existing AVC implementations. Instead, additional information such as dependencies and view identifiers were incorporated by introducing two new types of NAL units: prefix NAL units (type 14) and coded slice extension NAL units (type 20). . These are ignored by AVC decoders that do not support the appendix G or H of the specification because they are defined as “not used” in AVC.

  The prefix NAL unit is associated with a VCL AVC NAL unit that is to be located immediately after the prefix NAL unit in the bitstream and carries additional information about the base layer. The AVC decoder can decode the base layer by ignoring the prefix NAL unit.

  The coded slice enhancement NAL unit is used only in the enhancement layer of SVC or MVC. This represents extension information for the base layer or other enhancement layers. In addition to conveying the dependency and view identifier in the same manner as the prefix NAL unit, the coded slice extension NAL unit is composed of both the SVC or MVC NAL unit header and the corresponding VCL data. This is therefore a combination of a prefix NAL unit and a VCL AVC NAL unit. SVC and MVC enhancement layer NAL units are ignored by the AVC decoder.

  The extension of AVC to SVC and MVC is defined in a similar way. Their use is mutually exclusive, i.e. the syntax and semantics defined in the standard are partially inconsistent and SVC elements and MVC elements cannot be used simultaneously. In order to combine the SVC feature and the MVC feature, it is necessary to change the standard, and in particular, it is necessary to change the definition of the NAL unit header.

  HEVC is a next-generation video coding standard that is currently being standardized. HEVC aims to substantially improve coding, especially for high resolution video sequences, compared to AVC.

  With regard to advanced syntax design, the simplest method is to adopt the AVC advanced syntax concept, and in particular, to adopt the AVC NAL unit concept. However, this causes the following problems.

According to the latest technology, SVC and MVC are built from AVC for backward compatibility. The new NAL unit type 20 is designed with header extensions that can be used for any enhancement layer. In order to solve the problems of the conventional AVC decoder, the conventional NAL units (type 1, type 5 and other types) are maintained, and the method of associating the prefix NAL unit is a normal AVC VCL NAL unit (type 1 and Used for each of type 5). This approach may in principle be used for HEVC and later extensions, but has the following associated problems.
-Introducing a new feature or functionality requires a definition of the type of the new NAL unit, eg a coded slice extension NAL unit. This is undesirable because the maximum number of NAL unit types is usually limited, for example, by the defined length of the NAL unit type field.
-In order to take into account the conventional decoder, the base layer needs to be created using the conventional NAL unit type together with the prefix NAL unit, so that a second new NAL unit type is designed. There is a need. Therefore, the number of types of NAL units further increases.
-Signal transmission in the base layer and the enhancement layer is not uniform, and special processing via the network is required for each layer. This complicates the implementation example. The use of prefix NAL units is unnatural and provides only a weak link between the necessary header information and the corresponding VCL data. This link can easily break if, for example, one of the NAL units is lost during transmission.
-For future expansion, nesting of prefix NAL units is complex.
-By extending the advanced interface via an additional NAL unit header, the network functionality considered to process the NAL unit based on the information conveyed in the NAL unit header is Need to be updated.

  A further problem associated with modern AVC concepts relates to hierarchical representation. Currently, in SVC and MVC, all flags related to layer properties such as view_id, dependency_id and quality_id are simply put in the NAL unit header without any intelligent selection or classification. For example, if a client receiving a bitstream wants to thin out or manipulate the bitstream, the client needs to have detailed knowledge about the definition of the flag. Basically, the client needs to fully understand the meaning of each flag and their interrelationship. For example, if a view needs to be extracted from a multi-view bitstream, an erroneous operation is likely to occur if the view on which that view depends is not included. Alternatively, if the client only considers the view_id flag, the low quality version is selected. Even if the SEI element provides some assistance, it is very difficult for the network to find and understand all the information necessary to extract a particular video representation from the hierarchical bitstream.

  Furthermore, with the increasing number of applications and standards that cover 3D, new data elements such as depth maps and occlusion maps are transmitted along with textures, allowing more flexible rendering of the output view at the receiver. Since such elements form a hierarchical representation with “texture” video (multi-view or scalable), it is desirable to transmit all in the same bitstream. Alternatively, such bundling of different data elements may be achieved via high system level signaling such as transfer protocols or file formats. However, software and hardware implementations of such advanced protocols are often separated from video decompression implementations, and therefore differ in texture and depth synchronization, etc. if not supported at the bitstream level. Accurate time synchronization of data elements is very complex. Note that because different video elements need to be frame aligned, the synchronization of different video data elements such as texture and depth needs to be much closer than the synchronization of video and audio. In addition, video elements such as texture and depth may be compressed together, for example by reusing motion information between them ("motion vectors"), which requires tight coupling at the bitstream level It is.

  The initial target of HEVC development is video only. However, it may be expanded to scalable coding and / or multi-view coding in the future. In addition, the concept of packetization similar to the concept of NAL units in AVC may be used. Therefore, in the following, the presented method is mainly applicable to future video coding standards such as HEVC, but the term “NAL unit” is used interchangeably as defined in AVC. Also, because other AVC concepts such as SPS, PPS and SEI are expected to be used in HEVC, they may be referred to by different names in HEVC or some other future video coding standard, AVC terminology is used.

  It is an object of the present invention to provide an improved technique of the above technique and the prior art.

  More particularly, it is an object of the present invention to provide an improved general syntax for future video coding standards that facilitates hierarchical video representation.

  The above and other objects of the invention are achieved by the various aspects of the invention as defined by the independent claims. Embodiments of the invention are characterized by the dependent claims.

  To illustrate the present invention, a video signal is encoded into a compressed video bitstream and transmitted over a network, for example a local area network, a mobile telephone network or the Internet, eg on a television, computer, video player or mobile telephone. Assume that it is decrypted at some client. The network may include multiple network elements such as routers and switches.

  According to a first aspect of the present invention, a method for indicating a bitstream subset within a compressed video bitstream is provided. The compressed video bitstream includes a plurality of bitstream subsets, ie, at least two bitstream subsets. The method comprises receiving a compressed video bitstream, dividing the video bitstream into video packets, and marking each video packet with a subset identifier of the plurality of subset identifiers. Each video packet includes either video data or supplementary information. Each subset identifier of the plurality of subset identifiers is associated with a corresponding bitstream subset of the plurality of bitstream subsets.

  According to a second aspect of the present invention, a computer program is provided. The computer program comprises computer program code. The computer program code is configured to be executed to implement the method according to the first aspect of the present invention.

  According to a third aspect of the invention, a computer program product is provided. The computer program product comprises a computer readable medium. The computer program according to the second aspect of the present invention is realized on a computer-readable medium.

  According to a fourth aspect of the present invention, a method for extracting video packets from a compressed video bitstream is provided. The compressed video bitstream is divided into video packets. The compressed video bitstream includes a plurality of bitstream subsets. Each video packet includes either video data or supplementary information. Each video packet further includes a subset identifier of the plurality of subset identifiers. Each subset identifier is associated with a corresponding bitstream subset of the plurality of bitstream subsets. The method comprises providing at least one relevant subset identifier and receiving a video packet from the compressed video bitstream. For each received video packet, the method further comprises examining a subset identifier of the video packet and extracting the video packet from the compressed video bitstream. Video packets are extracted from the compressed video bitstream, provided that the extracted subset identifier matches one of the at least one related subset identifier.

  According to a fifth aspect of the present invention, another computer program is provided. The computer program comprises computer program code. The computer program code is configured to be executed to implement a method according to the fourth aspect of the present invention.

  According to a sixth aspect of the invention, a computer program product is provided. The computer program product comprises a computer readable medium. A computer program according to the fifth aspect of the present invention is realized on a computer-readable medium.

  According to a seventh aspect of the present invention, a bitstream marker is provided that indicates a bitstream subset within a compressed video bitstream. The compressed video bitstream includes a plurality of bitstream subsets. The bitstream marker comprises a receiving unit, a packetizing unit, and a marking unit. The receiving unit is configured to receive a compressed video bitstream. The packetization unit is configured to split the compressed video bitstream into video packets. Each video packet includes either video data or supplemental information. The marking unit is configured to mark each video packet with one of the plurality of subset identifiers. Each subset identifier is associated with a corresponding bitstream subset of the plurality of bitstream subsets.

  According to an eighth aspect of the present invention, there is provided a bitstream extractor that extracts video packets from a compressed video bitstream. The compressed video bitstream is divided into video packets. The compressed video bitstream includes a plurality of bitstream subsets. Each video packet includes either video data or supplemental information. Each video packet further includes a subset identifier of the plurality of subset identifiers. Each subset identifier is associated with a corresponding bitstream subset of the plurality of bitstream subsets. The bitstream extractor includes a subset selection unit, a reception unit, and an extraction unit. The subset selection unit is configured to provide at least one relevant subset identifier. The receiving unit is configured to receive a video packet from the compressed video bitstream. The extraction unit is configured to check the subset identifier of the video packet and extract the video packet from the compressed video bitstream for each received video packet. Video packets are extracted from the compressed video bitstream, provided that the extracted subset identifier matches one of the at least one related subset identifier.

  The present invention takes advantage of the view that the concept of a layered bitstream in modern video coding standards may be generalized to allow the identification of different bitstream subsets. In this case, each bitstream subset represents a layer with which a particular property is associated. For example, each bitstream subset can be a VCL layer that carries video data such as a texture base view layer, a depth map high quality layer, a temporal occlusion map layer, or non-VCL that carries non-video data, i.e., supplemental information, eg, a parameter set. Represents one of the layers. This is accomplished by associating each layer, ie, a bitstream subset, with a stream identifier (stream_id). The stream identifier is associated with a parameter describing a layer property, such as a specific view identifier (view_id) or a dependency identifier (dependency_id). The stream_id is signaled in the NAL unit header.

  By consolidating all the different properties of a particular layer into a single identifier stream_id, the process of interpreting and identifying video packets within the network and on the client side is simplified. The proposed syntax allows a concise and scalable system design for signal transmission of network friendly and advanced video bitstreams that is particularly applicable to hierarchical representations and is therefore compatible with future video codecs and applications. The video bitstream signal transmission according to an embodiment of the present invention is advantageous in that it reduces the problems inherent in the latest video coding standards, in particular the AVC NAL unit concept described above.

  More specifically, the new functionality does not require a new NAL unit type definition or a NAL unit header syntax update. Furthermore, since the layer properties are summarized into a single stream_id, the network elements and clients that process the video bitstream are concerned with all the information elements used in the NAL unit header, ie identifiers, indicators, parameters or flags. There is no need to have detailed knowledge. Rather, knowledge of the relevant stream_id is sufficient. Finally, signal transmission of hierarchical video bitstreams, ie, multiple bitstream subsets multiplexed into one compressed video bitstream, can achieve accurate time synchronization more easily than solutions that rely on advanced signal transmission. Is advantageous. In addition, redundancy between layers of interest in video representation may be utilized in video signal compression.

  Although we have stated that each video packet in a compressed video bitstream is marked with a single subset identifier, a book that only marks a subset of all video packets contained in a compressed video bitstream with a single subset identifier. Embodiments of the invention are also contemplated.

  According to an embodiment of the invention, the method further comprises providing a definition of at least one subset. Each subset definition describes the properties of the corresponding bitstream subset of the plurality of bitstream subsets. Utilizing the subset definition to define the associated bitstream subset properties is advantageous in that the corresponding video layer properties are clearly provided to network elements and clients.

  According to one embodiment of the invention, the definition of at least one subset is provided as a video packet in a compressed video bitstream. The video packet may carry multiple subset definitions, each corresponding to a different bitstream subset. Video packets containing one or more subset definitions may be carried in a parameter stream set (StPS). This is advantageous in that the definition of the associated bitstream subset, i.e. the subset describing the properties of the video or parameter layer, is provided to the network element and the client together with the video signal. Each subset definition includes information regarding at least one of temporal_id, view_id, quality_id, priority_id, or the type of data carried in the subset. It will be appreciated that one or more bitstream subsets may be reserved for signaling subset definitions or other parameters. Such reserved bitstream subsets may be associated with a predefined stream_id that is known to network elements and clients, eg, stream_id = 0.

  According to an embodiment of the present invention, each subset identifier of the plurality of subset identifiers may be a numerical value. Each subset identifier value corresponds to the relative priority of the bitstream subset associated with it. In other words, the subset identifier stream_id of each bitstream subset indicates the importance of the video data carried by the particular bitstream subset. Using such information, for example, a network element or client that needs to discard packets due to limited bandwidth discards packets with a high stream_id indicating low relevance and is highly relevant Hold a packet with a low stream_id indicating that

  According to an embodiment of the invention, the method further comprises providing a definition of at least one video representation. Each definition of the video representation includes at least one related subset identifier. The bitstream subset associated with at least one relevant subset identifier forms a decodable video representation. In other words, each video representation groups a plurality of stream_ids and their associated bitstream subsets to form a decodable video. This differs from the subset definition that identifies a single bitstream subset that cannot be independently decoded. The video representation includes, for example, all texture information for the basic view, or texture and depth map information. This is advantageous in that network elements or clients are provided with information about the bitstream subset that needs to be processed in order to successfully decode a particular video representation.

  According to one embodiment of the invention, the definition of at least one video representation is provided as a video packet in a compressed video bitstream. This is advantageous in that the information about the video representation, ie the list of stream_ids that need to be processed to form a decodable video, is signaled, ie multiplexed, with the video data. A video packet may carry multiple video representation definitions. Each definition of a video representation corresponds to a different decodable video representation having a particular characteristic, ie property. Video packets containing one or more video representation definitions may be carried in a bitstream subset called a representation parameter set (RPS) reserved for this purpose. The RPS may be associated with a predefined stream_id, for example, stream_id = 0. It will be further appreciated that the video representation may be associated with a numerical value that indicates the relative priority of the video representation.

  According to an embodiment of the present invention, the method further comprises, for each received video packet, transferring or decoding the extracted video packet or discarding the received video packet. The received video packet is discarded on condition that the extracted subset identifier does not match any of the at least one related subset identifier. In other words, a received video packet is processed if the stream_id associated with it matches the list of relevant stream_ids, i.e. forwarded or decoded, and discarded if it does not match. This is advantageous in that the network element or client processes the video packet depending on whether the received video packet is a relevant video packet or not. For example, the client is configured to process only a particular bitstream subset, eg, a decodable video, ie, a group of subsets that together form a video representation. Further, the network element may be configured to discard bitstream subsets that are considered less relevant, eg, high quality enhancement layers, when bandwidth is limited.

  According to an embodiment of the invention, the method further comprises providing a subset definition. The subset definition describes the properties of the corresponding bitstream subset of the plurality of bitstream subsets. The method further comprises using the subset identifier associated with the corresponding bitstream as at least one related subset identifier. Using the stream_id included in the subset definition is advantageous in that it allows network elements and clients to select video packets specified by the subset definition for processing.

  According to an embodiment of the invention, the method further comprises selecting a subset definition from a plurality of subset definitions. The definition of the subset is selected according to at least one property of the corresponding bitstream subset. This is advantageous in that a network element or client chooses to process a bitstream subset with specific properties when multiple subset definitions are provided. For example, the client selects a subset definition that includes a specific indicator, eg, a specific temporal_id, view_id, quality_id, priority_id, or indicating a specific type of data carried in the subset.

  According to one embodiment of the invention, the method further comprises receiving a subset definition from a video packet in the compressed video bitstream. This is advantageous in that the definition of the associated bitstream subset, i.e. the subset describing the properties of the video or parameter layer, is received by the network element and the client together with the video data.

  According to one embodiment of the invention, the method further comprises receiving a definition of the video representation from a video packet in the compressed video bitstream. This is advantageous in that information about the video representation that needs to be processed to form a decodable video, i.e. a plurality of stream_ids, is received with the video data.

  Although the advantages of the present invention have been described in several examples with reference to method embodiments according to the first and fourth aspects of the present invention, the corresponding rationale is not limited to the second and fourth aspects of the present invention. The computer program product according to the fifth aspect, the computer program product according to the third and sixth aspects of the present invention, and the apparatus according to the seventh and eighth aspects of the present invention are applied.

  Further objects, features and advantages of the present invention will become apparent upon review of the following detailed disclosure, drawings, and appended claims. Those skilled in the art will appreciate that the various features of the present invention can be combined to create embodiments other than those described below.

The above objects, features and advantages of the present invention, as well as additional objects, features and advantages will be further understood by reading the following non-limiting detailed description of embodiments of the present invention with reference to the accompanying drawings. Will be understood.
FIG. 1 is a diagram illustrating a system for encoding, transferring, and decoding video signals. FIG. 2 is a diagram illustrating the concept of bitstream subsets, subset definitions, and video representation definitions according to an embodiment of the present invention. FIG. 3 is a diagram illustrating a proposed syntax according to an embodiment of the present invention. FIG. 4 is a diagram illustrating a method for indicating a bitstream subset in a compressed video bitstream according to an embodiment of the present invention. FIG. 5 is a diagram illustrating a method for extracting video packets from a compressed video bitstream according to an embodiment of the present invention. FIG. 6 is a diagram illustrating a bitstream marker according to an embodiment of the present invention. FIG. 7 is a diagram illustrating a bitstream extractor according to an embodiment of the present invention. FIG. 8 is a diagram illustrating a video processing apparatus that executes computer program code according to an embodiment of the present invention.

  All drawings are schematic and are not necessarily to scale, generally showing only the portions necessary to describe the present invention. In that case, the other parts are omitted or simply suggested.

  To illustrate the present invention, a system 100 for encoding, transferring and decoding video signals is shown in FIG.

  The system 100 includes a video encoding device 110, a transfer network 120, and a video decoding device 130. Typically, the video encoder 110 is configured to receive a video signal from one or more sources, compress the video signal, and subdivide the resulting bitstream into video packets, eg, NAL units. The The resulting video packet is then transferred to the decoding device 130 via the transfer network 120. The transport network 120 typically comprises a plurality of interconnect nodes, ie network elements 121-123, configured to transfer video packets from the encoding device 110 to the decoding device 130. Network elements 121-123 are, for example, switches, routers or some other type of network node suitable for processing video packets. The transport network 120 is, for example, a local area network, a mobile telephone network, or the Internet.

  Decoding device 130 is configured to receive video packets from transport network 120 and to decode the received compressed video bitstream. Further, the decryption device 130 may be configured to display the decrypted video to a viewer. The decoding device 130 is, for example, a video player, a television, a computer, or a mobile phone.

  Hereinafter, an embodiment of the present invention will be described with reference to FIG. 2 showing the concept of stream identifiers, subset definitions and video representation.

  In FIG. 2, a portion of the compressed video bitstream 210 is shown. The portion includes six video packets, that is, NAL units 211 to 216, and each video packet includes supplementary information such as video data or parameters. Furthermore, each NAL unit 211-216 includes a flag stream_id for associating each NAL unit 211-216 with a corresponding bitstream subset of the compressed video bitstream. For example, for the portion 210 of the video bitstream illustrated in FIG. 2, NAL units 211, 213, and 216 are marked with stream_id = 0. That is, they are associated with the first bitstream subset. In addition, NAL units 212 and 214 are marked with stream_id = 1. That is, they are associated with a second bitstream subset that is different from the first bitstream subset. Finally, the NAL unit 215 is marked with stream_id = 2. That is, it is associated with a third bitstream subset that is different from the first bitstream subset and the second bitstream subset.

  The identifier stream_id included in each NAL unit carried in the compressed video bitstream indicates the bit stream subset to which the NAL unit belongs for each NAL unit. Compressed video where each bitstream subset carried in the compressed video bitstream is, for example, a texture base view layer, a depth map high quality layer, a parameter set layer, a temporal layer, an occlusion map layer, or some other type of video or supplemental layer Each NAL unit is associated with a corresponding layer to represent a particular layer of the signal. In this way, for example, all parameters associated with a particular layer, which are parameters describing the view that the layer represents, the video quality that the layer contains, or the dependency between layers, are summarized in a single stream identifier stream_id. Indirect designation is introduced by using a single identifier to describe the parameter set, and the NAL unit simplified processing by the network element through which the NAL unit is transferred or the NAL unit is decrypted when it is transferred Is possible.

  The properties of each layer, i.e. the bitstream subset, are predefined and may be known for all entities involved in the encoding, transmission and decoding of the video signal. For example, referring to FIG. 1, the decoding device 130 only decodes video packets belonging to different bitstream subsets, ie one or several of the layers, received from the encoding device 110 via the transport network 120. It may be configured as follows. For example, stream_id = 0 is a base layer, and stream_id = 1 and stream_id = 2 are enhancement layers for providing improved video quality, and the decoding apparatus 130 can display only a low-quality video signal. The case is true. Further, the network elements 121-123 may be configured to forward only video packets belonging to one or two of the three bitstream subsets when the available bandwidth is limited.

  With further reference to FIG. 2, the improved processing of video layers included in a multi-layer video bitstream will be described.

  According to one embodiment of the invention, the properties of each layer, i.e. the bitstream subset, are provided to entities involved in the encoding, forwarding and decoding of the video signal according to the subset definitions 221-223. For each layer, information describing the properties of the layer, ie corresponding subset definitions 221 to 223 including parameters, are provided. For example, the subset definition 221 describes the properties of the first bitstream subset indicated by stream_id = 0. Correspondingly, subset definition 222 describes a second bitstream subset, ie, a stream_id = 1 property, and subset definition 223 describes a third bitstream subset, ie, a stream_id = 2 property. The parameters included in each of the subset definitions 221-223, namely texture_flag, depth_flag, occlusion_flag and view_id, are whether the bitstream subset is a texture layer, a depth map layer, an occlusion texture layer, and Indicates the view to which it belongs. In the case of the subset definitions 221 to 223 illustrated in FIG. 2, all layers have view_id = 0 and belong to the same view. The first layer described by subset definition 221 contains the texture of the view, the second layer described by subset definition 222 contains the depth map of the view, and the third layer described by subset definition 223 The layer contains the occlusion texture of the view.

  With further reference to FIG. 2, the concept of video representation according to one embodiment of the present invention will be described.

  The video representation is used to group one or more layers, ie bitstream subsets, to form a decodable video. This grouping mechanism is used to provide information about bitstream subsets to network elements and clients involved in processing compressed video bitstreams. This is accomplished by providing network elements and clients with integrated information included in the definition of the video representation. To this end, the video representation may include, for example, all texture information in the bitstream, or may include texture and depth map information for the base view. For example, a network element or client that wants to extract a particular video representation from a compressed video bitstream in order to decode the video signal and display the video to the viewer identifies the relevant video representation and then selects one of the video representations. Extract all bitstream subsets that are parts.

  Video representations such as video representations 231-233 shown in FIG. 2 according to one embodiment of the present invention are marked with a video representation identifier representation_id to facilitate identification of the video representation by network elements and clients. In addition, each video representation includes a list of stream_ids that indicate the bitstream subsets needed to render meaningful video by decoding the video signal.

  For example, the first video representation defined by representation definition 231 is marked with representation_id = 0 and includes a single bitstream subset identifier stream_id = 0. Thus, given a bitstream subset definition 221 with stream_id = 0, the first representation is a mono 2D video sequence, ie a single view of texture information. Further, the second video representation defined by representation definition 232 is marked with representation_id = 1 and includes a list of two bitstream subset identifiers stream_id = 0 and stream_id = 1. Thus, given the subset definitions 221 and 222, the second video representation further includes a depth map carried by the layer with stream_id = 1 and can render a 3D video sequence. However, the quality of the rendered 3D video is limited because the second video representation does not contain occlusion texture information. This problem is solved by a third video representation that allows rendering of 3D video sequences containing occlusion texture information. For this purpose, the video representation definition 233 includes a list of three bitstream subset identifiers stream_id = 0, stream_id = 1 and stream_id = 2.

  It will be appreciated that the concept of hierarchical bitstream, subset definition and video representation is not limited to only one view as described above. The definition of the subset is used to describe the properties of layers that represent different views, for example with different camera angles, eg view_id = 0 and view_id = 1, or layers carrying video signals of different video quality.

  The concept of video representation allows simple processing of multi-layer video bitstreams by network elements and clients involved in processing video bitstreams. A network element that wants to transfer a particular video representation or a client that wants to decode simply identifies the corresponding representation definition, reads the list of required bitstream subset identifiers from the representation definition, and belongs to the required bitstream subset Video packets are extracted from the compressed video bitstream.

  Compared to known methods for identifying a bitstream subset, such as an extension of the SVC NAL unit header, which uses multiple identifiers such as temporal_id, priority_id, quality_id, and dependency_id, the proposed method is a single identifier. Much simpler implementations of both hardware and software are possible. In particular, a single identifier stream_id, for example, unlike dependency_id, does not have a predefined meaning, but its meaning is indicated via secondary means, for example a subset definition. With this indirection, the proposed concept can be easily extended when new functionality is introduced at a later stage, as information about such functionality is signaled in the subset definition, and NAL There is no need to change the unit header. In particular, the NAL unit header according to an embodiment of the present invention has a fixed length regardless of whether the extension is used. Thereby, parsing of the NAL unit header is much simpler than known solutions.

  With reference to FIG. 3, a more detailed description of embodiments of the present invention is presented below, particularly with respect to the proposed syntax.

  The subset identifier stream_id is carried in the NAL unit header. The NAL unit header may or may not include other identifiers such as the type of NAL unit or output_flag. An example of the NAL unit header 310 is shown in FIG. In this example, forbidden_zero_bit is assumed to be equal to 0, nal_unit_type specifies type data included in the NAL unit, and output_flag signals whether the decoded content of the current NAL unit is intended for screen output.

By marking different NAL units with the same stream_id, they are marked as belonging to the same bitstream subset, ie the same layer. In general, all NAL units belonging to the same bitstream subset share one or more of the specific properties illustrated below.
-All NAL units in the subset are parameter sets.
-All NAL units in the subset are intra-coded pictures.
-All NAL units in the subset indicate the starting point of a so-called "closed picture group (GOP)" random access point (instantaneous decoder refresh (IDR) picture in AVC).
-All NAL units in the subset indicate the starting point of a so-called “open GOP” random access point (intra picture from which decoding can start).
-All NAL units in the subset carry video frames representing the basic quality.
-All NAL units in the subset carry time refinement information.
-All NAL units in the subset carry spatial refinement information in case of spatial scalability.
-All NAL units in the subset carry information corresponding to a particular camera view in multi-view coding.
-All NAL units in the subset carry information about a particular video stream or eg a depth map stream.

  It should be noted that a plurality of such properties may be valid for a given subset at the same time.

  Some subset of properties and associated stream_id may be predefined. For example, stream_id = 0 indicates a subset including only NAL units that carry parameter sets such as SPS, PPS, StPS, or RPS. Alternatively, SPS, PPS, StPS and RPS have different predefined stream_id.

  Information about the properties of the subset may be explicitly provided, for example, by the definition of the subset. The definition of the subset may include parameters such as temporal_id or quality_id. An example of a subset definition 320 is shown in FIG.

  The syntax of a subset definition according to one embodiment of the invention, such as subset definition 320, may include conditional fields that depend on, for example, the type of NAL unit or other properties. The subset definition 320 illustrates syntax elements that are used only when the referenced NAL unit includes VCL data and does not include non-VCL data. (In this disclosure, the type of NAL unit in the example is the AVC specification. Follow).

  In the subset definition 320, stream_id identifies the hierarchical stream that is a subset of the bitstream, stream_type describes the type of the specified stream, and version_id specifies the version of the specification that follows when describing the stream. Each of the view_id, temporal_id, quality_id, and dependency_id flags identifies a property or corresponding layer. More specifically, view_id indicates a camera view, temporal_id indicates a temporal refinement layer that refines a frame rate of, for example, 30 Hz to the upper 60 Hz, quality_id indicates signal fidelity of compressed video, and dependency_id indicates spatial scalability. The spatial refinement layer of is shown.

  The subset definition itself may be carried in the bitstream, for example in the form of dedicated NAL units, ie StPS. Such dedicated NAL units are indicated by the type of dedicated NAL unit. This may be part of a dedicated subset, such as a bitstream subset of the parameter set indicated by a particular stream_id in the NAL unit header of the NAL unit carrying the parameter set. The subset carrying NAL units of such a stream parameter set may have a predefined stream_id, such as stream_id = 0.

  StPS includes parameters related to time extension (temporal_id), quality extension (quality_id), spatial extension (dependency_id), priority (priority_id), or some signal transmission as carried in the NAL unit extension header for SVC and MVC. But you can. The StPS may further include information regarding the type of data carried in the associated NAL unit, i.e., information regarding whether the NAL unit includes a parameter set, an SEI message, an intra picture, an anchor picture, or the like. StPS may further carry advanced information about the content represented by the compressed data, for example, information about whether the associated NAL unit represents texture data, depth information, occlusion information, or the like.

  The syntax of the stream parameter set according to an embodiment of the present invention, such as the subset definition 320 illustrated in FIG. 3, is defined in an extensible manner. For this, it contains syntax elements defined by a specific version of the specification, for example version N. In a future update of the specification, for example version N + 1, additional syntax elements will be included. In this case, a receiving device that conforms to version N of the specification can only interpret version N syntax elements, but a receiving device that conforms to version N + 1 can also interpret additional syntax elements. In this case, StPS has a variable length and can be extended in new versions of the specification as needed. When a version N receiving device receives a version N + 1 StPS including a parameter compliant with version N and further parameters compliant with version N + 1, the receiving device reads the syntax element of version N and reads the syntax element of version N + 1. ignore. Alternatively, if a receiving device of version N finds a syntax element that cannot be interpreted because it does not conform to version N, it determines that it does not process the NAL unit associated with StPS. The behavior of the device, i.e. ignoring the unknown syntax or not processing the associated NAL unit, is signaled, for example, by a separate flag.

  The syntax of the stream parameter set may further include a version identifier indicating the version of the specification to which it conforms. For example, the version identifier may be expressed as a numerical value. In this case, the larger the numerical value, the larger the version number. When a version N receiving apparatus receives an StPS having a version number of N or less, the receiving apparatus can analyze the syntax. When a version N receiving apparatus receives an StPS having a version number greater than N, the receiving apparatus cannot parse the syntax, and at least a part specific to a version having a version number greater than N cannot be parsed. In that case, the receiving apparatus does not process the NAL unit associated with StPS. Alternatively, the receiving apparatus decodes the NAL unit associated with StPS without interpreting StPS itself. This behavior may be controlled by an additional flag.

  As shown in the subset definition 320, the association between the stream parameter set and the NAL units included in the subset is provided by the parameter stream_id, and is the same as the NAL unit associated with the stream parameter set, ie the NAL unit containing the subset definition Marked with stream_id. Alternatively, StPS indicates an association with two or more stream_ids by including a list of stream_id, for example, indicates that StPS corresponds to a NAL unit having any of these stream_ids. As a further alternative, StPS indicates a bit mask field such as M = “11111111000000” (decimal 65280) and a value field such as V = “1010101000000” (decimal 43520), and “&” is a bit. All NAL units having a stream_id that matches a condition such as “stream_id & M = V” as the unit “and” operation are associated with the StPS. Associating multiple stream_ids with the same StPS has the advantage that fewer StPS packets are needed to carry the parameters for the subset properties. In this way, information valid for a plurality of bitstream subsets corresponding to a plurality of different stream_ids is carried in a single StPS. Additional information that is only valid for one or several of these subsets can be carried in a separate StPS.

  The stream_id may be a numerical value, a NAL unit having a low stream_id carries high importance data such as a parameter set or an intra frame, and a NAL unit having a high stream_id is low importance such as SEI or time extension data It may be defined to carry data. Using such priority information, for example, a network node that needs to discard a packet because its bandwidth is limited discards a packet with a high stream_id value and retains a packet with a low stream_id value.

  Compared to using multiple parameters such as temporal_id and dependency_id, such as SVC and MVC, the advantage of the stream_id concept is that it uses indirect designation, so when new codec functionality is newly set. There is no need to introduce new fields. In addition, if the stream_id represents the relative priority of the associated bitstream subset, in some instances it may not be necessary to analyze the indirection, further simplifying parsing and dispatching of NAL units.

  By providing a list of stream_ids in the video representation, a part of the bitstream that can be decoded independently of the other parts of the compressed video bitstream is identified. The associated bitstream subset forms a video that can be decoded. That is, there is no dependency between pictures or views on NAL units outside the representation. This is different from a bitstream subset that is identified by a single stream_id but is not decoded independently, for example when it contains only time extension information. Video representations are associated with specific properties such as content type, eg texture, depth or occlusion information.

  The definition of the video representation, i.e. the list of required stream_ids and the additional properties of the representation, may be provided in a separate video packet in the bitstream, i.e. RPS. There are at least two ways of providing a parameter set of representations. One way is to provide only one RPS in one NAL unit. Alternatively, multiple RPSs are provided in a single NAL unit as illustrated by NAL unit 330 according to one embodiment of the present invention.

In the video representation definition 330, num_representations is the number of representations specified in this NAL unit, representation_id identifies the video representation, representation_priority_id defines the priority of the representation, and representation_type is the type of video representation. The type of video expression is, for example, one of the following.
-Planar video with texture only.
Stereo video with texture only and stereo application.
Planar video that has texture and depth information and uses view synthesis to generate some autostereoscopic / stereoscopic effects.
Stereoscopic video with texture and depth information and using view synthesis to produce sufficient autostereoscopic effects.
Planar video with texture, depth and non-occlusion information and using view compositing to generate sufficient autostereoscopic / stereoscopic effects.
SPS and PPS that are sent out-of-band to network elements and clients in advance using a reversible channel.

  With further reference to the video representation definition 330, num_streams signals the number of required streams included in the representation, and each stream_id specifies the required streams. Optionally, a NAL unit carrying RPS information may be marked with a predefined stream_id, eg, stream_id = 0.

  As shown in the representation definition 330, one of the properties defined by the RPS may be a priority indicator representation_priority_id that indicates a priority value of the current representation relative to other video representations.

  Based on the representation definition RPS and the subset definition StPS, the dynamic streaming approach is applied in a simple manner. For example, the server provides a video bitstream having three bitstream subsets, or layers, identified by quality_id = 0, 1, and 2, respectively. These streams are selectively combined into three video representations of low quality, intermediate quality and high quality. In this case, the low quality representation includes a subset with quality_id = 0, the intermediate quality representation includes two subsets with quality_id = 1 and quality_id = 2, and the third representation includes all three subsets. The client may initiate the download of the first stream with the lowest quality but the smallest size to obtain a short buffer time. After playing the video for a while and detecting that the network bandwidth is available, the client may switch to an intermediate or high quality representation. If congestion occurs in the network, the client may reduce it to a low quality level.

  Having the quality_id and other parameters defined in the subset definition facilitates the selection of the required bitstream subset. To this end, the client is simply instructed using the rules for stream_id, rather than parsing and interpreting multiple identifiers as in the SVC NAL unit header. Note that, since the bandwidth usually has a positive correlation with the quality, the required bit rate may be indicated by signal transmission of bandwidth_id instead of quality_id.

  According to an embodiment of the present invention, a network element such as a video packet receiver or video packet forwarder that receives or forwards a packet, or a decoding device that receives and decodes a video packet interprets stream_id as follows: . Assume that a list of stream_ids that are considered to be related to receiving, forwarding or decoding operations is provided to the network or decoding device. When a video packet is received, stream_id is examined. Depending on the value of stream_id, the video packet is received / transferred / decoded, ie extracted from the bitstream or discarded. That is, if the stream_id in the video packet matches one of the relevant stream_ids, the packet is further processed. If the stream_id in the video packet does not match any of the relevant stream_ids, the packet is discarded and not processed further.

  The list of related stream_ids may include a predefined stream_id. For example, this may include stream_id = 0 indicating that the associated bitstream subset includes a parameter set. Thus, the receiving / forwarding / decoding device receives all parameter sets. The device then interprets, for example, one or several of StPS or RPS and updates the list of relevant stream_ids accordingly. For this purpose, the receiving apparatus may receive all StPS and check the syntax of StPS for a specific property. For example, when a StPS having a predetermined property that is a type of video data such as texture data is detected, a video packet having an associated stream_id is extracted from the bitstream. Thus, the StPS and the associated stream_id can be selected based on parameters such as temporal_id, view_id, quality_id or video data type carried in the StPS.

  The relevance of the subset may be further determined based on the version identifier carried in the associated StPS. If the receiving device complies with the version identifier indicated in StPS, the device extracts the associated NAL unit. If the receiving device does not comply with such a version identifier, the device discards the associated NAL unit.

  A list of relevant stream identifiers may be obtained by examining the definition of expressions that are considered relevant. In this case, the list of stream identifiers is extracted from the definition of the expression and used as the relevant stream identifier.

  The receiving device receives all the RPSs and checks the syntax of the RPS for specific properties, eg priority, video content type or video resolution. If an RPS with a given property is detected, all associated stream_ids are considered relevant stream_id and are extracted from the bitstream.

  Alternatively, the receiving / forwarding / decoding device determines whether to extract or discard the packet based on the priority of each NAL unit. In particular, if the stream_id is defined according to the relative priority of the bitstream subset, the receiving / forwarding / decoding device extracts a packet with stream_id “low”, that is, data with high importance, and the stream_id is “ Discard packets that contain "high", ie, less important data. The determination that the value of stream_id is “low” or “high” may be based on a threshold. This approach is used for package deprocessing or bitstream decimation in network elements when bandwidth is limited.

  To further illustrate the present invention, an example is presented below. The example relates to stereoscopic video, i.e. two views with view_id = 0 and view_id = 1 respectively and with accompanying depth data. Assume that both texture and depth video provide some temporal scalability indicated by different temporal_id. In this case, temporal_id = 0 indicates the basic quality of time, for example, 30 Hz frame rate video, and temporal_id = 1 has time extension information for extending the frame rate from 30 Hz to 60 Hz, for example.

  NAL units that carry data regarding both texture and depth for available views and temporal resolution are multiplexed into the same bitstream using multiple bitstream subsets. When using the concept of bitstream subsets and video representation according to embodiments of the present invention, they are uniquely identifiable.

The following table provides an example of a subset definition.

  There are 5 subsets with texture data, stream_id = 1-5 corresponding to view_id = 0 and view_id = 1, and 5 further subsets with stream_id = 6-10 corresponding to depth data. Subsets with stream_id = 1 and stream_id = 6 indicate sequence parameter sets (SPS) and picture parameter sets (PPS), ie, subsets that carry non-VCL data, respectively. The remaining stream_id indicates the subset that carries the VCL data.

Furthermore, an example of the definition of the corresponding expression is shown in the following table.

  In the table, the representation with representation_id = 0 includes a NAL unit with stream_id = 1 which is only SPS and PPS for the texture portion. As another example, an expression with representation_id = 3 includes stream_id = 1, 2, and 3 and represents a single view video with view_id = 0 that contains only texture information. As a further example, representation_id = 7 corresponds to a complete bitstream.

  After receiving the representation definition, the receiving or forwarding device determines the most appropriate representation for a given application, depending on the signaled properties, thereby obtaining a list of relevant stream_ids. The device can easily extract the NAL units associated with these stream_id by examining the NAL unit headers of the received NAL units.

  In the following, referring to FIG. 4, an embodiment of a method for indicating a bitstream subset within a compressed video bitstream will be described. One embodiment of the method is performed in a sending device, such as the encoding device 110 described with reference to FIG. In particular, one embodiment of the method may be performed in a bitstream marking device that receives a compressed video bitstream from a video encoder, ie, a bitstream marker. For this, an embodiment of the method may be implemented in a video encoder. Bitstream markers subdivide the bitstream into compressed video data, each of which is, for example, a video frame, supplemental information, or video packets that generally include NAL units. Each video packet is then marked with a single subset identifier using the syntax element stream_id in the NAL unit header according to one embodiment of the invention. Optionally, the video encoder may provide a video bitstream that is packetized into bitstream markers, in which case the bitstream marker need not subdivide the bitstream into packets. Furthermore, the marking procedure according to an embodiment of the invention may be performed by a video encoder rather than a separate bitstream marker.

  One embodiment 410 of a method illustrating a bitstream subset within a compressed video bitstream is shown in FIG. The method 410 may, for example, receive a compressed video bitstream from a video encoder (411), split the compressed video bitstream into video packets (412), and a single subset identifier of the plurality of subset identifiers. Marking each video packet (413). Each subset identifier of the plurality of subset identifiers is associated with a corresponding bitstream subset of the plurality of bitstream subsets.

  Optionally, the method 410 further comprises providing 414 a definition of at least one subset. Each subset definition describes the properties of the corresponding bitstream subset of the plurality of bitstream subsets. The subset definition may be provided as video packets in a compressed video bitstream and sent to network elements and clients.

  In addition to the subset definition provided in step 414, the method 410 may further comprise providing (415) a definition of at least one video representation. Each definition of the video representation includes at least one relevant subset identifier, and all bitstream subsets associated with the at least one relevant subset identifier form a decodable video representation. The definition of at least one video representation may be provided as a video packet in a compressed video bitstream and sent to network elements and clients.

  Hereinafter, an embodiment of a method for extracting a video packet from a compressed video bitstream divided into video packets will be described with reference to FIG. One embodiment of the method is performed in a receiving device or decoding device 130, such as the network elements 121-123 described with reference to FIG. In particular, an embodiment of the present invention may be implemented in a bitstream extraction device that receives a compressed video bitstream divided into video packets, ie, a bitstream extractor. To this end, an embodiment of the method may be implemented in a video decoder or a network element configured to route video packets.

  One embodiment 510 of a method for extracting video packets from a compressed video bitstream divided into video packets is shown in FIG. The method 510 provides at least one relevant subset identifier (511), receives video packets from the compressed video bitstream (512), and for each received video packet (513), Examining the subset identifier (514) and extracting a video packet from the compressed video bitstream, provided that the extracted subset identifier matches one of the at least one related subset identifier (515). 516).

  Optionally, the method 510 transfers or decodes the extracted video packet (517) and provided that the extracted subset identifier does not match any of the at least one related subset identifier (515), Discarding (518) the received video packet.

  Further, the method 510 may comprise providing a definition of a subset that describes properties of a corresponding bitstream subset of the plurality of bitstream subsets (519). The subset identifier associated with the corresponding bitstream is used as at least one relevant subset identifier in step 511.

  Optionally, in step 519, the subset definition may be selected from a plurality of subset definitions according to at least one property of the corresponding bitstream subset. The definition of the subset may be received from a video packet in the compressed video bitstream.

  Further, the method 510 may further comprise providing (520) a definition of a video representation that includes at least one relevant subset identifier. The bitstream subset associated with the at least one related subset identifier forms a decodable video representation. The definition of the video representation may be received from a video packet in the compressed video bitstream.

  Hereinafter, with reference to FIG. 6, a bitstream marker indicating a bitstream subset in a compressed video bitstream according to an embodiment of the present invention will be described. One embodiment of the bitstream marker is located in the encoding device 110 described with reference to FIG. 1, for example. In particular, one embodiment of a bitstream marker may be implemented in a video encoder.

  Bitstream marker 620 receives a compressed video bitstream 602 from video encoder 610 that is configured to encode video source signal 601. Bitstream marker 620 subdivides bitstream 602 into compressed video data, each of which is, for example, a video frame, supplemental information, or video packets that generally include NAL units. Each video packet is then marked with a single subset identifier using the syntax element stream_id in the NAL unit header, as described above. The bitstream marker 620 sends the packetized and marked bitstream 603 to a transport network such as the network 120 and finally a client or peer such as the decoding device 130 described with reference to FIG. Send to a peer in a to-peer network.

  To this end, the bitstream marker 620 includes a receiving unit 621 that receives the compressed video bitstream, a packetization unit 622 configured to split the compressed video bitstream into video packets, and a single subset identifier stream_id. A marking unit 623 for marking each video packet.

  Further, the bitstream marker 620 may optionally comprise a subset definition unit 624 that provides a definition of at least one subset. The definition of the subset may be provided as a video packet in the compressed video bitstream.

  Further, the bitstream marker 620 may optionally comprise a video representation definition unit 625 that provides a definition of at least one video representation. The definition of the video representation may be provided as a video packet within the compressed video bitstream.

  Receiving unit 621, packetizing unit 622, marking unit 623, subset defining unit 624 and video representation unit 625 may be a network, integrated circuit (IC), application specific integrated circuit (ASIC), on one or more processors. It is realized by a computer program module executed in the above or a combination thereof. The units 621 to 625 may be realized as separate units or may be realized in combination.

  It will be appreciated that video encoder 610 may provide packetized video bitstream 602 to bitstream marker 620, in which case bitstream marker 620 need not subdivide bitstream 602 into packets. right. Furthermore, the marking procedure according to one embodiment of the present invention described above may be implemented by the video encoder 610 instead of a separate bitstream marker. Further, the existing video encoder is configured to perform bitstream marking according to an embodiment of the present invention by updating the software of the existing video encoder using an embodiment of the computer program. May be.

  Hereinafter, a bitstream extractor for extracting video packets from a compressed video bitstream according to an embodiment of the present invention will be described with reference to FIG. One embodiment of the bitstream extractor is located, for example, in the decoding device 130 or the network elements 121 to 123 described with reference to FIG. In particular, one embodiment of a bitstream extractor may be implemented in a video decoder or may be implemented in a network element configured to route video packets.

  The bitstream extractor 710 receives a compressed video bitstream 701, video packets, ie, NAL units associated with a plurality of bitstream subsets. The video bitstream 701 is received from a transfer network such as the network 120 described with reference to FIG. The bitstream extractor 710 identifies relevant NAL units contained in the bitstream 701 and extracts them for further processing.

  To this end, the bitstream extractor 710 includes a subset selection unit 711 that provides at least one relevant subset identifier, a reception unit 712 that receives video packets from the video bitstream 701, and for each received video packet, An extraction unit 713 that examines the subset identifier and extracts a video packet from the video bitstream 701 on the condition that the extracted subset identifier matches one of the at least one relevant subset identifier. Optionally, the extraction unit 713 is further configured to forward or decode the extracted video packet and discard (704) the video packet for each received video packet. If the video packet is forwarded, it is transmitted (702) to, for example, video decoder 720. Video decoder 720 decodes the video signal and outputs decoded video signal 703 for further processing, such as display to the viewer. The received video packet is discarded (704) on condition that the extracted subset identifier does not match any of the at least one related subset identifier.

  Optionally, the bitstream extractor 710 may further comprise a subset definition unit 714 that provides a subset definition. The subset definition unit 714 is configured to select a subset definition from the plurality of subset definitions according to at least one property of the corresponding bitstream subset. The subset definition unit 714 may be further configured to receive a subset definition from video packets in the compressed video bitstream.

  One embodiment of a bitstream extractor, such as bitstream extractor 710, may further comprise a video representation definition unit 715 that provides a definition of the video representation. Video representation definition unit 715 may be further configured to receive a video representation definition from a video packet in the compressed video bitstream.

  Subset selection unit 711, reception unit 712, extraction unit 713, subset definition unit 714 and video representation unit 715 may be a network, an IC, an ASIC, a computer program module executing on one or more processors, or a combination thereof. Realized. The units 711 to 715 may be realized as separate units or may be realized in combination.

  It will be appreciated that the procedure of extracting a bitstream subset, i.e., video packet, from a video bitstream may be performed by a video decoder 720 rather than a separate bitstream extractor. Furthermore, the existing video decoder is configured to perform bitstream extraction according to an embodiment of the present invention by updating the software of the existing video decoder using an embodiment of the computer program. Also good.

  Referring to FIG. 8, a computer program and a computer program product according to an embodiment of the present invention are shown.

  FIG. 8 shows a video processing device 800 that processes a video bitstream 801 and outputs a processed video bitstream 802. The video processing device 800 includes a processor 803 and a storage medium 804. The storage medium 804 is a computer program product including a computer program 805. Alternatively, the computer program 805 is transferred to the storage medium 804 by an appropriate computer program product such as a floppy disk or a memory stick. As a further method, the computer program 805 is downloaded to the storage medium 804 via a network. The processor 803 implements an embodiment of the method according to the first aspect or the fourth aspect of the present invention by loading the computer program 805 from the storage medium 804 and executing the computer program code included in the computer program 805. Configured to do. For example, the processor 803 is configured to implement one embodiment of a method for indicating a bitstream subset in a compressed video bitstream when executing the computer program 805. Alternatively, the processor 803 is configured to implement an embodiment of a method for extracting video packets from a compressed video bitstream when executing the computer program 805. The processor 803 may be a general purpose processor, a video processor, or some other type configured to implement an embodiment of the method according to the first or fourth aspect of the invention when executing the computer program 804. It is a network. The processing device 800 may be, for example, a mobile phone, tablet, user equipment (UE), personal computer, video player / recorder, multimedia player, media streaming server, set top box, television, or some other type of device with computing capabilities. May be included.

  Furthermore, all embodiments of the invention described above are implemented in a video encoder or decoder in software, hardware or a combination thereof. The encoder and / or decoder may be implemented in a network node in a communication network between a sending device and a receiving device or belonging to such a network node. Such a network device may, for example, have one video code if it is established that the receiving device only supports or prefers another video coding standard that is not the video coding standard sent from the sending device. It is an apparatus for converting a video that complies with an encoding standard into another video encoding standard. Although the video encoder and / or decoder disclosed above is disclosed as a physically separate device and may be included in one or more application specific circuits such as an ASIC, the present invention Or, an embodiment of an apparatus in which a portion of the decoder is implemented as a computer program module executing on one or more general purpose processors is included in the scope.

  Those skilled in the art will appreciate that the present invention is not limited to the embodiments described above. Many modifications and variations are possible within the scope of the appended claims. For example, the proposed concept of layered stream signal transmission applies in principle to all types of media such as audio, subtitles and graphics. In addition, transmission protocols such as HTTP and RTP are used to transmit the remaining subset of bitstreams containing video data while clients or network elements can advantageously obtain StPS and RPS over a reliable transmission channel. Also good. Finally, it will be further understood that the NAL unit header may include additional information elements in addition to the single subset identifier stream_id.

Claims (32)

A method (410) of indicating a bitstream subset in a compressed video bitstream (210) that includes a plurality of bitstream subsets, comprising:
Receiving the compressed video bitstream (411);
Dividing (412) the compressed video bitstream into video packets (211 to 216) each containing either video data or supplemental information;
Marking each video packet with a subset identifier of a plurality of subset identifiers (413), wherein each subset identifier of the plurality of subset identifiers corresponds to a corresponding bit of the plurality of bitstream subsets A method characterized by being associated with a stream subset.   The method further comprising providing (414) at least one subset definition (221-223, 320) each describing a property of a corresponding bitstream subset of the plurality of bitstream subsets. Item 2. The method according to Item 1.   The method of claim 2, wherein the at least one subset definition (221-223, 320) is provided as a video packet in the compressed video bitstream (210).   The method according to any one of claims 1 to 3, wherein each subset identifier of the plurality of subset identifiers is a numerical value corresponding to a relative priority of an associated bitstream subset.   Providing (415) at least one video representation definition (231-233, 330), each including at least one related subset identifier, associated with the at least one related subset identifier 5. A method according to any one of the preceding claims, wherein the bitstream subset forms a decodable video representation.   The method of claim 5, wherein the at least one video representation definition (231-233, 330) is provided as a video packet in the compressed video bitstream (210).   A computer program (805) configured to implement the method of any one of claims 1 to 6 when executed on a processor (803).   A computer readable medium (804) in which a computer program (805) according to claim 7 is stored. A method (510) of extracting video packets from a compressed video bitstream divided into video packets (211 to 216), wherein the compressed video bitstream (210) includes a plurality of bitstream subsets, each video packet being Including either video data or supplemental information and a subset identifier of a plurality of subset identifiers, each subset identifier being associated with a corresponding bitstream subset of the plurality of bitstream subsets;
Providing (511) at least one relevant subset identifier;
Receiving a video packet from the compressed video bitstream (512);
For each received video packet,
Examining (514) the subset identifier of the video packet;
If the extracted subset identifier matches one of the at least one relevant subset identifier (515), extracting the video packet from the compressed video bitstream (516);
A method comprising the steps of: For each received video packet,
Transferring or decoding the extracted video packet (517);
If the extracted subset identifier does not match any of the at least one relevant subset identifier (515), discard the received video packet (518);
10. The method of claim 9, further comprising: Providing (519) a subset definition (221-223, 320) describing properties of a corresponding bitstream subset of the plurality of bitstream subsets;
Using the subset identifier associated with the corresponding bitstream as the at least one related subset identifier (511);
The method according to claim 9 or 10, further comprising:   The method of claim 11, further comprising selecting (519) a definition of the subset from a plurality of subset definitions (221-223, 320) according to at least one property of the corresponding bitstream subset. Method.   13. The method of claim 11 or 12, further comprising receiving (519) the subset definition (221-223, 320) from a video packet in the compressed video bitstream (210).   Providing (520) a definition (231) of a video representation that includes the at least one related subset identifier, the bitstream subset associated with the at least one related subset identifier; 14. A method according to any one of claims 9 to 13, characterized in that forms a decodable video representation.   The method of claim 14, further comprising receiving (520) the video representation definition (231-233, 330) from a video packet in the compressed video bitstream (210).   The method according to any one of claims 9 to 15, wherein each subset identifier of the plurality of subset identifiers is a numerical value corresponding to a relative priority of an associated bitstream subset.   A computer program (805) configured to implement the method according to any one of claims 9 to 16 when executed on a processor (803).   A computer readable medium (804) in which a computer program (805) according to claim 17 is stored. A bitstream marker (110, 620) indicating a bitstream subset in a compressed video bitstream (210, 602) comprising a plurality of bitstream subsets,
A receiving unit (621) configured to receive the compressed video bitstream;
A packetization unit (622) configured to divide the compressed video bitstream into video packets (211 to 216) each containing either video data or supplemental information;
A marking unit (623) configured to mark each video packet with one of the plurality of subset identifiers, each subset identifier corresponding to a corresponding one of the plurality of bitstream subsets A bitstream marker characterized by being associated with a bitstream subset.   A subset definition unit (624) configured to provide at least one subset definition (221-223, 320) each describing a property of a corresponding bitstream subset of the plurality of bitstream subsets; The bit stream marker according to claim 19, further comprising:   21. The bitstream marker of claim 20, wherein the at least one subset definition (221-223, 320) is provided as a video packet in the compressed video bitstream (210, 603).   The bitstream marker according to any one of claims 19 to 21, wherein each subset identifier of the plurality of subset identifiers is a numerical value corresponding to a relative priority of an associated bitstream subset. .   A video representation definition unit (625) configured to provide at least one video representation definition (231-233, 330), each including at least one relevant subset identifier, the at least one 22. Bitstream marker according to any one of claims 19 to 21, wherein the bitstream subset associated with a relevant subset identifier forms a decodable video representation.   24. The bitstream marker of claim 23, wherein the at least one video representation definition (231-233, 330) is provided as a video packet in the compressed video bitstream (210, 603). A bit stream extractor (121 to 123, 130, 710) for extracting a video packet from a compressed video bit stream divided into video packets (211 to 216), wherein the compressed video bit stream (210, 701) includes a plurality of compressed video bit streams (210, 701). Bitstream subsets, each video packet includes either video data or supplemental information and a subset identifier of a plurality of subset identifiers, each subset identifier corresponding to a corresponding one of the plurality of bitstream subsets Associated with a bitstream subset, the bitstream extractor
A subset selection unit (711) configured to provide at least one relevant subset identifier;
A receiving unit (712) configured to receive video packets from the compressed video bitstream;
For each received video packet,
Examining the subset identifier of the video packet and extracting the video packet from the compressed video bitstream if the extracted subset identifier matches one of the at least one relevant subset identifier An extraction unit (713) configured as follows:
A bitstream extractor comprising: For each video packet received, the extraction unit (713)
Forwarding or decoding (702) the extracted video packet;
26. The method of claim 25, further configured to discard (704) the received video packet if the extracted subset identifier does not match any of the at least one related subset identifier. Bitstream extractor. A subset definition unit (714) configured to provide a subset definition (221-223, 320) describing properties of a corresponding bitstream subset of the plurality of bitstream subsets;
27. The subset selection unit (711) is further configured to use the subset identifier associated with the corresponding bitstream as the at least one relevant subset identifier. The described bitstream extractor.   The subset definition unit (714) is further configured to select the subset definition from a plurality of subset definitions (221-223, 320) according to at least one property of the corresponding bitstream subset. 28. A bitstream extractor according to claim 27.   29. Bitstream extractor according to claim 27 or 28, wherein the subset definition (221-223, 320) is received from a video packet in the compressed video bitstream (210, 701).   The video representation definition unit (715) configured to provide a video representation definition (231-233, 330) that includes the at least one related subset identifier, further comprising the at least one related subset. 30. A bitstream extractor according to any one of claims 25 to 29, wherein the bitstream subset associated with an identifier forms a decodable video representation.   The video representation definition unit (715) is further configured to receive the video representation definitions (231-233, 330) from video packets in the compressed video bitstream (210, 701). The bit stream extractor according to claim 30. The bitstream extractor according to any one of claims 26 to 31, wherein each subset identifier of the plurality of subset identifiers is a numerical value corresponding to a relative priority of an associated bitstream subset. .

Images