STORING OR TRANSMITTING CODING PARAMETERS WITHIN CODED AUDIO-VISUAL DATA
The invention relates to a method and to an apparatus for storing or transmitting coding parameters within coded audio-visual data.
Background
In the field of compressing audio-visual (AV) data the technological trend goes towards increasingly parameter-oriented descriptions. Seen from an encoding point of view, compression parameters in general specify, in which of several pos- sible modes certain parts of the encoder input (AV data) shall be processed. Correspondingly, from a decoding point of view, those same compression parameters indicate, according to which of several allowable rules a section of a given compressed bit stream shall be converted back into a piece of AV data.
Compression parameters can be of different nature; assuming a compression scheme where pictures are subdivided into slices, they may be "global parameters (GP)", "picture layer parameters (PLP)" and "slice layer parameters (SLP)". It is expected that GP are mostly constant, since they express parameters such as picture size or color space. However, they could be changed at the start of a random access unit, such as a GOP. Therefore, a small set of different GP sets will be referenced from within an AV bit stream. PLPs express parameters that might change more frequently, at a maximum on a per-picture basis. Hence, each coded picture will reference one out of a potentially larger set of such PLPs. Similarly, different SLPs could be referenced by each coded slice within a coded picture.
It is expected that GP are mostly predetermined by the nature of the specific AV material being compressed, whereas PLP and SLP can be freely chosen as part of the encoder optimization. For the latter, the selection is likely to be based on some kind of compression efficiency measure quantifying either the coding error or the compression bit rate or both.
The above-mentioned technology will be used in the develop- ing MPEG-4 Part 10, also known as H.26L or JVT codec.
Invention
The invention is based on the recognition of the following facts.
In addition to specifying and appropriately selecting the compression parameters, they themselves must in most cases be transmitted or stored along with the AV data proper, in order to allow a meaningful decoding. This task needs to be tackled separately for each application domain, wherein the transmission or storage of compression parameters has to •fulfil several, partly conflicting requirements.
• Compactness :
On a global as well as local level,, the compression parameter description must be kept compact, such that the total bandwidth needed for AV data plus parameters does not out- weigh any gain in compression efficiency achieved by a parameterized approach. This includes a strong motivation to avoid any unnecessary repetition or other redundancy present in the compression parameters. Consequently, the indexing mechanism from a coded picture or slice to the parameter set(s) will bound the number of parameter sets of each type that can be addressed, in order to
reach the compactness goal. Hence, it must be specified whether it is possible that two different parameter sets are identified by the same (numeric or other) label . In other words, a name scope for the labels of parameter sets needs to be established.
• Decodability:
With AV data, especially on a storage medium, being prone to be accessed in non-sequential, random ways, any parameter set allocation scheme must ensure that all required compres- sion parameters are made available when entering and decoding a bit stream at any of its entry points. This includes the important cases of any kind of trick play, and may entail allowing certain controlled forms of data redundancy.
• Compatibility: When designing an extension of an existing specification, such as DVD for optical storage, it is desirable to maintain as many as possible of the well-understood and established system concepts. Solutions taking this into account are likely to be judged favorably in the standardization proc- ess.
A problem to be solved by the invention is to provide a method that fulfills the above requirements.
This problem is solved by the method disclosed in claim 1.
An apparatus that utilizes this method is disclosed in claim 6. A corresponding decoding method and apparatus is claimed in claim 7 and 8, respectively.
According to the inventive method a number of coding parameter sets being referenced from within the coded data of a picture or of a part of a picture are used. A first group of coding parameter sets that are being referenced within the first picture of a group of independently accessible coded pictures are inserted before the coded data of said first picture. A second group of said coding parameter sets that
are being referenced in subsequent coded pictures of said group of independently accessible coded pictures are inserted after the coded data of said first picture.
Therefore, compression parameter sets needed for decoding are instantaneously available at an entry point, hence optimizing for decodability without sacrificing too much bit rate .
Advantageously, the second group of said coding parameters sets is placed immediately after the coded data of said first picture.
Advantageously, each coding parameter set of said second group of coding parameter sets is individually placed within said group of independently accessible coded pictures before the coded data of that picture where said coding parameter set is first being referenced.
Furthermore, it is of advantage that the numeric or other labels of each said coding parameter set need only be unique within each of said groups of independently accessible coded pictures.
According to a further advantageous embodiment both the coded data of pictures or parts of pictures as well as the coding parameter sets are conveyed in data packets for placement in an underlying storage or transmission format or protocol .
Further advantageous embodiments of the invention result from the following description.
Drawing
Exemplary embodiments of the invention are described with reference to the accompanying drawings, which show in:
Fig.l an AV stream with separate parameter sets; Fig.2. an AV stream with in-band parameter sets; Fig.3. an AV stream with in-band parameter sets, only included before their first reference.
Exemplary embodiments
An audiovisual data format on a storage medium as well as in a broadcast transmission system will usually exhibit individually accessible data units (IADU) , that permit the playback to be started at multiple locations within the AV bit stream. In the realm of MPEG video compression, IADU corresponds to the concept of "Group of Pictures" (GOP) , in DVD to "Cell" or "Video Object Unit" (VOBU) .
Fig.l shows an IADU of an AV stream with separately generated parameter sets. The IADU comprises pictures pi, p2 , p3,pi,... each subdivided into slices si, s2,..., sn. For the slices there are pointers to separately generated compression parameters, exemplarily, to global parameters GP1 and GP2, picture layer parameters PLP1, PLP2 and PLP3, and slice layer parameters SLP1, SLP2, SLP3, SLP4 and SLP5. The IADU commonly starts with an entry picture pi (intra- coded frame) that is decodable independent from any other data in the AV bit stream, with the exception of the reference to the needed parameter sets, e.g. one reference each to a global parameter GP1, a picture layer parameter PLP1 and slice layer parameters SLP1, SLP2 for each coded slice within the picture.
According to the invention the initially separate parameter sets shown in Figure 1 are reinserted into the main AV bit stream. This eliminates the name scope problem for parameter set labels and allows for easy random access into the AV bit stream without the need to read any auxiliary data from elsewhere on the medium.
Those parameter sets GP1, PLP1, SLP1, SLP2 that are being referenced within the entry picture pi are reinserted before the coded data of that same entry picture as depicted in Figure 2 or, alternatively, in a systems-level container (e.g. NAV PACK in DVD) that precedes the entry picture pi. The reinserted parameter sets can now be referenced by their label from any picture within the IADU. The name scope of the label is limited to the IADU. Other pictures after the entry picture may reference other parameter sets that have not yet been conveyed.
These parameter sets PLP2, SLP3, SLP4, SLP5 are conveyed right after the coded data of the entry picture. This has the advantage that those parameter sets need not to be read in trick mode, when only the entry picture of each IADU is needed. Parameter sets GP2 , PLP3 referenced not at all in this IADU are not inserted.
Furthermore, to ease the adaptation of AV data where parameter sets are generated on the fly as needed during encoding, an option may be foreseen, to insert each parameter set at any valid location preceding the point in the bit stream where it is first being referenced as depicted in Figure 3. The parameter sets GP1, PLP1, SLP1, SLP2 are located before the entry picture pi, entry picture, the picture layer parameter PLP2 and slice layer parameters SLP first referenced in picture p2 are inserted before the coded data of picture p2, the slice layer parameters SLP4, SLP5 first referenced in picture pi in front of picture pi .
One specific embodiment of the present invention, assuming carriage of MPEG- Part 10 video content over MPEG-2 Systems transport, requires that, within a PES stream, any network abstraction layer (NAL) units carrying a parameter set with a specific numeric label (parameter_set_id) shall precede any NAL units conveying coded video data which either directly reference said specific parameter set or for the decoding of which said specific parameter set is necessary. Furthermore, such parameter set NAL units shall be retained by the decoder for further reference within the name scope of the parameter_set_ids . Finally, the definition of the name scope for parameter_set_ids could explicitly be delegated to application specifications, such as an extension to the DVD specification.
As an embodiment of the present invention on the DVD side, a future extension of DVD encompassing parameterized video compression might be based on the following: That the scope of parameter_set_ids of Global Parameters (GPs) corresponds to a Cell, and therefore video data in the first video packet in a Cell shall start with NAL units containing all GPs being referenced within that Cell, followed by video data for the VOBUs of the Cell; and these GPs shall be re- tained in the decoder until the end of the Cell playback. Further, the scope of parameter_set_ids of PLPs and SLPs corresponds to a VOBU, and therefore video data for each VOBU shall start with NAL units containing all PLPs and SLPs being referenced within the first reference picture of the VOBU; subsequent NAL units might also contain further PLPs and SLPs, using different parameter_set_ids, if they are being referenced in subsequent pictures of the VOBU; and any such PLPs and SLPs shall be retained in the decoder until the end of the VOBU playback.
An advantage of the invention is that placing the entry pic-
ture parameter sets between the entry point and the subsequent entry picture data optimizes access speed and data availability. Placing the other needed parameter sets after the coded data of the entry picture avoids having to read such data in trick modes where they are not needed.
The necessity to repeat all used parameter sets for each IADU slightly increases the bit rate of the overall stream, however, since each used parameter set occurs only once in the bit stream, this solution is still superior to classical repetition of coding parameters on a picture or slice basis.
The invention is especially useful for transport or storage of AV bit streams. However, the invention is also applicable to other kinds of bit streams.