KR20070012808A - Audio bitstream format in which the bitstream syntax is described by an ordered transveral of a tree hierarchy data structure - Google Patents

Abstract

A bitstream format for representing audio information in which the bitstream syntax is described by an ordered transversal of a tree hierarchy data structure, has a tree hierarchy comprising a plurality of tree hierarchy levels, each having one or more nodes, in which at least some progressively smaller subdivisions of the audio information are represented in progressively lower levels of the tree hierarchy, wherein the audio information is included among nodes in one ore more of said levels. ® KIPO & WIPO 2007

Description

AUDIO BITSTREAM FORMAT IN WHICH THE BITSTREAM SYNTAX IS DESCRIBED BY AN ORDERED TRANSVERAL OF A TREE HIERARCHY DATA STRUCTURE}

The present invention relates to a bitstream format for representing audio information in which a bitstream is described by sequential traversal of a tree hierarchical data structure, a bitstream formatted according to the bitstream format, a medium for storing or transmitting such a bitstream, A system for encoding and decoding a bitstream having a format according to such a bitstream format, an encoder for encoding a bitstream having a format according to such a bitstream format, a decoding of a bitstream having a format according to such a bitstream format A decoder for, a process for encoding and decoding a bitstream having a format according to such a bitstream format, a process for generating a bitstream having a format according to such a bitstream format, a bitstream having a format according to such a bitstream format Encoding Relates to a process, a process for decoding a bitstream having a format in accordance with such a bit stream format.

<Start of invention>

According to aspects of the present invention, a bitstream format for representing audio information in which a bitstream is described by sequential traversal of a tree hierarchical data structure has a tree hierarchy comprising a plurality of tree hierarchy levels, each tree hierarchy level With one or more nodes, at least some progressively smaller subdivision of audio information is represented by progressively lower levels of the tree hierarchy, with audio information included among the nodes of one or more levels. Incrementally smaller subdivisions of audio include one or more temporal subdivisions, spatial subdivisions, and resolution subdivisions. The first level of the tree hierarchy includes a root node representing all the audio information and at least one lower level includes a plurality of nodes representing the temporal segmentation of the audio information, and the at least one lower level represents the spatial segmentation of the audio information. It may include a plurality of nodes representing. Alternatively, or in addition, the audio information provides a plurality of resolutions such that the basic resolution audio information layer is included in one level and the one or more audio information resolution enhancement layers are included in the same layer or in one or more other levels. It may be a layer structure so as to. Other aspects of the invention are disclosed throughout the description and claims.

The bitstream format according to aspects of the present invention is

-Minimize audio processing latency,

Adding, removing, or otherwise manipulating metadata without extensive modification to the bitstream,

Associating arbitrary metadata with specific aspects of the audio material contained in the bitstream,

-Minimize bitstream structural overhead,

-Provide flexible bitstream structure for future / existing compatibility

Enabling efficient transmission over various interfaces,

-Enabling frame-based editing,

-Enabling the encapsulation of encoded or unencoded audio information

 It may be useful for one or more of the.

Definitions and examples of tree hierarchical data structures can be found in NIST, the National Institute of Standard and Technology, website "Dictionary of Algorithms and Data Structures" (http://nist.gov/dads/). Demonstration of sequential traversal of tree hierarchical data structures can be found at Data Structures, Algorithis, Binary Tree Traversal Algorithm (http://www.cosc.cantebury.ac.nz/) of the Department of Computer Science, University of Canterbury (New Zealand) website. people / mukundan / dsal / BTree.html).

1A and 1B are schematic diagrams illustrating hierarchical tree representations of audio information (sometimes referred to herein as "audio essences") components of a bitstream and this bitstream in accordance with aspects of the present invention.

FIG. 2 is a schematic diagram illustrating an example of a hierarchical tree representation similar to FIG. 1B but including metadata.

3 is a schematic diagram illustrating a serialized bitstream, in accordance with aspects of the present invention, as a result of traversal in the order of the tree hierarchy of FIG. FIG. 2 differs from FIG. 1B in that it also shows segments of metadata appended to the beginning and / or end of each node.

4A-4D are schematic diagrams illustrating a transcoding process using a bitstream in accordance with aspects of the present invention.

5 is a schematic diagram of the structure of a node in a tree hierarchy in accordance with aspects of the present invention.

6 is a schematic diagram of the structure of a short node.

7 is a schematic diagram of an example of a hierarchical tree in accordance with the present invention.

8A is a schematic diagram illustrating the mapping of two AC-3 sync frames to a bitstream in accordance with aspects of the present invention.

8B is a schematic diagram of the AC-3 encapsulation bitstream of FIG. 8A with two complementary audio channels added.

9 is a flow diagram or functional block diagram illustrating various functional aspects of an encoder or encoding process for generating a bitstream similar to the example of FIG. 3, in accordance with aspects of the present invention.

FIG. 10 is a flow diagram or functional block diagram illustrating various functional aspects of a decoder or decoding process for deriving audio essence and metadata from a bitstream such as the examples of FIGS. 3 and 9, in accordance with aspects of the present invention. .

1A and 1B are schematic diagrams illustrating hierarchical tree representations of audio information (sometimes referred to herein as "audio essences") components of a bitstream and this bitstream in accordance with aspects of the present invention. The bitstream representation of FIG. 1A represents two consecutive audio frames, each having first and second channels, namely channel 1 and channel 2. FIG. The latter may correspond, for example, to audio information to be reproduced by the left and right speakers, respectively. Channels 1 and 2 are labeled 1a and 2a in the first frame and 1b and 2b in the second frame. In FIG. 1A, the vertical direction represents channels and the horizontal direction represents frames and time.

In the example of FIG. 1B, the tree hierarchy underlying the bitstream of FIG. 1A, in accordance with aspects of the present invention, has Level 1, Level 2, and Level 3 as three levels. At level 1 a single root node 3 represents the audio material of the entire bitstream. Indeed, as will be described later, the bitstream format and the representation of the tree hierarchical data structure underlying the "audio material" include audio information or audio "essence", "metadata" which is information about audio essence, and other data. can do. In this simple example, however, only audio essence is shown with respect to the tree hierarchy of the bitstream.

At level 2 of the hierarchical example of this example, the audio material can be broken down into any number of individual audio frames, each of which is fixed or variable periods or bit lengths (for simplicity of representation, only two frames). This is illustrated in the example of FIGS. 1 and 1B). Frame nodes 4, 5 each having a root node 3 as its parent represent the first and second audio frames, respectively, at level 2 of the hierarchy of this example. Each audio frame is decomposed into any number of audio channels (for simplicity in presentation, only two channels per frame are shown in the examples of FIGS. 1A and 1B), each of which is, for example, " Correspond to spatial directions such as "left" and "right". The channel nodes 6, 7, 8, and 9 each have their own frame node as their parent, with audio channels 1a, 2a, 1b, 2b in successive frames at level 3 of the hierarchy. Respectively.

 In the example of FIG. 1B, the channel nodes 6-9 are leaf nodes and each has an audio essence in the form of at least one essence element. In principle, although audio essence does not need to be implied in leaf nodes, it is advantageous to actually place audio essence in leaf nodes, as one will read and understand the description of the present invention. In the case of "layered" audio, such as in which the basic resolution layer of the independence is provided along with the enhancement layers of the above exploded view, the audio essence is placed in the leaf nodes and in the nodes of the layers of one or more higher layers).

Whatever is placed within the layer, it is a feature of the present invention that the audio essence is in one or more nodes of the layer, and hence the audio essence is present in the resulting bitstream. This does not exclude, for example, the possibility that information relating to encoding or decoding or audio essence lies outside the bitstream and its underlying layer. For example, a pointer to metadata associated with an audio essence may point to a specific decoding process other than the bitstream and its underlying layer.

As mentioned above, the bitstream format and the underlying tree hierarchical data structure representation of "audio material" may include not only audio information or audio "essences" but also "metadata" and other data relating to audio essences. .

A useful discussion of audio metadata is "Exploring the AC-3 Audio Standard for ATSC" in Audio Notes by Tim Carroll, June 26, 2002, at http://tvtechnology.com/features/audio_notes/f-TC-AC3- 06.26.02.shtml, "A Closer Look at Audio Metadata" in Audio Notes by Tim Carroll, July 24, 2002, at http://tvtechnology.com/features/audio_notes/f-tc-metadata.shtml and "Audio Metadata : You Can Get There From Here "in Audio Notes by Tim Carrol, August 21, 2002, at http://tvtechnology.com/features/audio_notes/f-TC-metadata-08.21.02.shtml. Each document is incorporated herein by reference in its entirety.

Bitstreams based on hierarchical representations in accordance with aspects of the present invention allow for the precise synchronization of this information with the audio essence described by any metadata information. This may be accomplished by placing metadata to associate with a particular audio essence within the same node as the audio essence or within any parent node of the node that contains the audio essence. In accordance with an embodiment of the present invention, as described below, one or more metadata elements may be appended to the beginning or end of any node in the hierarchy. Thus, in a three-level hierarchy such as in the example of FIG. 1B, the metadata associated with a particular audio essence is level 2 which is the beginning or end of the audio material of the entire bitstream in the root node of level 1, or the parent of a channel with a particular audio essence May be appended to the beginning or end of each frame in a frame node, and / or to the beginning or end of a channel in a channel (leaf) node at level 3 containing a specific audio essence. Examples of such configurations are described below in the example of FIG. 2.

Preferably, metadata is distributed between hierarchical levels in a manner that contributes to "significant independence" of the individual nodes. For example, in the configuration of the FIG. 1B type, the metadata at the root node preferably applies only to the entire audio material, and the metadata to the frame node preferably applies only to the particular frame and its channels, and to the channel node. The data preferably applies only to specific channels. By appropriate definition of metadata information, it can be assured that manipulation of a given node does not require modification of the metadata carried on other nodes. For example, when a frame node does not contain metadata specific to any particular channel node and the channel node does not contain metadata required by another channel node, not only the channel metadata but also the entire channel node is another node. It can be added, removed, or modified without modifying the metadata in it. In this respect, features of the invention make the nodes semantically independent. That is, in terms of metadata and essence, any given node can be independent of its sibling if it has metadata that can apply equally to itself and to all its children (if any). As a result, the bitstream according to the present invention with suitably distributed metadata facilitates transcoding, as described below.

The bitstream according to the present invention is generated using an ordered traversal of the tree hierarchical data structure to serialize the hierarchical representation of the audio material. Preferably, the traversal in order is similar to preorder traversal (also called "prefix traversal"). The preorder traversal algorithm may be defined as processing all nodes of a tree by processing the root node and then recursively processing all sub-trees. In particular, if body tags are not employed (see below for "body tags"), a suitable preorder traversal algorithm for use in serializing a layer in accordance with aspects of the present invention, starting from the root node, applies the following algorithm: Can be described.

a) A "start tag" segment indicating the start of the node may be written to the bitstream.

b) Each of the one or more metadata or essence elements attached to the beginning of the node may be written as a separate segment.

c) Starting from step "a", the algorithm is applied to each of the child nodes of the node under consideration.

d) Each of the one or more metadata or essence elements attached to the end of the node may be written as a separate segment.

e) A "end tag" segment indicating the end of the node may be written to the bitstream.

The traversal algorithm may be expressed as a simplified C language-like code as follows.

If body tags are employed, a suitable preorder traversal algorithm can be described by applying the following algorithm, starting from the root node.

a) A "start tag" segment indicating the start of the node may be written to the bitstream.

b) Each of the one or more metadata or essence elements attached to the beginning of the node may be written as a separate segment.

c) Steps d) to g) can be omitted if the root node has no child nodes and no metadata or essence elements are appended to it.

Starting from step "a", the algorithm is applied to each of the child nodes of the node under consideration.

d) A "body start tag" segment indicating the start of a child node of a node may be written to the bitstream.

Each of the one or more metadata or essence elements appended to the end of the node may be written as a separate segment.

e) Starting from step "a", an algorithm is applied to each of the child nodes of the node under consideration.

f) A "body end tag" indicating the end of a node's child node may be written to the bitstream.

g) Each of the one or more metadata or essence elements attached to the end of the node may be written as a separate segment.

h) A "end tag" segment indicating the end of the node may be written to the bitstream.

FIG. 2 shows a simple example of a hierarchical tree representation similar to FIG. 1B but including metadata. 3 illustrates a serialized bitstream as a result of traversal according to the order of the tree hierarchy of FIG. 2.

FIG. 2 differs from FIG. 1B in that it also shows segments of metadata appended to the beginning and / or end of each node. To indicate that the nodes are modifications to the nodes of FIG. 1B, reference signs with prime symbols have been used in FIG. 2. Thus, the root node 3 'has, for example, title and copyright metadata attached to its beginning. Frame nodes 4 ', 5' have, for example, a timecode appended to the beginning of each node and loudness metadata appended to the end of each node. Channel nodes 6 ', 7', 8 ', 9' have, for example, downmix metadata appended to the beginning of each node.

3 illustrates an example of a serialized bitstream and a layer according to the present invention. The bitstream has segments (also referred to as "atomic elements") 10 to 37, resulting from the sequential traversal of the hierarchy of FIG. 2 according to the "without body-tags" algorithm above. Each element, whether with audio essence or metadata, or with other data, is labeled using a unique identifier that indicates its contents. Suitable identifiers are described below.

As will be described later, the root node 3 'includes segments 10 to 37 which are all audio material. Nesting of frame nodes 4 ', 5' in the root node 3 'and then nesting of channel nodes in each frame node are shown in FIG. The bitstream of the example of FIG. 3 is a metadata (title) segment 11 and metadata (copyright) (segment) 12 beginning at the root node start tag segment 10 representing the beginning of the audio material and attached to the root node start. Is followed. Subsequently, a visit is made to the frame node 4 'which is the first child and indicated by the frame node start tag 13 followed by the metadata (timecode) segment 14 attached to the beginning of the frame node 4'. . Next, as indicated by the channel node start tag 15, a visit is made to the channel node 6 ', which is the first child of the frame node. Following the channel node start tag segment, comes the metadata (downmix) segment 16 attached to the start of the channel node 6 '. Following the metadata segment 16, the channel node 6 '(channel 1) audio essence 17 and the channel node end tag 18 are followed. Next, a visit is made to the channel node 7 'which is the second child of the frame node 4' as indicated by the channel node start tag 19. The channel node start tag segment is followed by a metadata (downmix) segment 20 appended to the beginning of the channel node 7 '. The metadata segment 20 is followed by the (channel 2) audio essence 21 and the channel node end tag 22 of the channel node 7 '. Since there are no other children of the frame node 4 'and the channel nodes 6', 6 'are leaf nodes, they are revisited to the frame node 4' so that the loudness metadata 23 is written. (The loudness metadata depends on the process of visiting the audio essence of the channels 1, 2 to determine the value of the loudness metadata). The end of frame tag segment 24 is then written to the bitstream. Then, the next frame node 5 'is visited.

In a similar manner as has now been described with respect to the sub-tree of the frame node 4 ', the bitstream is written by the frame 5' and its child leaf nodes 8 ', 9', so that the frame start segment ( 25), metadata (timecode) segment 26, channel node start tag 27, channel node metadata (downmix) segment 28, (channel 1) channel node audio essence segment 29, channel node End tag 30, channel node start tag 31, channel node metadata (downmix) segment 32, (channel 2) channel node audio essence segment 33, channel node end tag 34, frame node End metadata (sound strength) 35 and end 36 of the frame tag segment are generated. Since this simple example has only two frames, it is revisited to the root node. Unless there is metadata appended to the end of the root node, the root node end tag segment 37 is written to indicate the end of the audio material.

In addition to being semantically independent, as mentioned above, each segment is structurally independent in that each segment has its own type and length, no other segments and no nesting within another segment. Therefore, a segment can be processed without prior knowledge of other segments, and as a result the bitstream can be parsed into one segment at a time, enabling low latency computation. In addition, adding, deleting, and modifying a node or segment does not necessarily require manipulation of any other node or segment.

With this structural flexibility, if metadata and audio essences are optimally distributed, segments, and in fact entire nodes, can be added, removed, and manipulated without affecting other segments and nodes. This makes it possible, for example, to remove a particular audio channel from some audio material without having to remaster the entire bitstream. In particular, the nodes preferably do not contain any length or synchronization information that may require systematic modification (ie, modification of other nodes in the bitstream). The length information is not needed because the start tags and the end tags delimit the node. Synchronization information is not necessary because the presence of a segment in a node is clearly synchronized with the content of the node. On the other hand, metadata and / or audio essences may be distributed, for example, to create dependencies between nodes at certain levels of the hierarchy, in which case the latency will be increased. For example, certain embodiments of aspects of the present invention may require that each frame node include a time stamp and that the time stamps are consecutive. At this time, removal of one frame node would require modification of all subsequent frame nodes, which would be an undesirable design decision.

As mentioned above, each element in the hierarchy, whether including audio essence, metadata, or other data, is preferably labeled using a unique identifier representing its contents. Therefore, a given application receiving a bitstream formatted according to the present invention may ignore elements it does not recognize. This allows new types of elements to be introduced into the bitstream without disturbing existing applications. For example, one or more audio essence enhancement layers may be added to the bitstream along with the associated metadata to allow for existing and future compatibility. Alternatively, one or more enhancement layers may be included in the metadata.

4A-4D illustrate a transcoding process using a bitstream in accordance with aspects of the present invention. Segments are processed in series as they appear in the bitstream. 4A illustrates a two channel bitstream in accordance with the present invention prior to the transcoding process. Segments (a) and (b) have audio information corresponding to channel 1 and channel 2 of frame 1. In FIG. 4B, the transcoding process has read six segments when it encounters segment (a) containing audio information. It reads the segment from the bitstream, extracts the audio information, transcodes the audio information into the target format, and wraps the audio information back into segment (a ') instead of writing (a) into the bitstream. . If the channel nodes are not interdependent in the context of transcoding, there is no need to know about previous or future nodes. This is important for low latency operations, which may start before the entire bitstream or most of the bitstream is received by the transcoding process. In FIG. 4C, the transcoding process reaches segment (b) which writes as segment (b ') in the manner described in connection with FIG. 4B. 4D shows a fully transcoded bitstream.

The following describes an embodiment of aspects of the present invention. The invention is not limited to this or other embodiments. Although the following description discloses the syntax and syntax of the bitstream, the structure of the atomic elements of the bitstream, and the adaptive arrangement of these elements, it does not describe the semantic content of the bitstream, such as the relationship between metadata and audio essences. Do not. This relationship is outside the scope of the present invention.

The terms employed herein and in particular in connection with this embodiment may be defined as follows.

Inherent audio material: Audio information represented by a self-built bitstream comprising nodes and segments and formatted according to aspects of the present invention.

Node: Zero or more consecutive bitstream segments that belong to the hierarchy level and are scoped by start tag and end tag pairs. Nodes can be nested.

Segment (atomic element): The smallest bitstream element that can be manipulated (eg, packaged or encrypted) as a distinct entity. The three stream segments are the Audio Essence Segment, the Metadata Segment (the Audio Essence and Metadata Segment are "Content" Segments), and the Tag Segment (Tag Segments can associate the bitstream and tree hierarchy with each other, for example). "Structural" segment). A segment may have information about its length, type, and / or content.

Audio Essence Segment: A content segment with audio essence (audio information). An audio essence segment is for example a series of unencoded pulse code modulation (PCM) audio data or encrypted PCM audio data (eg, perceptually encoded PCM).

Metadata Segment: A content segment with metadata information related to the associated audio essence.

Tag Segment: A non-content segment used to scope nodes.

Frame: a bitstream node comprising one or more audio essence segments representing a time interval of audio material and one or more metadata segments related to these audio essence segments.

A group of frames: a series of frames preceded by one or more metadata segments, optionally accompanied by one or more additional metadata segments.

Bitstreams formatted in accordance with the present invention are defined independently of audio coding, audio metadata, and transport methods, and thus may not include features such as metadata specific to error correction and compression.

Segment

As noted above, a segment or atomic element is the smallest bitstream element that can be manipulated (eg, packaged or encrypted) as a distinct entity. In practice, each segment may be a structure arranged by byte, including a header containing type and size information, and in the case of audio essence and metadata segments, a payload. Tag segments contain structure information and have no payload. Content segments embed metadata or essence information as their payloads. The type of segment and its meaning can be further detailed using unique identifiers. Segment syntax is specified in detail below.

Node

Segments are arranged into nodes, which are hierarchically nested structures. In this embodiment, a node may consist of a series of segments delimited by matching the start tag segment and the end tag segment. As shown in FIG. 5, the structure of a node in the tree hierarchy has a header 40, body 41, and Taylor 42 contexts as three distinct contexts (or portions). Header and Taylor contexts may each have one or more content segments, and the body context may have zero or more child nodes. Optionally, the body context can be bounded by body start- and end-tag segments.

Referring to the details of FIG. 5, the node structure starts with a start tag segment 43 and ends with an end tag segment 44. Each of the tag segments 43, 44 is marked with an "X" because the type of tag depends on the position of the node in the hierarchy. In the case of the root node in this embodiment, the tag segments may be a group of frame (GOF) tags. After the start tag 43, the header context 40 can have one or more content segments 45. The body start tag 46 may then have one or more body contexts 41 including one or more nodes 47 nested at one or more hierarchical levels below the node shown in FIG. 5. Body end tag 48 may delimit the end of body context 41. Following the body end tag 48, the Taylor context 42 may have one or more content segments 49. Finally, the node structure ends with an end tag segment 44.

This may occur in the case of a leaf node containing audio essence and possibly related metadata, where both body tags and the Taylor context are empty, the body tags may be omitted and the node may be short as shown in FIG. Become a node. Short nodes are limited to header context 40 'because they cannot distinguish between header context and footer context in the absence of body tags. Referring to the details of FIG. 6, the node structure begins with a start tag segment 50 and ends with an end tag segment 51. As in the case of the example of FIG. 5, tag segments are marked with an "X" because the type of tag depends on the position of the node in the hierarchy. In the case of a leaf node in this embodiment, the tag segment may be channel tags. The header context 40 'is between the start tag and the end tag and includes one or more content segments 45'.

Hierarchy

The hierarchical structure of the bitstream can be specified by the structure of the body context of the nodes. The content and semantics of the header and Taylor contexts associated with the nodes are specific to the environment in which the bitstream format of the present invention is employed and do not form part of the present invention.

For ease of scalability, content segments and nodes other than context may be omitted and ignored by an application that receives and processes a bitstream formatted according to aspects of the present invention. However, nodes that are within the context but out of order can be treated as errors. "In context" refers to segments and nodes defined as belonging to a particular node context. For example, as discussed below, the top channel (TOC) node will be in the context when it is in the frame body but out of context if it is in the GOF node. This approach enables future compatibility by allowing future applications to insert additional content segments and nodes while maintaining compatibility with older applications.

As shown in Figure 7, a bitstream in accordance with aspects of the present invention is a hierarchy having a series of one or more groups of frames (GOF) nodes at the root. Only GOF nodes are in context at the root node of this example.

Nodes of a Group of Frames (GOF)

GOF nodes 60 ... 61 (FIG. 7) are entities having information necessary for accurately reproducing portions of audio material carried in the bitstream. Frame nodes are nested within each GOF node. Ideally, the GOF node has enough information to be easily manipulated (eg, followed) at the GOF boundary of the bitstreams.

 Node name  GOF  tag_id  (See below)  Nodes in context  frame  Body structure  Zero or more frame nodes

Frame nodes

Frame nodes 62 ... 63 (FIG. 7) contain metadata information corresponding to audio essences and time intervals. One top channel (TOC) node and one bottom channel (BOC) node may be nested within each frame node. Metadata at the frame level can complement what is already found at the GOF level and can be changed across frame nodes. Frame nodes can be independent if metadata at the frame level does not change over frames. Although not a requirement, the frames may be synchronized with the accompanying picture essence. Alternatively, the channels may be grouped into three or more nodes or the channels may be nested directly under each frame such that the channel nodes are nodes in the context.

 Node name  frame  tag_id  (See below)  Nodes in context  TOC and BOC  Body structure  One TOC node followed by one BOC node

TOC and BOC Nodes

Each of the TOC and BOC nodes has metadata and essence information corresponding to approximately half of the information contained in the frame. This configuration can reduce latency by allowing encoders and decoders to begin processing the frame before receiving or transmitting the entire frame. TOC and BOC body contexts have zero or more channel nodes.

 Node name  TOC  tag_id  (See below)  Nodes in context  channel  Body structure  Zero or more channel nodes

 Node name  frame  tag_id  (See below)  Nodes in context  channel  Body structure  Zero or more channel nodes

Channel node

Each channel node represents a single independent essence entity and typically has one or more essence segments accompanied by zero or more metadata segments. In this bitstream format embodiment, if the body of the channel node is empty and no taylor is defined, the node structure may take the form of a short node.

 Node name  channel  tag_id  (See below)  Nodes in context  <None>  Body structure  <Empty>

Segment specification

Segments can be specified in more detail by the following pseudocode, based on the simplified C language syntax. For chunk elements larger than 1 bit, the arrival order of the bits is always MSB first. Fields or elements included in the frame are shown in bold.

Tag Segment Parameters

"is_tag" parameter

Word size: 1

Effective range: 1

The tag segment always has an is_tag value of 1.

"start_or_end" parameter

Word size: 1

Valid Range: 0 (start), 1 (end)

The value of this parameter indicates whether the tag is a start tag (0) or an end tag (1).

"is_long_id" parameter

Word size: 1

Valid Range: 0 (5-bit id field), 1 (13-bit field)

The value of this parameter indicates whether the tag_id field is 5 bits or 13 bits.

"tag_id" parameter

Word size: 5 to 13 (see previous parameter)

Valid Range: [0..31] or [0..2 ¹³ -1]

The value of this parameter indicates which tag the segment represents. The following tags can be defined.

 tag  Tag Description  0  Frame tag  One  Best Channel (TOC) Tag  2  Lowest Channel (BOC) Tag  3  Channel tag  4  Tags of a group of prems (GOF)  5  Body tag

Content segment parameters

"is_tag" parameter

Word size: 1

Valid Range: 0

The content segment always has an is_tag value of zero.

metadata_or_essence parameter

Word size: 1

Effective range: 0 (metadata), 1 (essence)

The value of this parameter indicates whether the segment contains metadata (0) or essence (1).

"is_long_id" parameter

Word size: 1

Valid Range: 0 (5-bit id field), 1 (13-bit id field)

The value of this parameter indicates whether the content_id field is 5 bits or 13 bits.

"content_id" parameter

Word size: 5 to 13 (see previous parameter)

Valid Range: [0..31] or [0..2 ¹³ -1]

The value of this parameter uniquely identifies the type of information contained in the segment.

"content_length_class" parameter

Word size: 2

Valid Range: [0..3]

The content_length_class parameter may determine the maximum length of a segment according to the following table.

content_length class content_length parameter depth 0 6 bit One 14 bit 2 22 bit 3 30 bit

"content_length" parameter

Word size: (content_length_class + 1) * 8-2

Valid Range: [0..63] (content_length_class == 0)

           [0..16383] (content_length_class == 1)

           [0..2 ^ 22] (content_length_class == 2)

           [0..2 ^ 30] (content_length_class == 3)

The content_length parameter determines the total length of the payload in bytes.

Example of Encapsulation of AC-3 Serial Coded Audio Bitstreams

As mentioned above, the encoded audio information can be encapsulated as segments of the bitstream formatted according to aspects of the present invention. As an example of this, the essential parts of the AC-3 serial coded audio bitstream may be encapsulated in the following manner.

AC-3 digital audio compression standards are described in the ATSC standard: Digital Audio Compression (AC-3), Revision A, Document A / 52A, Advanced Television Systems Committee, 20 August 2001 (the "A / 52A Document"). The entirety of A / 52A is incorporated herein by reference.

AC-3 bitstream syntax is described in section 5 (and elsewhere) of the A / 52A document. An AC-3 serial coded audio bitstream consists of a series of synchronization frames (“sync frames”). 8A illustrates matching two AC-3 sync frames to a bitstream in accordance with aspects of the present invention. Each AC-3 sync frame contains six coded audio blocks AB0 through AB5, each of which represents 256 new audio samples. At the beginning of each frame, a synchronization information (SI) header has the information needed to obtain and maintain synchronization. The Bitstream Information (BSI) header follows the SI and has parameters that describe the coded audio service. Coded audio blocks may be followed by an auxiliary data (Aux) field. Often, the ancillary data includes the pressed "padding" bits needed to adjust the bit length of the AC-3 frame. In some cases, however, the auxiliary data has information. At the end of each frame is an error check field containing a CRC word for error detection. An additional CRC word is placed in the SI header, the use of which is optional.

FIG. 8A shows the mapping of two AC-3 sync frames to a bitstream consisting of a group of frame nodes consisting of two frame nodes, each representing one or more AC3 channels. Metadata items included in the Si and BSI headers are divided into two groups: (1) metadata items specific to the frame, eg timecode, and (2) metadata specific to each of AC and its channels. Divided into. Specific metadata is wrapped in the "GFM" metadata segment, and specific metadata is wrapped in the "AC3M" metadata segment. Aux blocks are wrapped in Aux segments if they have user bits, and may be omitted if they are used only for padding. Since a given bitstream may move across various interfaces that provide some of their own error correction mechanisms, error correction and detection information may be omitted (CRC blocks may be omitted (not shown)).

In particular, in FIG. 8A, two AC-3 sync frames are shown, each containing SI, AB0 to AB5, Aux and CRC elements in order. A bitstream according to aspects of the present invention, in which two AC-3 sync frames are mapped for encapsulation, is first a GOF first tag followed by a frame start tag (FRM), a unique frame metadata (GFM), an AC-3 channel start tag. (AC3), AC-3 specific metadata (AC3M), AC-3 content segments (AB0 through AB5 and Aux), AC-3 channel end tag (AC3), end frame tag (FRM), and second AC -3 contains the same column mapped from the sync frame.

8B shows the AC-3 encapsulated bitstream of FIG. 8A with the addition of two supplemental audio channels. Each channel may be included in a unique channel (GCH) node. The first channel may comprise a director annotation (DC) channel, which may include linear PCM samples. The unique channel metadata (GCM) segment identifies this channel as including a DC channel. The second channel may comprise a visually impaired (VI) channel, which may consist of Code-Excited Linear Prediction (CELP) encoded audio. Again, the unique channel metadata (GCM) segment can identify the channel as containing VI material. The duration of the audio content included in each additional channel is preferably matched to the duration of the audio content in the AC3 node having a certain duration. In addition, metadata identifying the bitstream may be added to the metadata segment (GOFM) of a group of frames.

In particular, in FIG. 8B, the details of the mapped first AC-3 sync frame with supplemental director annotations and visually corrupted audio channels are shown. The bitstream first contains GOF start tag followed by bitstream identification metadata (GOFM), frame start tag (FRM), specific frame data (GFM), AC-3 channel start tag (AC3), and AC-3 specific metadata. (AC3M), AC-3 Content Segments (AB- through AB5 and Aux), AC-3 Channel End Tag (AC3), Unique Channel Start Tag (GCH), Unique Channel Metadata (GCM), Linear PCM Audio Essence Segment ( PCM), unique channel end tag (GCH), unique channel start tag (GCH), unique channel metadata (GCM), CELP-encoded audio essence (CELP), unique channel end tag (GCH), and end of frame tag ( FRM). The second frame (only some are shown) repeats the same column with the second frame information.

An advantage of the format of the present invention is that the insertion of two additional channels does not require modification to AC3 data, which could have been done when the original bitstream was being streamed, i.e., VI in the second frame (not shown). Insertion of the channel is that there is no need to know about the content of the first frame. Also, decoders that cannot interpret VI and / or DC channels can easily ignore these channels. For example, VI and DC channels may be added to an amendment to the specification that specifies the content of the bitstream. Therefore, the bitstream may be compatible with the existing.

9 is a flow diagram or functional block diagram illustrating various functional aspects of an encoder or encoding process for generating a bitstream similar to the example of FIG. 3, in accordance with aspects of the present invention. For example, audio essence stream 91, which may be samples of linear PCM encoded audio, segments the audio into suitable periods (fixed or variable) of several blocks and compresses (eg, bit rate reduction encoding). Additional processing may be applied to the audio segmentation and processing function or device 93 where applicable. The resulting audio data can be wrapped in audio content segments, one of which is schematically illustrated 95. Information about the audio essence may be supplied to the metadata generator 97. The latter may use this information and possibly other information, such as information from the user or from other functions or devices (not shown), for synchronization into the bitstream, which may or may not be synchronized with the audio essence. Create metadata segments.

The audio content segments are sent to the channel node serialization function or device 99 and the one or more audio content segments and one or more associated metadata segments obtained from the metadata generator 97 (in this example downmixing (DM) metadata). Create a channel node (compare with Level 3 of the hierarchy of FIG. 2) along with the channel node start- and end-tags. Example 101 of a channel node is schematically illustrated as including a channel start tag (CHAN), downmix metadata (DM), audio essence segment, and channel end tag (CHAN).

The channel node is supplied to the frame node serializer 103 and includes the frame node metadata (in this example, a segment of timecode (TC) metadata) obtained from the input channel node and the metadata generator 97. (Compared to level 2 of the layer of FIG. 2) with frame node start- and end-tags. An example 105 of a frame node is schematically illustrated as including a frame start-tag (REAM), timecode metadata (TC), channel node sequence, and frame end-tag (FRAM).

The frame node is a group of frames node serializer function or device 107 which is supplied with a group of title (TITL) metadata of consecutive frame nodes and associated segment obtained from metadata generator 97. Combine into a complete bitstream (compare level 1 of the layer of FIG. 2) with the frame start and end tags. An example of a complete bitstream is shown schematically as including a group of frame start tags (GOF), title metadata (TITL), two frame columns, and a group of frame end tags (GOF).

 FIG. 10 is a flow diagram or functional block diagram illustrating various functional aspects of a decoder or decoding process for deriving audio essence and metadata from a bitstream such as the examples of FIGS. 3 and 9, in accordance with aspects of the present invention. .

A bitstream as generated by the example of FIG. 9 is applied to a group of frames node deserializer 121. The gof node deserializer recognizes and removes gof start and end tags and gof metadata (in this example, title (TITL) metadata), sends the metadata to the metadata resolver 123, and sends the frame nodes to the frame node. To the deserializer 125. An example of a frame node 105 that is essentially the same as the frame node 105 of FIG. 9 is shown schematically.

Frame node deserializer 125 recognizes and removes frame node start and end tags and frame metadata (in this example, timecode (TC) metadata), sends the metadata to metadata interpreter 123, The channel nodes are sent to channel node deserializer 127. An example of a channel node 101, which may be essentially the same as the channel node 101, is schematically illustrated in FIG. 9.

Channel node deserializer 127 recognizes and removes channel node start and end tags and channel metadata (in this example downmix (DM) metadata), sends the metadata to metadata interpreter 123, Audio essence segments are sent to an audio rendering process or device 129 that reassembles an audio essence 91 stream that may be essentially the same as the audio essence applied to the encoder or encoding process of FIG. 9.

Metadata interpreter 123 may interpret various metadata and apply it to functions and / or devices (not shown) and to audio rendering 129.

The present invention and its various aspects can be implemented in a variety of ways, such as by software functions performed on digital signal processors, programmed general purpose digital computers, and / or dedicated digital computers. Interfaces between analog and / or digital signal streams may be performed in software and / or firmware with suitable hardware and / or functions. Although the present invention and its various aspects may have analog audio signals as their source, most or all of the processing functions that implement the aspects of the present invention are digital to digital signal streams in which audio signals are represented by samples. Will be performed in the area.

A bitstream formatted according to aspects of the present invention may be stored or transmitted by any known one or more data storage and transmission media.

It will be appreciated that implementations and other aspects of other variations and modifications of the invention will be apparent to those skilled in the art and are not limited to these specific embodiments described. Therefore, it is intended to cover any and all modifications, variations, or equivalents within the spirit and scope of the underlying underlying principles disclosed and claimed herein.

Claims (19)

In a bitstream format for representing audio information in which bitstream syntax is described by sequential traversal of a tree hierarchical data structure, the tree structure is: A plurality of tree hierarchy levels, each having one or more nodes, at least some progressively smaller subdivision of the audio information being represented by progressively lower levels of the tree hierarchy, wherein the audio information is Bitstream format included among one or more of the levels of nodes. 2. The bitstream format of claim 1, wherein the progressively smaller subdivision of the audio comprises one or more temporal subdivisions, spatial subdivisions, and resolution subdivisions. 2. The bitstream format of claim 1 wherein the first level of the tree hierarchy includes a root node representing all audio information and at least one lower level comprises a plurality of nodes representing time intervals of audio information. 4. The bitstream format of claim 3 wherein the at least one lower level comprises a plurality of nodes representing spatial subdivision of audio information. The method of claim 1, wherein the bitstream comprises a series of independent tags and content segments, each tag segment serving as a delimiter, each content segment being associated with audio information or audio information. A payload containing data, wherein said segments are arranged into hierarchically nested nodes that are structurally independent between levels of said tree hierarchy. 6. The bitstream format of claim 5 wherein each node is delimited by start and end tag segments. 7. The bitstream format of claim 6, wherein the start- and end-tag segments delimit header and footer contexts within a node. 8. A node as claimed in any preceding claim, wherein the node comprising one or more content segments carrying audio information comprises one or more content segments carrying metadata related to the audio information in the one or more content segments carrying audio information. Bitstream format. A bitstream formatted according to the bitstream format of claim 1. A system for encoding and decoding a bitstream having a format according to the bitstream format according to claim 1. An encoder for encoding a bitstream having a format according to the bitstream format according to claim 1. A decoder for decoding a bitstream having a format according to the bitstream format according to claim 1. An apparatus for transcoding a bitstream having a format according to the bitstream format according to claim 1. A process of generating a bitstream formatted according to the bitstream format according to claim 1. A process for encoding and decoding a bitstream having a format according to the bitstream format according to claim 1. A process for encoding a bitstream having a format according to the bitstream format according to claim 1. A process for decoding a bitstream having a format according to the bitstream format according to claim 1. A process for transcoding a bitstream having a format according to the bitstream format according to claim 1. A medium for storing or transmitting a bitstream according to claim 9.

Images