RU2780733C2 - Method and device for processing auxiliary media streams embedded in the mpeg-h 3d audio stream - Google Patents

Abstract

FIELD: audio technology.

SUBSTANCE: invention relates to systems for local processing of packaged media streams. The expected result is achieved by the fact that the device contains: a receiver for receiving a bitstream and a separation module for identifying the type of packet in the bitstream and performing separation based on identifying the value of the packet type in the bitstream into the main stream, for example, MPEG-H 3D audio packed in MHAS format, and an auxiliary stream. Media streams presented as packaged streams typically use packet type identifiers to distinguish between several sub-streams in a common media stream. To send additional (media) data that is not related to the main stream of media data, the device can pack additional data into packets formatted according to the main stream of media data, but marked with a special label in the packet header. This special label will force the receiving device to separate packets carrying an additional stream of media data.

EFFECT: increase in the efficiency of decoding/rendering.

13 cl, 6 dwg, 2 tbl

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to the following priority applications: U.S. Provisional Application 62/634136 (Ref: D18027USP1) filed February 22, 2018, U.S. Provisional Application 62/641098 (Ref: D18027USP2) filed March 9, 2018, European Application 18166319.6 (Ref: D18027EP) filed April 9, 2018, and Provisional Application US 62/697536 (Ref: D18027USP3) dated July 13, 2018, which are incorporated herein by reference.

FIELD OF TECHNOLOGY

The present invention relates to the provision of an apparatus, system and method for processing packetized media.

DETAILED DESCRIPTION

For consumer electronics (CE) equipment, functionality may be distributed across multiple devices (eg set-top boxes, televisions, audio/video receivers) where such devices are connected via standardized interfaces (eg HDMI).

The first device may receive media streams via broadcast and/or broadband connections. That first device may additionally have a high level of artificial intelligence (eg, a "smart speaker" feature set). The second fixture may be for decoding, rendering, and displaying the media to users.

Typically, the media program is received by fixture #1 and sent to fixture #2 for playback. This media program may be known as the "main program". From time to time or additionally, another media program (such as an advertisement) received on a different transmission channel, or media generated by means of a "smart device" in device No. 1, with both the program and the data usually presented in a different media format, should be inserted inside or overlaid on top of the main media program.

This can be achieved by decoding both the main and supporting programs into a common, usually uncompressed representation, stream switching, or mixing/rendering both of them into a combined representation and re-encoding the generated media stream into an encoded transmission format. This method can be performed in any fixture, for example, performed entirely in fixture #1. However, significant processing power may be required, while the rendering steps and intermediate representation may not provide an optimal match with the actual playback system in fixture #2.

In FIG. 1, exemplary fixture #1 and exemplary fixture #2 are illustrated. Fixture #1 may include a primary sink 101, a secondary sink/local engine 103, and a packetized stream combiner 102. Packetized stream combining module 102 may combine data streams received from primary sink 101 (eg, main stream) and from secondary sink/local engine 103 (eg, auxiliary data stream). Device No. 2 may include a stream splitter 201, a primary decoding/rendering module 202, and a secondary decoding/rendering module 203. The stream splitter 201 may separate streams such as a main stream (which may then be provided to the primary decoding/rendering unit 202) and an auxiliary stream (which may then be provided to the secondary decoding/rendering unit 203). Both output signals of the decoding module can be presented to the listener at the same time.

To overcome the above limitation, fixture #1 can directly send the incoming mainstream media to fixture #2. This mode of fixture #1 may be referred to as a "pass-through" mode. However, the disadvantage of this approach is that the standardized interfaces are defined to send only one single data stream for video and audio, and thus a second media stream cannot be sent by default through the interface to playback device #2.

The present invention may provide methods and apparatus for a first receiver for receiving packetized media, a secondary receiver for receiving an additional media stream, and a transmitter interface for sending packetized media.

The present invention proposes to combine the second auxiliary media stream with the packetized main media stream as follows.

Media streams represented as packetized streams typically use packet type identifiers to distinguish between multiple substreams in the overall media stream. To send additional (media) data not related to the main media stream, the first device #1 can pack the additional data into packets formatted according to the main media stream, but marked with a special label in the packet header. This special label will cause receiver #2 to separate packets carrying the extra media stream. Appliance #2 may then optionally provide an additional media stream to an instance of the secondary decoding/rendering module 203 while simultaneously receiving the main stream by the primary decoding/rendering module 202.

In one example, legacy receivers that do not recognize this new label in the packet header for additional packed media streams are already expected to ignore those packets.

The label can be provided in any encoded audio stream environment such as MPEG-H, AC-4, Dolby Digital+, etc.

If the additional inserted data streams significantly exceed the data rate of the original media data stream, the sink fixture must filter the incoming packetized stream and optionally separate the additional data packets to maintain the sink buffer model of the downstream (legacy) media decoding module.

Next, in FIG. 3 illustrates exemplary fixture #1 and exemplary fixture #3. Fixture #1 may include a primary sink 101, a secondary sink/local engine 103, and a packetized stream combiner 102. Packetized stream combining module 102 may combine data streams received from primary sink 101 (eg, main stream) and from secondary sink/local engine 103 (eg, auxiliary data stream). Appliance #3 may only contain the primary decoding/rendering module 301, but not the stream splitter. In one example, it may not be possible to provide an auxiliary stream to a secondary decoding/rendering module. In this example, the modified primary decoding module can decode/convert/mix the sub stream with the main stream within itself.

MPEG-H Ecosystem

MPEG-H 3d Audio according to ISO/IEC 23008-3 is packaged in MHAS (MPEG-H Audio Stream) format. This format uses a packetized format in which each packet consists of a packet header and a packet payload. Although the payload can be any binary data, the header specifies the type and length of the payload. (An optional label can be used to distinguish between multiple instances, but it is not used in this case).

By assigning a new MHAS packet type to the secondary media stream (named for example PACTYP_MEDIA), additional audio data represented either as uncompressed PCM (pulse code modulation) data, optionally further defined using the RIFF/WAV format, or compressed audio data such as as MPEG-4 audio according to ISO/IEC 14496-3 or any other encoded representation (eg according to ATSC A/52 or ETSI TS 103 190) can be packed into MHAS packets and hence can be combined with the main MPEG-stream. H 3d audio. Different formats intended for packaging can be distinguished either by different types of packets (eg PACTYP_PCM, PACTYP_MPEG4AUDIO, ...) or, as shown in the example below, by an additional qualifier that forms the subheading of the MHAS packet.

Since (media) data may need configuration data but may not be represented as separate streams, this data may be packed in the header of the containing MHAS packet, or an additional MHAS packet may be assigned (e.g. PACTYP_MEDIA_CONFIG or another type of MHAS packet name , indicating the configuration, for example, PACTYP_PCMCONFIG), which optionally also carries information about the type of additional data. The MHAS packet type may carry configuration information for the PCM payload data in order to provide the configuration information to the decoding module. For example, if the MHAS packet type for configuration information (e.g., PACTYP_MEDIA_CONFIG or PACTYP_PCMCONFIG) is present in the bitstream (e.g., after PACTYP_MEDIA_CONFIG), PCT data configuration information in the form of a data structure (e.g., pcmDataConfig()) may be supplied to the decoding module. .

In general, an MHAS packet type (eg, PACTYP_PCMDATA) can be used to embed PCM payload data corresponding to PCM signals specified in a configuration structure and to supply PCM data in the form of a PCM data payload structure to a decoding module. If the MHAS packet type (eg, PACTYP_PCMDATA) is present in the bitstream, a PCM data payload structure (eg, pcmDataPayload()) may be used during decoding.

In FIG. 2 illustrates an exemplary MPEG-H 3D audio packet flow in accordance with the present invention.

In one example, the present invention may be based on identifying information based on the following syntactic changes.

1. Changesheet 220 - MHASPacketPayload() syntax with the following.

2. Assign an appropriate MHASPacketType to PACTYP_MEDIA in table 223. Alternatively, the value of MHASPacketType may vary or may depend on a predefined value, such as a value published by a standards organization in a final standards document.

3. The next media data type must be defined using the mhasMediaDataType enumeration.

In FIG. 2 exemplarily shows a packetized main stream containing exemplary MPEG-H 3D audio packets with 3D audio data (example containing configuration data such as PACTYP_MPEGH3DACFG as metadata and encoded audio data such as PACTYP_MPEGH3DAFRAME).

In addition, in FIG. 2 exemplarily shows a packetized secondary stream (auxiliary stream) containing exemplary packets according to the present invention, as indicated by header information related to the above-described exemplary additional packet type PACTYP_MEDIA containing additional audio data (auxiliary audio data / secondary audio data), as an example in a compressed format such as MPEG4Audio.

Alternatively or additionally, the additional audio data may be packetized into a packet having a header according to the format of the packetized main stream, in this case as an example MPEG-H 3D audio, which contains a sub-header indicating different formats, packed as discussed above.

In accordance with exemplary aspects of the present invention, the main stream and the auxiliary (secondary) stream may be combined by a stream combining module, including, for example, a packetized stream combining module 102.

The outgoing stream (joint stream) contains packets related to the encoded audio data of the main stream and packets related to the audio data of the sub-stream within a single packetized bitstream of the same format (such as MPEG-H 3D audio in FIG. 2 as an example) .

It should be noted that unmodified (legacy) MPEG-H 3D audio decoders may not understand the newly added packet type (e.g., PACTYP_MEDIA), and such unmodified (legacy) MPEG-H 3D audio decoders may ignore or remove packets from memory. , containing the newly added package type (for example, PACTYP_MEDIA) specified in their header. Such unmodified (legacy) MPEG-H 3D audio decoding modules can still decode audio data related to the main stream, but will not process the additional auxiliary/secondary audio data.

To decode and process the combined stream into main and sub streams, decoding module fixtures can be modified to include a modified decoding module capable of filtering and decoding/processing packets related to audio ancillary data.

In FIG. 4 schematically illustrates another example of a primary decoding/rendering module in accordance with the present invention.

In FIG. 4 illustrates how this can be achieved in an MPEG-H 3D Audio decoder where the media format is PCM data. As an example, the primary decoding/rendering module 301 is implemented as comprising an MPEG-H 3D Audio core decoding module M1 and corresponding renderers such as an object renderer M2, for example, including those defined according to the MPEG-H 3D audio (ISO/IEC 23008-3).

The modified decoding module 301 may further filter and separate MHAS packets having a header indicating a new optional packet type (eg, PACTYP_MEDIA), and input packets containing auxiliary audio data to the format converting block 301c1 and then to the sample rate converting module ( such as, by way of example, the sample rate converting module M3 present in the architecture of the decoding module after the decoding module M1 of the MPEG-H 3D Audio core as specified according to the MPEG-H 3D audio standard (ISO/IEC 23008-3)).

Accordingly, the modified decoding unit 301 can perform sample rate conversion (for example, by the sample rate converting unit M3) and format conversion (for example, by the format converting unit 301c1) on the input media data (MHASPacketType == PACTYP_MEDIA) to match the media sampling rate and structure channels with output sample rate and channel configuration of the decoding module. In addition, the modified decoding unit can mix input media data or input media whose sample rate has been converted with audio media data that has been created by the MPEG-H 3D Audio core decoding unit M1 in a mixer (such as, for example, the mixing unit M4 present in architecture of the decoding module after the decoding module M1 of the MPEG-H 3D Audio core, as specified according to the MPEG-H 3D audio standard (ISO/IEC 23008-3)).

The above example in FIG. 4 refers to ancillary media data containing uncompressed media data such as PCM data or the like by way of example. It should be noted that if the uncompressed media data such as PCM data or the like additionally contains position metadata, e.g. for 3D audio applications, the ancillary data may be further processed by an object renderer that processes the position data, such as an object renderer M2 present in the architecture of the decoder after the MPEG-H 3D Audio core decoder M1, as specified by the MPEG-H 3D audio standard (ISO/IEC 23008-3).

In FIG. 5 schematically illustrates another example of a primary decoding/rendering module in accordance with the present invention. As an example, the primary decoding/rendering module 301 is implemented as comprising an MPEG-H 3D Audio core decoding module M1 and corresponding renderers such as an object renderer M2, for example, including those defined according to the MPEG-H 3D audio (ISO/IEC 23008-3).

In FIG. 5 illustrates by way of example how the above can be achieved in an MPEG-H 3D Audio decoding module where the media format is encoded data (such as MPEG-4 Audio). The modified decoding module 301 can decode the input media data (MHASPacketType == PACTYP_MEDIA) using an additional decoding module 301c2 (a converting module and/or a decoding module adapted to convert and/or decode the auxiliary media data), not necessarily defined in ISO/IEC 23008-3 . In addition, the modified decoding module can mix the decoded media data with the audio media data that was created by the MPEG-H 3D Audio core decoding module M1 as specified according to the MPEG-H 3D audio standard (ISO/IEC 23008-3).

Time Alignment of Multiple MHAS Substreams Originating from Different Sources

In exemplary aspects, in accordance with the present invention, additional time alignment units may be provided for time alignment of substream packets, for example, to provide time alignment of multiple MHAS subpackets originating from different sources.

As stated in section 14.6 of ISO/IEC 23008-3, "MHAS substreams are generated by the same coding unit, [and hence] the different incoming streams are considered to be […] fully aligned and have no phase shift." In this case, data group alignment can be completed using the MHASPacketLabel number. With the proposed method in the present invention, this above limitation can no longer be considered mandatory. With different data group lengths for different codecs or sampling rates, the time offset of successive MHAS packets of a secondary stream that is merged with the main MHAS stream varies over time. In each particular time interval, the timing offset of the secondary stream relative to the main stream must be signaled. For example, in associated sub-stream packets indicating a packet type related to metadata associated with media data contained in the sub-stream packet payload, as shown in FIG. 6.

In FIG. 6 illustrates exemplary packet types to achieve the objectives of the present invention based on MHAS, for example, if another MHAS packet type is specified to carry appropriate time offset signaling information. To associate this signaling information with the corresponding stream, the timeshift packet must have the same assigned MHASPacketLabel number as the MHASPacket of type PACTYP_MEDIA to which it belongs. From FIG. 6, it becomes apparent that a direct one-to-one relationship of stream data corresponding to one particular time slot of each stream is not guaranteed, on the contrary, two or more time slots from one stream may correspond to one time slot from another stream in terms of time.

Another option for signaling a time offset is to add this time offset to the MHAS packet of type PACTYP_MEDIA itself.

In view of the foregoing, in some exemplary aspects in accordance with the present invention, the converting and/or decoding unit of the modified primary decoding/rendering module 301 according to the examples of FIG. 4 and/or FIG. 5 may include a time alignment unit for time alignment of the sub-stream data with the main stream data, for example, before a mixer, such as the mixing unit M4 according to the examples of FIG. 4 and FIG. 5, or also before the sampling unit (for example, the sampling rate converting unit M3) according to the examples of FIG. 4 and FIG. 5, or also as part of a secondary decoding/rendering module 203 according to the example of FIG. one.

Mixing control of the main and secondary audio streams

Additional data may be needed to control the mixing of the secondary (auxiliary) audio stream with the main audio streams. Among other options, this data may contain static gains or dynamic gain sequences, as an example, formed as DynamicRangeControl data according to ISO/IEC 23003-4 for processing the main stream when playing a secondary audio stream. That data is usually generated by fixture #1 and can be included in the stream either by separate MHAS packets (eg, with identifier PACTYP_MPEGH_MEDIA_CFG) as another addition to the secondary stream header, or by any other kind of stream packaging.

In view of the foregoing, in some exemplary aspects in accordance with the present invention, the converting and/or decoding unit of the modified decoding module 301 according to the examples of FIG. 4 and/or FIG. 5 may comprise a gain adjuster for engaging static and/or dynamic gains (eg for volume control), for example before a mixer, such as the mixer M4 according to the examples of FIG. 4 and FIG. 5, or also before the sampling unit (for example, the sampling rate converting unit M3) according to the examples of FIG. 4 and FIG. 5, or also as part of a secondary decoding/rendering module 203 according to the example of FIG. one.

Ecosystem Dolby

Also Dolby AC-4 (ETSI TS 103 190), and Dolby Digital and Dolby Digital Plus (ETSI TS 102 366) offer the ability to carry any binary data in an EMDF payload, which can be used to carry the same or similar data, as specified in the above section (MPEG-H Ecosystem).

For these purposes, the emdf_info() syntax element, as specified in ETSI TS 103 190, or the emdf_container() syntax element, as specified in ETSI TS 102 366 Annex H, and their subject elements can be used. To do this, one or more emdf_payload_id statements can be simply given, which can be used to identify binary data that has a format that is the same or similar to that described above with respect to PACTYP_MEDIA and/or PACTYP_MPEGH_MEDIA_CFG.

System audio mixing for media streams containing uncompressed/unencoded data can be achieved similarly in Dolby AC-4 or Dolby Digital/Dolby Digital Plus as shown in FIG. 4 and FIG. 5, with the difference being that the signaling path to use is distinguished not based on MHASPacketType , but based on the value of emdf_payload_id.

The media streams to which the present invention relates, both the main stream and the side data streams, may be of the following type:

audio streams, both compressed and uncompressed;

video streams;

subtitles.

The present invention can also be applied to video presentation devices (monitors) that will send an overhead picture, video or text in addition to the main (usually compressed) video stream via a standardized interface connection.

Links:

US20170223429A1, EP3149955A1;

ISO/IEC 23008-3: (MPEG-H 3d Audio 2nd edition).

Numbered exemplary embodiments of the present invention refer to the following.

EEE1. A method for processing an audio signal, including:

receiving a combined packetized media bitstream that contains packets associated with a main stream indicating main audio information and packets associated with a side data stream indicating auxiliary audio information, identifying a packet type value in the packet header information of the combined packetized media bitstream, and separating of the combined packetized media bitstream based on identifying a packet type value in the header information of the packets of the combined packetized media bitstream into a main stream indicating main audio information and a side data stream indicating auxiliary audio information.

EEE2. The method according to item EEE1, further comprising:

mixing an audio output signal based on an output signal obtained from the main audio information of the main stream and an output signal obtained from the auxiliary audio information of the side data stream.

EEE3. The method according to EEE2, wherein output signals from the main and auxiliary audio information are output to the listener at the same time.

EEE4. The method according to item EEE1, further comprising:

decoding the main stream by the primary decoding module.

EEE5. The method according to item EEE1, further comprising:

decoding, when the side data stream is related to the compressed auxiliary audio information, of the side data stream by the secondary decoding unit.

EEE6. The method according to item EEE1, further comprising:

converting, when the side data stream refers to the compressed auxiliary audio information, the media data contained in the packets of the side data stream, by the converting unit.

EEE7. A method according to EEE5 or EEE6, wherein

the compressed auxiliary audio information contains MPEG-4 audio data.

EEE8. A method according to EEE1, wherein

the side data stream refers to the uncompressed auxiliary audio information.

EEE9. A method according to EEE8, wherein

uncompressed auxiliary audio information contains PCM data.

EEE10. A method according to EEE1, wherein

the packet type indicates an internal raw format, in particular, an internal raw format with respect to the decoding module.

EEE11. The method according to item EEE1, further comprising:

performing signal processing on the side data stream.

EEE12. A method according to EEE11, wherein

performing signal processing on the side data stream is performed before mixing the audio output signal based on the output signal obtained from the main audio information of the main stream and the output signal obtained from the auxiliary audio information of the side data stream.

EEE13. A method according to EEE11, wherein

performing signal processing on the side data stream includes performing gain adjustment.

EEE14. The method according to paragraph EEE13, in which

the gain adjustment is performed based on the static gain or the dynamic gain.

EEE15. The method according to paragraph EEE13, further comprising:

receiving mixing information related to at least one of a static gain(s) or a dynamic gain(s).

EEE16. The method according to paragraph EEE13, in which

gain adjustment is performed to adjust the volume.

EEE17. A method according to EEE11, wherein

performing signal processing on the side data stream includes performing time alignment.

EEE18. The method according to paragraph EEE17, in which

performing the alignment in time is performed to align the auxiliary audio information of the side data bitstream with the main audio information of the main bitstream in time.

EEE19. The method according to item EEE17 or item EEE18, further comprising:

receiving temporal alignment information relating to the temporal alignment of the auxiliary audio information of the side data bitstream with the main audio information of the main bitstream.

EEE20. A method according to EEE1, wherein

specifying a packet type for carrying time shift signaling information, in particular for time alignment.

EEE21. The method according to item EEE1, further comprising:

receiving an indication of a timing offset of the side data stream relative to the main stream.

EEE22. A method according to EEE11, wherein

performing signal processing on the side data stream includes rendering an object when the auxiliary audio information is associated with the position information.

EEE23. A method according to EEE22, wherein

object rendering is performed by an object rendering module performing object rendering for the main stream and for the side data stream.

EEE24. A method according to EEE11, wherein

performing signal processing on the side data stream includes format conversion.

EEE25. The method according to item EEE1, further comprising:

decoding of the main stream by a standardized MPEG-H 3D audio decoding module.

EEE26. A method according to EEE25, further comprising:

mixing media data contained in side data stream packets with media data generated by a standardized MPEG-H 3D audio decoding module.

EEE27. A method according to EEE26, wherein

the media data contained in the packets of the side data stream is uncompressed data, in particular PCM data, or compressed data, in particular MPEG4 audio data.

EEE28. A method according to EEE26, wherein

the media contained in the packets of the side data stream is decoded using a decoding module not standardized in MPEG-H 3D audio.

EEE29. A method according to EEE1, wherein

the combined packetized media bitstream contains MHAS packets.

EEE30. A method for processing an audio signal, including:

- receiving the main stream indicating the main audio information;

- generating, based on the auxiliary audio information, or receiving a side data stream indicating the auxiliary audio information; and

combining the main stream and the side data stream to generate a combined packetized media bitstream that contains packets associated with the main stream indicating main audio information and packets associated with the side data stream indicating auxiliary audio information.

EEE31. The method according to item EEE30, further comprising

packetizing media data indicating auxiliary audio information to obtain packets of the combined packetized media bitstream format.

EEE32. A method according to EEE30, wherein

the side data stream refers to the compressed auxiliary audio information.

EEE33. A method according to EEE32, wherein

the compressed auxiliary audio information contains MPEG-4 audio data.

EEE34. A method according to EEE30, wherein

the side data stream refers to the uncompressed auxiliary audio information.

EEE35. A method according to EEE34, wherein

uncompressed auxiliary audio information contains PCM data.

EEE36. A method according to EEE30, wherein

the packet header information of the combined packetized media bitstream indicates the packet type.

EEE37. A method according to EEE36, wherein

a packet type value related to packets associated with a side data stream indicating auxiliary audio information indicates media data associated with the auxiliary audio information.

EEE38. A method according to EEE30, wherein

the combined packetized media bitstream contains MHAS packets.

EEE39. An audio signal processing device, comprising:

a receiver adapted to receive a combined packetized media bitstream that contains packets associated with a main stream indicating main audio information and packets associated with a side data stream indicating ancillary audio information; and

a splitter adapted to split the combined packetized media bitstream based on identifying a packet type value in the packet header information of the combined packetized media bitstream into a main stream indicating main audio information and a side data stream indicating auxiliary audio information.

EEE40. An EEE39 device further comprising:

a mixer adapted to mix an audio output signal based on an output signal obtained from the main audio information of the main stream and an output signal obtained from the auxiliary audio information of the side data stream.

EEE41. An EEE39 device further comprising:

a primary decoding module adapted to decode the main stream.

EEE42. An EEE39 device further comprising:

a secondary decoding unit adapted to decode, when the side data stream is related to compressed auxiliary audio information, of the side data stream.

EEE43. An EEE39 device further comprising:

a converting module adapted to convert when the side data stream refers to compressed auxiliary audio information, the media data contained in the packets of the side data stream.

EEE44. A device according to item EEE42 or item EEE43, in which

the compressed auxiliary audio information contains MPEG-4 audio data.

EEE45. A device according to EEE39, in which

the side data stream refers to the uncompressed auxiliary audio information.

EEE46. A device according to EEE45, in which

uncompressed auxiliary audio information contains PCM data.

EEE47. A device according to EEE39, in which

the packet type indicates an internal raw format, in particular, an internal raw format with respect to the decoding module.

EEE48. An EEE39 device further comprising:

a gain adjustment unit adapted to perform gain adjustment with respect to the side data stream.

EEE49. An EEE39 device further comprising:

a time alignment unit adapted to perform time alignment on the side data stream.

EEE50. An EEE39 device further comprising:

an object rendering module adapted to render an object in relation to the side data stream.

EEE51. A device according to EEE50, in which

the object renderer is contained in the primary decoder.

EEE52. An EEE39 device further comprising:

a format converting unit adapted to perform format converting on the side data stream.

EEE53. An EEE39 device further comprising:

standardized MPEG-H 3D audio decoding module for main stream decoding.

EEE54. A device according to EEE39, in which

the combined packetized media bitstream contains MHAS packets.

EEE55. An audio signal processing device, comprising:

a receiver adapted to receive the main stream indicating the main audio information;

a local mechanism adapted to generate, based on the auxiliary audio information, a side data stream indicating the auxiliary audio information, and/or a receiver adapted to receive the side data stream; and

a combining module adapted to combine the main stream and the side data stream to generate a combined packetized media bitstream that contains packets associated with the main stream indicating main audio information and packets associated with the side data stream indicating auxiliary audio information.

EEE56. An EEE55 device further comprising:

a packetization module adapted to packetize the media data indicative of the auxiliary audio information to obtain packets of the combined packetized media bitstream format.

EEE57. A system containing an EEE55 device and an EEE39 device.

Claims (22)

1. A method for processing a combined packetized media bitstream, the method comprising: extracting from the combined packetized bitstream the media data of the main stream, which contains MPEG-H 3D audio data packed in the MHAS format; extraction from the combined packetized media bitstream of an auxiliary stream, which contains additional audio data, packed as packets in the MHAS format, wherein the packets are of a new MHAS packet type, wherein the packet payload format related to the packets is different from the MPEG-H 3D MPEG-H 3D audio data format, and the new MHAS packet type indicates an internal raw format; identifying time offset signal information that indicates the presence of a time offset of the sub stream relative to the main stream, wherein the time offset signal information indicates that the sub stream signals must be offset to arrive at the mixer aligned with the main stream; and temporally aligning the additional audio data with the MPEG-H 3D audio data based on the time offset signaling information. 2. The method of claim 1, further comprising decoding the base stream based on the MPEG-H 3D format. 3. The method according to claim 1, characterized in that the packets contain a subheader identifying the format of the packet's payload. 4. The method of claim 1, further comprising decoding the auxiliary stream based on the packet payload format related to the packets. 5. The method according to claim. 1, characterized in that it further includes issuing at least an output signal obtained from the main stream and the auxiliary stream. 6. The method according to claim 1, characterized in that the MHAS format is compatible with the ISO/IEC 23008-3 standard. 7. A non-volatile computer-readable storage medium with instructions stored on it, which, when executed by one or more processors, ensure that one or more processors execute the method according to claim 1. 8. An audio decoding device for processing a combined packetized bitstream of media data, the device comprising: a splitter adapted to extract media data from the combined packetized bitstream of the main stream, the main stream containing MPEG-H 3D audio data packed in MHAS format, and the auxiliary stream, the auxiliary stream containing additional audio data packed in the form of packets in .mhas format, wherein the packets are of a new MHAS packet type, wherein the packet payload format related to the packets is different from the MPEG-H 3D MPEG-H 3D audio data format, and the new MHAS packet type indicates an internal raw format; an identification unit adapted to identify the time offset signal information that indicates the presence of a time offset of the auxiliary stream relative to the main stream, while the time offset signal information indicates that the signals of the auxiliary stream must be offset to arrive at the mixer aligned with the main stream; and a time alignment unit for temporally aligning the additional audio data with the MPEG-H 3D audio data based on the time offset signaling information. 9. The apparatus of claim 8, further comprising an audio decoder for decoding the main stream based on the MPEG-H 3D format. 10. The device according to claim 8, characterized in that the packets contain a subheader identifying the payload format of the packet. 11. The apparatus of claim 8, further comprising an audio decoder for decoding the sub-stream based on the packet payload format related to the packets. 12. The device according to claim. 8, characterized in that it further comprises an output unit for issuing at least an output signal obtained from the main stream and the auxiliary stream. 13. The device according to claim 8, characterized in that the MHAS format is compatible with the ISO/IEC 23008-3 standard.

Images