DE102017212340B3 - Method for transmitting an audio stream - Google Patents

Abstract

The invention relates to a method for transmitting an audio stream from a transmitter (2) to at least one receiver, in which the audio stream from the transmitter (2) is divided into audio samples (ai / j), each audio sample (ai / j) being a pixel (vi / j) of a video stream (24), wherein the video stream (24) is transmitted from the transmitter (2) to the at least one receiver, wherein the pixels (vi / j) of the video stream (24) from the at least one Receivers are decoded as audio samples (ai / j) of the audio stream.

Description

The invention relates to a method for transmitting an audio stream and a system for transmitting an audio stream.

For a synchronous transmission of audio data in a motor vehicle nowadays powerful bus systems, for example MOST (Media Oriented Systems Transport) with a transmission rate of up to 150 Mbps or A2B (Automotive Audio to Bus) with a transmission rate of up to 50 Mbps are used. Such bus systems, and thus networks, include Network Interface Controllers (NICs) interfaced on one side to a host (HOST), such as a microcontroller, DSP (Digital Signal Processor), or SoC (System on a Chip) Chip system), synchronous audio interfaces, for example, as I2S (Inter IC sound), S / PDIF or TDM (Time Division Multiplexer) are provided.

However, a bandwidth of such audio interfaces is limited and is, for example, 64 bytes, which corresponds to 32 channels of 16 bits each. In order to avoid an associated restriction of a bandwidth for transmitting the audio data, further, proprietary interfaces, for example Media-LB, have been defined. Furthermore, it is possible to use several TDM interfaces. Furthermore, it is possible to realize a USB interface exclusively for providing a point-to-point connection.

With a connection of such Network Interface Controller (NIC) via the above-mentioned interfaces to current graphics SoCs of a mobile computer platform, for example nVidia Tegra, Samsung Exynos or Qualcom Snapdragon, restrictions arise for the interfaces, if, for example, no Media-LB is present is. Furthermore, it is possible that a number of the interfaces are too small if a large amount of audio data with, for example, 150 Mbps to be fed into an audio network, for example, to provide a plurality of audio zones in the audio network with audio files.

A technical installation for transmitting multimedia content, such as movies or music, for a vehicle is from the publication CH 709 209 A2 known.

A method of transmitting data from a video system to an audio system is disclosed in the document US 2010/0135641 A1 known.

The publication US 2016/0080685 A1 describes a system and method for outputting multilingual audio data.

The publication DE 696 09 151 T2 describes a method for transmitting a communication signal, wherein a video signal is generated and an audio signal is sampled. As the communication signal, the video signal is transmitted with a video image portion and audio samples included to replace a portion of the video signal. In addition, the sampled audio signal is encoded using ASK encoding.

The publication US 2002/0008778 A1 describes a method for transmitting audio signals over a video connection.

Against this background, it was an object to increase a bandwidth for transmitting audio data and / or to enable an additional interface.

This object is achieved by a method and a system having the features of the independent patent claims. Embodiments of the method and the system are evident from the dependent claims and the description.

The method according to the invention is intended for transmitting an audio stream comprising audio data from a transmitter to at least one receiver, wherein the audio stream or the audio data of the audio stream are divided by the transmitter into audio samples, each audio sample remaining as a pixel of a video stream, eg. is encoded and / or inserted as pixels of at least one frame of the video stream, the video stream being transmitted with the audio samples encoded as pixels, from the transmitter to the at least one receiver. The pixels of the video stream are respectively decoded by the at least one receiver back into respective audio samples of the audio stream and / or removed as such. Thus, the audio data of the audio stream for transferring the audio data into video data of the video stream is transformed and then transformed back again.

The method is performed for an audio stream having an audio sampling rate and a value f, the video stream having a frame rate with a value g. Typically, the frame rate corresponds to an integer divisor of the audio sample rate, i. H. s = f / g, where the values f, s and g are integers.

In an embodiment, an audio interval comprising at least part of the audio data of the audio stream is divided into a plurality of audio samples which are assigned to the at least one frame of the video stream, wherein each audio sample is assigned to a pixel of the at least one frame.

In this case, the s audio samples are assigned in design to a field or a matrix of pixels, wherein the field has a plurality of rows and columns, wherein, for example, each line is assigned to an audio channel and each column is assigned to a sampling instant. In an embodiment, the field or the matrix of a frame has a number of s * z audio samples which are encoded as s * z pixels in the s columns and z lines of the frame. It is provided, for example, that an audio system, for example, comprises a plurality of loudspeakers, wherein each loudspeaker is assigned an audio channel. Conversely, it is also possible to associate each line with a sample time and each column with an audio channel.

As an alternative or in addition, it should be taken into consideration that a defined number of audio samples can be played by an audio device within a defined time interval, the audio samples being arranged relative to one another in accordance with a chronological sequence. It is conceivable that the audio samples within the audio stream and the same audio samples encoded as pixels within the video stream are synchronized relative to each other. so that, for example, an audio sample within the audio stream at a time tx and a resulting and / or coded pixel within the video stream also appears at the time tx.

The audio samples of the audio stream are encoded as pixels of a video stream formed as a video stream, with the pixels arranged in multiple rows and columns. The audio samples of the audio stream are stored as pixels within the video stream in a different structure and sorted, for example, taking into account the rows and columns.

The audio samples of the audio stream are first transformed into the pixels of the video stream, transmitted with the video stream, and then transformed back from the pixels of the video stream to the audio samples of the audio stream.

The audio samples of the audio stream are transmitted as a video stream and / or as a video file via at least one video interface, via which the transmitter and the at least one receiver are connected to each other, wherein in each case a video interface is formed as a component of the transmitter or the receiver.

Each pixel of the video stream has a bit length with a value b. Each audio sample has a bit length which has a maximum value of b, for example 24 bits, whereby in the event that the bit length of the audio samples is smaller than the bit length of the pixel, for example 16 bits, the pixel is left-aligned or " leftaligned "(" left-justified ") or alternatively right-justified with zeros.

In an embodiment, the audio stream is transmitted in addition to or in addition to the video stream, so that the audio stream and the video stream are sent together by the sender, transmitted together and received jointly by the receiver. The audio stream and the video stream are disconnected by the receiver after a reception. A common transmission of the audio stream and the video stream can be realized in that so-called blanking in the horizontal lines and the vertical columns makes free space or spaces available for pixels. The audio stream is inserted into the blanks as part of the video stream. This blanking is used to control a display or display field.

The system according to the invention is designed to transmit an audio stream and / or an audio file from a transmitter to at least one receiver. The transmitter is configured to split the audio stream into audio samples, to encode each audio sample as pixels of a video stream, and to transmit the video stream to the at least one receiver. The at least one receiver is configured to decode the pixels of the video stream again as audio samples of the audio stream. The transmitter and the at least one receiver are formed as components of the system.

The transmitter has an audio-video multiplexer and at least one memory configured as a video memory, wherein the audio-video multiplexer is configured to distribute the audio samples to the pixels and write to the video memory. The at least one receiver has a video-audio demultiplexer, which is designed to re-transmit the audio samples from the pixels according to an intended time sequence in the audio stream.

The system also includes at least one video interface configured to transmit the audio samples encoded in the video stream from the transmitter to the at least one receiver.

Thus, the method of transmitting audio data of the audio stream as audio samples encoded as pixels of the video stream is provided within the video stream via the at least one video interface or interface. In this case, it is possible for the audio data of the audio stream to be embodied as PCM audio data or audio samples which are pulse-code modulated (PCM). The audio samples of the audio data are encoded as pixels of the at least one frame of the video stream.

It is provided that the pixels are encoded, divided and / or sorted within a frame into a plurality of rows and columns. Depending on the audio sampling rate / sample rate and video sampling rate / frame rate, a frame will contain at least a defined number of audio samples, namely at least the number resulting from the ratio of the audio sampling rate to the frame rate. The audio samples can then be assigned to any respective pixels of a frame or encoded as such. In this case, each line corresponds to an audio channel and each column corresponds to a sampling time of an audio processing and thus a processing of the audio samples of the audio stream taking into account the sampling rate. In addition, each pixel within the field emerges from an audio sample.

In this case, it is possible, for example, to transmit the audio samples from a graphics SoC (system on a chip) to a NIC (Network Interface Controller) using the at least one video interface, for example HDMI, eDP or DSI. It is provided that the audio samples of the audio stream in the one-chip system (SoC) supplied by at least one decoder and, for example, preprocessed by performing a sound processing (sound processing) with a sampling rate of 48 kHz, for example.

The sender of the system's audio-video multiplexer writes the audio samples or audio data to at least one video memory or to at least one common memory of a display controller of the single-chip system. Such writing is carried out, for example, at a frame rate which corresponds to an integer divisor of the sampling rate of 48,000 Hz, for example, formed as the audio sampling rate, the frame rate corresponding to 60 Hz in this case. The ratio of the audio sampling rate and the frame rate results in a length of the lines of the frame or frame of the video stream, in which case 800 pixels are arranged chronologically in a line according to the sampling times.

Furthermore, each pixel includes 24 bits (RGB-8-8-8) and describes an audio sample or resolution of an audio sample of 24 bits. For an audio sample of less than 24 bits, for example 16 bits, this audio sample will be left justified within the 24 bits of the pixel and padded at the bottom with zeros. Alternatively, it is conceivable to display the sample right-justified and to fill in the upper area with zeros. Thus, if an audio sample contains a smaller amount of data than can be stored within the pixel, any excess space for storing the audio sample within the pixel is either left-aligned or right-aligned with zeros.

Each audio channel containing audio samples up to 24 bits corresponds to one line. For example, it is possible for a first audio decoder to provide a 2-channel or stereo audio stream, whereas a second audio decoder supplies a 7.1- and thus a multi-channel audio stream. In addition, it is possible for the at least one frame or frame of the video stream to comprise ten lines of 800 pixels each, the first two lines containing, for example, a video of the first audio decoder and the last eight lines of audio data or audio samples of a second audio decoder.

The audio-video multiplexer of the transmitter uses in design a double buffer memory (double buffer), whereby it is possible to work independently of a synchronization module (Sync) of the display control device. Furthermore, it is provided that the audio-video multiplexer is designed as a computer core, for example as a CPU core or GPU core. Conversely, the audio samples received in the video stream and thus audio data can be read out via a capture controller of the at least one receiver and supplied to corresponding audio sinks in a reverse transformation.

By an embodiment of the method results over the prior art u. a., That a bandwidth for transmitting audio data, since they are now transmitted within the video stream via the at least one video interface, is increased. The method is suitable, for example, for distributing a large amount of audio data of an audio stream from a server as a sender to many users or clients as recipients within a network, the method providing a broadband connection in the server.

It is possible, in one embodiment of the method, to use a plurality of video or display interfaces on conventional one-chip systems for providing a graphic. The audio samples are transmitted synchronously with the video stream, using, for example, to transmit the video stream sequences of 30 Hz or 60 Hz, each of an integer natural divisor of a conventional sampling rate, d. H. Audio sampling rate or sample rates, for example 48 kHz, 96 kHz or 192 kHz.

With the method, it is also possible, an audio synchronization of the server, for example. The one-chip system, as a sender, and the client, for example. SoC or NIC, as a receiver via a timer or a clock of a video - respectively. Synchronize display interface. The audio samples are sent with extra added, invisible lines that contain the audio samples, with a signal for the video stream.

The method is, for example, for transmitting audio data of the audio stream within a vehicle, between vehicles and / or between an example. Fixed device and a vehicle possible. At least one component of the system, d. H. the transmitter, the at least one receiver and / or the video interface is designed as a component of at least one vehicle. Thus, the audio data within the video stream is broadcastable by a transmitter of a vehicle and / or receivable by a receiver of a vehicle.

Further advantages and embodiments of the invention will become apparent from the description and the accompanying drawings.

It is understood that the features mentioned above and those yet to be explained below can be used not only in the particular combination indicated, but also in other combinations or in isolation, without departing from the scope of the present invention.

• 1 zeigt in schematischer Darstellung Details einer ersten Ausführungsform des erfindungsgemäßen Systems zur Durchführung einer ersten Ausführungsform des erfindungsgemäßen Verfahrens.
• 2 zeigt in schematischer Darstellung eine zweite Ausführungsform des erfindungsgemäßen Systems zur Durchführung einer zweiten Ausführungsform des erfindungsgemäßen Verfahrens.
• 3 zeigt in schematischer Darstellung eine dritte Ausführungsform des erfindungsgemäßen Systems zur Durchführung einer dritten Ausführungsform des erfindungsgemäßen Verfahrens.
• 4 zeigt in schematischer Darstellung eine vierte Ausführungsform des erfindungsgemäßen Systems zur Durchführung einer vierten Ausführungsform des erfindungsgemäßen Verfahrens.

The invention is schematically illustrated by means of embodiments in the drawings and will be described schematically and in detail with reference to the drawings.

• 1 shows a schematic representation of details of a first embodiment of the system according to the invention for carrying out a first embodiment of the method according to the invention.
• 2 shows a schematic representation of a second embodiment of the system according to the invention for carrying out a second embodiment of the method according to the invention.
• 3 shows a schematic representation of a third embodiment of the system according to the invention for carrying out a third embodiment of the method according to the invention.
• 4 shows a schematic representation of a fourth embodiment of the system according to the invention for carrying out a fourth embodiment of the method according to the invention.

The figures are described coherently and comprehensively. Same components are assigned the same reference numerals.

1a shows a schematic representation of an example of a transmitter 2 , which is formed as a component of the first embodiment of the system according to the invention. 1b shows a schematic representation of a diagram for carrying out the first embodiment of the method according to the invention by the transmitter 2 , 1c shows an example of a receiver 4 as a component of the first embodiment of the system according to the invention. 1d shows a diagram for carrying out the first embodiment of the method according to the invention by the receiver 4 ,

The in 1a schematically illustrated transmitter 2 of the system comprises a decoding module 6 with several audio decoders 8-1 . 8-2 , ..., 8-z , an audio-video multiplexer 10 , several stores 12-1 . 12-2 , a display control device 14 and a video interface 16 , which is designed here as a display interface.

In the embodiment of the method, the transmitter 2 an audio stream 18 decodes, here several audio files 20-1 . 20-2 , ..., 20-z having. This will be any audio file 20-1 . 20-2 , ..., 20-z with a timer 22 (please refer 1b ) or clock of the transmitter 2 taking into account a sampling rate f from here 48 kHz from one audio decoder each 8-1 . 8-2 , ..., 8-z divided into audio samples ai / j, wherein a first run parameter i = 1, 2, ... z each an audio channel 21-1 . 21-2 . 21-z ( 1b ), one audio file each 20-1 . 20-2 . 20-z in such an audio channel 21-1 . 21-2 , ..., 21-z ( 1b ) is transformed. The first run parameter i gives a number of a respective audio file 20-1 . 20-2 , ..., 20-z and a respective audio channel 21-1 . 21-2 , ..., 21-z at.

A second run parameter j = 1, 2, 3, 4, ... 800 gives a number of a respective audio sample ai / j in a respective audio channel 21-1 . 21-2 , ..., 21-z ( 1b ) at. In this case, a respective audio sample ai / j has a time interval of 1 / (48 kHz). These audio samples ai / j are still from the audio-video multiplexer 10 in video pixels vi / j of the video stream 24 transfer. This video stream 24 here comprises a total of z * 800 video pixels vi / j, the first run parameter i = 1, 2, ... z and the second run parameter j = 1, 2, 3, 4, ... 800. In each case an audio sample ai / j of an audio channel 21-1 . 21-2 , ..., 21-z into a corresponding pixel vi / j of the video stream 24 transformed. It is provided here that the audio samples ai / j of an ith audio channel 21-1 . 21-2 , ..., 21-z as the pixel vi / j of an i-th line of the video stream 24 be transmitted. It is here provided that each line of the video stream 24 has a total of 800 pixels vi / j. Within a respective line, a video pixel vi / j is assigned to a jth sample time of here a total of 800 sample times. Thus, the first run parameter i further gives a number of the respective line and the second run parameter j gives a number of a respective column as well as one respective sampling time. It includes the in 1a illustrated video stream 24 a frame with 800 * z pixels vi / j, where each audio sample ai / j is encoded as pixel vi / j. As part of the process, it is possible to use multiple video streams 24 and thus to transmit several frames in succession.

Alternatively, it is also possible for the video stream to have z columns and 800 rows, so that lines and columns are reversed within the video stream as compared to the described embodiment. In this case, each line is a sampling time and each column is assigned an audio channel.

The video stream 24 with the pixels vi / j resulting from the audio samples ai / j will continue from the memories 12-1 . 12-2 the display control device 14 provided and via the video interface 16 to one in 1c shown further video interface 25 Recipient 4 transmit, with this additional video interface 25 trained here as a camera interface and in 1c is shown schematically. As further components, the receiver comprises a detection control device 26 , Storage 28-1 . 28-2 , a video-audio demultiplexer 30 and several audio taps 32-1 . 32-2 , ..., 32-z , Here is the video-audio demultiplexer 32 adapted to the pixels vi / j of the video stream 24 be transferred back into the audio samples ai / j or rückzufransformieren from which they from the transmitter 2 were originally transformed, in which case a timer 34 is used, the same sampling rate f of 48 kHz as the timer 22 the transmitter 2 having. In this case, the pixels vi / j of an ith row again become audio samples ai / j of an ith audio sink 32-1 . 32-2 , ..., 32-z and thus again in an ith audio channel 21-1 . 21-2 , ..., 21-z the original audio stream 18 transfer.

By the measure, the audio samples ai / j of the audio channels 21-1 . 21-2 , ..., 21-z into the pixels vi / j of the video stream 24 becomes a bandwidth for transmitting the audio samples ai / j of the audio stream 18 that are now inside the video stream 24 via video interfaces 16 . 25 between the transmitter 2 and the receiver 4 are transferable, increased.

In the 2 schematically illustrated second embodiment of the system according to the invention 40 also includes a transmitter 42 and a receiver 44 , This includes the transmitter 42 a one-chip system 46 and a video interface formed as a display interface 48 , The recipient 44 also includes a one-chip system 50 and a video interface designed as a camera interface 52 , The two video interfaces 48 . 52 the transmitter 42 and the recipient 44 are here via a level converter 54 (glue-logic), which is designed to set a compatibility of signals from the video interface 48 the transmitter 42 to the video interface 52 Recipient 44 be transmitted.

In one implementation of the second embodiment of the method, audio data of an audio stream is divided into audio samples and from an audio-video multiplexer of the one-chip system 46 the transmitter 42 in pixels of a video stream and within the video stream from the video interface 48 over the level converter 54 to the video interface 52 Recipient 44 transfer. This shows the one-chip system 50 Recipient 44 a video-audio demultiplexer adapted to re-transmit the pixels of the video stream into audio samples. Furthermore, the one-chip system 50 designed to re-associate these audio samples with audio channels, and thus audio files, to form the original audio stream again.

The third, in 3 schematically illustrated embodiment of the system according to the invention 60 also includes a transmitter 62 and a receiver 64 , This includes the transmitter 62 also a one-chip system 66 with an audio-video multiplexer 68 and a video interface formed as a display interface 68 , Furthermore, the transmitter includes 62 a network interface controller 70 with a video interface 72 at an entrance and a physical video interface 74 at an exit.

The recipient 64 also includes a network control device 76 with a physical video interface 78 at an entrance and a video interface 80 at an exit. In addition, the receiver includes 64 a one-chip system 82 comprising a video-audio demultiplexer and a video interface 84 ,

The fourth, in 4 schematically illustrated embodiment of the system according to the invention 160 also includes a transmitter 162 and several recipients 164 . 264 . 364 , This includes the transmitter 162 also a one-chip system 166 with an audio-video multiplexer 168 and a video interface formed as a display interface 168 , Furthermore, the transmitter includes 162 a network interface controller 170 with a video interface 172 at an entrance and a physical video interface 174 at an exit.

Every receiver 164 . 264 . 364 also includes a network control device 176 with a physical video interface 178 . 278 . 378 at an entrance and a video interface 180 . 280 . 380 at an exit. In addition, each recipient includes 164 . 264 . 364 a one-chip system 182 . 282 . 382 containing a video-audio demultiplexer and a video interface 184 . 284 . 384 having.

Both the third and fourth embodiments of the system 60 . 160 is for transmitting an audio stream from a transmitter 62 . 162 to at least one recipient 64 . 164 . 264 . 364 trained, the transmitter 60 . 160 configured to split the audio stream into audio samples, to encode each audio sample as pixels of a video stream, and to distribute the video stream to the at least one receiver 164 . 264 . 364 transferred to. The at least one recipient 164 . 264 . 364 is configured to decode the pixels of the video stream again as audio samples of the audio stream while providing the original audio stream.

In all the embodiments of the method described above, a temporal correlation of the audio samples of the audio stream is transmitted to the pixels of the video stream and is thus not lost.

Claims (6)

A method for transmitting an audio stream (18) from a transmitter (2, 42, 62) to at least one receiver (4, 44, 64, 164, 264, 364), wherein the audio stream (18) from the transmitter (2, 42 , 62) is divided into audio samples (ai / j), each audio sample (ai / j) being encoded as a pixel (vi / j) of a video stream (24), an audio interval of the audio stream (18) being included in audio samples (ai / j ) associated with at least one frame of the video stream (24), each audio sample (ai / j) being associated with a pixel (vi / j) of the at least one frame, the audio samples (ai / j) being a field of pixels (vi / j), where the field has a plurality of rows and columns, each row being associated with an audio channel (20-i) and each column being sampled, the video stream (24) being received by the transmitter (2, 42, 62 ) is transmitted to the at least one receiver (4, 44, 64, 164, 264, 364), wherein the pixels (vi / j) of the video stream (24) from the at least one receiver ger (4, 44, 64, 164, 264, 364) are decoded as audio samples (ai / j) of the audio stream (18). Method according to Claim 1 performed for an audio stream (18) having an audio sample rate with a value f, the video stream (24) having a frame rate with a value g, where s = f / g, where s, f and g are integer natural numbers , Method according to Claim 1 or 2 in which each pixel (vi / j) has a bit length with a value b, each audio sample (ai / j) having a maximum bit length b, in which case the bit length of the audio sample (ai / j) is smaller than the bit length of the pixel (vi / j), the pixel is left-aligned with zeros. The method of any one of the preceding claims, wherein the audio stream (18) is transmitted in addition to a video stream (24), the audio stream (18) and the video stream (24) being received by the receiver (4, 44, 64, 164, 264, 364). A system for transmitting an audio stream (18) from a transmitter (2, 42, 62) to at least one receiver (4, 44, 64, 164, 264, 364) in which the transmitter (2, 42, 62) is adapted thereto splitting the audio stream (18) into audio samples (ai / j) to encode each audio sample (ai / j) as a pixel (vi / j) of a video stream (24), wherein an audio interval of the audio stream (18) in audio samples (ai / j) associated with at least one frame of the video stream (24), each audio sample (ai / j) being associated with a pixel (vi / j) of the at least one frame, the audio samples (ai / j) being one field Pixels (vi / j), the array having a plurality of rows and columns, each row being associated with an audio channel (20-i) and each column being assigned a sample time, the transmitter (2, 42, 62) being adapted to to transmit the video stream (24) to the at least one receiver (4, 44, 64, 164, 264, 364), the at least one receiver (4, 44, 64, 164, 264) 4, 364) is adapted to decode the pixels (vi / j) of the video stream (24) as audio samples (ai / j) of the audio stream (18). System after Claim 5 in which the transmitter (2, 42, 62) comprises an audio-video multiplexer (10) and at least one memory (12-1, 12-2), the audio-video multiplexer (10) being designed to associate each audio sample (ai / j) with a pixel (vi / j) and write it to the video memory (12-1, 12-2) and the at least one receiver (4, 44, 64, 164, 264, 364 ) comprises a video-audio demultiplexer (30) adapted to transmit the audio samples (ai / j) from the pixels (vi / j) into the audio stream (18).

Images