KR20230091661A - Android automotive audio system and method for stream conversion using the same - Google Patents

Abstract

Provided is a method for stream conversion, which is performed by an Android automotive system operated by at least one processor. The method of the present invention comprises the steps of: initializing a multichannel plug-in module and allocating a plurality of stereo stream buffers and one multichannel buffer; when receiving a plurality of stereo audio streams transmitted by a plurality of media applications from an audio HAL, storing audio data for each stereo audio stream for each media application in the plurality of the stereo stream buffers; converting the audio data stored in the plurality of the stereo stream buffers into a multichannel stream; and playing the converted multichannel stream to output the same to an audio device. Accordingly, hardware resources can be saved.

Description

Android automotive audio system and method for stream conversion using the same}

The present invention relates to an Android automotive audio system that converts a plurality of stereo streams into a single multi-channel stream in real time in the Android automotive audio system and a stream conversion method using the same.

ALSA (Audio Linux Sound Architecture) library provides an interface capable of mixing sound data from various application programs, so that an improved audio function can be given to the existing Linux system structure.

Meanwhile, the Android Automotive OS uses the tinyALSA library, not the ALSA library, for audio reproduction/recording. tinyALSA is a small library for interfacing with the ALSA library in the Linux kernel used in the Android OS, and corresponds to a lightweight version of the ALSA library.

The ALSA library provides a plug-in that can convert multiple stereo streams into a single multi-channel stream, but the tinyALSA library does not provide a plug-in that converts multiple stereo streams into a single multi-channel stream. Therefore, the Android automotive audio system cannot convert multiple stereo streams into a single multi-channel stream.

Accordingly, the present invention provides an Android automotive audio system supporting a multi-channel plug-in that converts multiple channels into one stream according to hardware requirements and a stream conversion method using the same.

As a method for converting a stream by an Android automotive system operated by at least one processor, which is one feature of the present invention for achieving the technical problem of the present invention,

Allocating a plurality of stereo stream buffers and one multi-channel buffer by initializing the multi-channel plug-in module. When a plurality of stereo audio streams transmitted from a plurality of media applications are received from the audio HAL, each stereo audio stream is received for each media application. Storing audio data for a stream in the plurality of stereo stream buffers, converting the audio data stored in the plurality of stereo stream buffers into a multi-channel stream by arranging the audio data stored in the plurality of stereo stream buffers in frame order according to the stream order, and converting the audio data into a multi-channel stream. It includes the step of reproducing one multi-channel stream and outputting it to an audio device.

After the step of outputting to the audio device, checking the buffer occupancy rate of each stereo stream buffer, and determining a data write cycle in which the audio HAL writes the audio data to each stereo stream buffer based on the checked buffer occupancy rate. It may include setting steps.

In the setting of the data write cycle, if the buffer occupancy rate corresponds to the first interval, the data write cycle may be set to a preset value.

In the setting of the data write cycle, when the buffer occupancy rate corresponds to the second interval, the data write cycle may be set to a value proportional to the buffer occupancy rate.

In the setting of the data write cycle, if the buffer occupancy rate corresponds to the third interval, the data write cycle may be set based on a frame size and a constant value for preventing overflow.

As an Android automotive system, which is another feature of the present invention for achieving the technical problem of the present invention,

An audio HAL (Hardware Abstraction Layer) that stores a plurality of stereo audio streams transmitted from a plurality of media applications, and converts the plurality of stereo audio streams transmitted from the audio HAL into a multi-channel stream, and converts the multi-channel stream. It includes a multi-channel plug-in module that reproduces based on the library and outputs it to an audio device.

The multi-channel plug-in module includes: a plurality of stereo stream buffers for storing audio data for a plurality of stereo audio streams transmitted from the audio HAL for each media application; It may include a channel converter, a multi-channel buffer for storing the multi-channel stream converted by the multi-channel converter, and a playback module for reproducing the multi-channel stream stored in the multi-channel buffer based on the library.

The library may be a tinyALSA library.

The multi-channel plug-in module may further include a management module preventing underflow or overflow of the plurality of stereo stream buffers.

The management module may determine a buffer occupancy rate of each stereo stream buffer, and set a data write cycle in which the audio HAL writes the audio data to each stereo stream buffer based on the buffer occupancy rate.

The management module may prevent the underflow by setting the data write cycle to a preset value when the buffer occupancy rate corresponds to the first interval.

When the buffer occupancy rate corresponds to the second interval, the management module may set the data writing period to a value proportional to the buffer occupancy rate.

When the buffer occupancy rate corresponds to the third interval, the overflow may be prevented by setting the data write cycle based on the frame size and a predetermined constant value.

According to the present invention, when operating as a single channel stream, devices must be allocated as many as the number of streams, but when operating as a multi-channel stream, hardware resources can be saved because it is allocated to one device.

In addition, multi-channel plug-ins that were not supported in the Android environment can be supported.

In addition, it can support dynamic HAL (Hardware Abstraction Layer) suitable for the Android automotive audio environment.

1 is an exemplary view comparing a general Android automotive system and an Android automotive system according to an embodiment of the present invention.
2 is an exemplary diagram of an Android automotive system according to an embodiment of the present invention.
3 is a structural diagram of a multi-channel plug-in module according to an embodiment of the present invention.
4 is an exemplary view of a management module according to an embodiment of the present invention.
5 is a flowchart of a method for converting a stream according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily carry out the present invention. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a certain component is said to "include", it means that it may further include other components without excluding other components unless otherwise stated.

Hereinafter, a vehicle multimedia system according to an embodiment of the present invention and a method for controlling the level of a streaming buffer using the same will be described with reference to the drawings. Prior to describing an embodiment of the present invention, a general method for processing audio data will be first described with reference to FIG. 1 .

1 is an exemplary view comparing a general Android automotive system and an Android automotive system according to an embodiment of the present invention.

Figure 1 (a) is an exemplary diagram showing a general Android automotive system, Figure 1 (b) is an exemplary diagram showing an Android automotive system according to an embodiment of the present invention.

First, as shown in (a) of FIG. 1, in a general Android automotive system, when a plurality of media applications 10 play audio, the Android automotive OS 20 receives a stereo audio stream and connects the IVI (In- Vehicle Infotainment). At this time, the conventional Android automotive system requires as many audio output devices as the number of media applications 10 .

Here, the conventional Android automotive OS 20 uses the tinyALSA library to reproduce or record audio reproduced in the media applications 10 . The tinyALSA library for interfacing with the ALSA library in the Linux kernel used in the Android-type OS is a lightweight version of the ALSA library used for basic audio playback, audio recording, and audio mixing. The tinyALSA library provides fewer APIs and plug-ins than the ALSA library, and cannot convert multiple stereo streams into a single multi-channel stream.

Therefore, in the embodiment of the present invention, as shown in (b) of FIG. 1, the Android automotive system 100 is implemented in the form of inserting a multi-channel plug-in into the Android automotive OS. Through this, even if audio is executed in a plurality of media applications 200, the multi-channel plug-in converts a plurality of stereo streams into one multi-channel stream. Also, audio may be output through one audio output device (not shown).

The Android automotive system 100 will be described in detail with reference to FIG. 2 .

2 is an exemplary diagram of an Android automotive system according to an embodiment of the present invention.

As shown in FIG. 2, the Android automotive system 100 is driven by the Android automotive OS composed of an audio framework 110, an audio HAL 120, a multi-channel plug-in module 130, and a library 140 that is tinyALSA. do.

The audio framework 110 is basically installed in the Android automotive system 100 and decodes encoded audio data received from each media application 200 and converts the encoded audio data into digital audio data, i.e., stereo audio data. That is, the audio framework 110 decodes encoded audio data such as MP3, converts it into digital PCM (Pulse Code Modulation) data, and transfers the converted digital PCM data, stereo audio data, to the audio HAL 120. .

The audio framework 110 processes data such as resampling, bit converting, and mixing according to a predetermined policy on the stereo audio data. Since the audio framework 110 decodes the encoded data and converts it into digital stereo audio data, or processes the converted stereo audio data, various methods are known, so the embodiment of the present invention is limited to one method. I never do that.

The audio HAL 120 provides an interface for the media application 200 or other software programs to interact with hardware devices (eg, speakers, etc.). In addition, the audio HAL 120 stores stereo audio data transmitted from each media application 200 in each stereo stream buffer of the multi-channel plug-in module 130.

The multi-channel plug-in module 130 is composed of a plurality of stereo stream buffers. The multi-channel plug-in module 130 stores stereo audio data received from the audio HAL 120 in a stereo stream buffer.

And, the multi-channel plug-in module 130 extracts stereo audio data from each stereo stream buffer in real time to convert a plurality of stereo streams into a single multi-channel stream. And, the multi-channel plug-in module 130 reconstructs the extracted stereo audio data into multi-channel audio data and stores them in a multi-channel buffer.

In addition, the multi-channel plug-in module 130 transfers the multi-channel audio data stored in the multi-channel buffer to the library 140, and finally converts the audio data for the multi-channel stream through a hardware device (not shown) such as a speaker. to output

Here, the structure of the multi-channel plug-in module 130 will be described with reference to FIG. 3 .

3 is a structural diagram of a multi-channel plug-in module according to an embodiment of the present invention.

As shown in FIG. 3, the multi-channel plug-in module 130 includes a plurality of stereo stream buffers 131, a management module 132, a multi-channel converter 133, a multi-channel buffer 134, and a playback module ( 135).

When a plurality of stereo audio data is transmitted from the audio HAL 120, they are respectively stored in the plurality of stereo stream buffers 131. In an embodiment of the present invention, an example in which the number of stereo stream buffers is the same as the number of media applications 200 is described, but is not necessarily limited thereto.

The management module 132 manages underflow or overflow of each stereo stream buffer 131 . When underflow occurs in the stereo stream buffer 131, the user hears audio in the form of interruption. Conversely, if overflow occurs in the stereo stream buffer 131, the user hears the audio output through the speaker as noise.

To manage this, the management module 132 calculates the buffer occupancy rate of the stereo stream buffer 131 in real time, and the audio HAL 120 stores stereo audio data in each stereo stream buffer 131 based on the calculated buffer occupancy rate. determines the write cycle. The management module 132 will be described in detail later.

The multi-channel converter 133 converts a plurality of stereo streams stored in each stereo stream buffer 131 into one multi-channel stream. The multi-channel converter 133 arranges two-channel PCM data according to the stream order in units of frames to configure one multi-channel stream as an example, but is not necessarily limited thereto.

To this end, the multi-channel converter 133 extracts stereo audio data from each stereo stream buffer 131 in real time. And the multi-channel converter 133 reconstructs the extracted stereo audio data into multi-channel audio data and stores them in the multi-channel buffer 134 .

The playback module 135 reproduces multi-channel audio data stored in the multi-channel buffer 134 based on the tinyALSA library and outputs audio data.

Here, an example in which the management module 132 prevents underflow or overflow of each stereo stream buffer 131 will be described with reference to FIG. 4 .

4 is an exemplary view of a management module according to an embodiment of the present invention.

)을 계산한다. 여기서 점유율은 오디오 HAL(120)이 n번째에 스테레오 스트림으로 오디오 데이터를 쓰기 한 후의 스테레오 스트림 버퍼(131)의 점유율을 의미한다. 관리 모듈(132)이 각 스테레오 스트림 버퍼(131)의 점유율을 계산하는 방법은 다양한 방법으로 계산할 수 있으므로 본 발명의 실시예에서는 어느 하나의 방법으로 한정하지 않는다.As shown in FIG. 4, the management module 132 controls the occupancy (occupancy) of each stereo stream buffer 131 to prevent underflow or overflow of each stereo stream buffer 131.

) is calculated. Here, the occupancy means the occupancy of the stereo stream buffer 131 after the audio HAL 120 writes audio data as the n-th stereo stream. Since the management module 132 calculates the occupancy rate of each stereo stream buffer 131 in various ways, it is not limited to any one method in the embodiment of the present invention.

)를 조절하며, 다음 수학식 1과 같다.After the management module 132 calculates the occupancy rate of each stereo stream buffer 131, the audio data writing cycle of each stereo stream buffer 131 according to the calculated occupancy rate (

), and is shown in Equation 1 below.

는 오디오 HAL(120)이 n+1번째에 i번째 스테레오 스트림으로 오디오 데이터를 스테레오 스트림 버퍼(131)에 쓰는 주기로, 본 발명의 실시예에서의 오디오 데이터 쓰기 주기의 단위는 ㎲인 것을 예로 하여 설명하나, 반드시 이와 같이 한정되는 것은 아니다.here,

is a cycle in which the audio HAL 120 writes audio data to the stereo stream buffer 131 as the i-th stereo stream in n + 1 th, and the unit of the audio data writing cycle in the embodiment of the present invention is μs. However, it is not necessarily limited in this way.

And, FS is a frame size, a value obtained by converting the data size (unit: bytes) delivered to the multi-channel plug-in module 130 whenever the audio HAL 120 writes audio data once into a time value (unit: ㎲ )am. Here, a method of converting the data size into a time value is a known technique, and description is not limited to any one method in the embodiment of the present invention.

And, k is a constant value used to prevent overflow of the stereo stream buffer 131. P is a constant value used to prevent underflow of the stereo stream buffer 131. The values of the two constants are not limited to either one.

Referring to Equation 1, the management module 132 sets the audio data writing cycle to a specific value in order to quickly empty each stereo stream buffer 131 when confirming that the calculated occupancy corresponds to the preset first section. . In the embodiment of the present invention, 0 < occupancy rate < 20 will be described as an example for the first section, and 10 μs as a specific value for setting the audio data write cycle will be described as an example.

Meanwhile, the management module 132 determines that the buffer occupancy of the stereo stream buffer 131 is stable when it is confirmed that the calculated occupancy corresponds to the preset second section. Accordingly, the management module 132 sets the audio data writing cycle in proportion to the buffer occupancy rate. In the embodiment of the present invention, 20 ≤ occupancy rate < 90 will be described as an example in the second section.

In addition, when the management module 132 confirms that the calculated occupancy corresponds to the preset third interval, the buffer occupancy of the stereo stream buffer 131 is very high, and thus, it is determined that overflow is very likely to occur. Therefore, the management module 132 sets the audio data writing cycle to a value calculated by FS * k in order to rapidly fill each stereo stream buffer 131 . In the embodiment of the present invention, 90 ≤ occupancy rate < 100 will be described as an example in the third section.

Next, a method of converting a plurality of stereo streams into a single multi-channel stream by the above-described Android automotive system 100 will be described with reference to FIG. 5 .

5 is a flowchart of a method for converting a stream according to an embodiment of the present invention.

As shown in FIG. 5, the Android automotive system 100 initializes the audio HAL 120 (S100). Then, the Android automotive system 100 initializes the multi-channel plug-in module 130 (S110).

Through the initialization of the multi-channel plug-in module 130 executed in step S110, the management module 132 allocates a stereo stream buffer 131, which is a ring buffer for each stereo audio stream operation, and for multi-channel audio stream operation. A multi-channel buffer 134, which is a ring buffer, is allocated. And, when the management module 132 requests the playback module 135 to generate a device for outputting a multi-channel audio stream, the playback module 135 outputs the multi-channel audio stream to the library 140 based on the tinyALSA interface. Request device creation for

When a plurality of stereo audio streams are delivered from the audio HAL 120, the multi-channel plug-in module 130 stores audio data in each stereo stream buffer 131 allocated in step S110 (S120). The multi-channel converter 133 converts the audio data stored in the stereo stream buffer 131 into a multi-channel stream in step S120 (S130).

Then, the playback module 135 reproduces the multi-channel stream converted in step S130 based on the tinyALSA interface and outputs it to an audio device (eg, speaker), which is the final stage of the system (S140). When a plurality of stereo audio streams are reproduced as one multi-channel stream through these steps, the management module 132 checks the buffer occupancy of each stereo stream buffer 131 (S150).

Then, the management module 132 sets a period for the audio HAL 120 to write audio data to the stereo stream buffer 131 based on the confirmed occupancy rate (S160).

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims are also included in the scope of the present invention. that fall within the scope of the right.

Claims (13)

A method in which an Android automotive system operated by at least one processor converts a stream,
Initializing a multi-channel plug-in module to allocate a plurality of stereo stream buffers and one multi-channel buffer;
When a plurality of stereo audio streams transmitted from a plurality of media applications are received from the audio HAL, storing audio data for each stereo audio stream for each media application in the plurality of stereo stream buffers;
converting the audio data stored in the plurality of stereo stream buffers into a multi-channel stream by arranging them in frame order according to the stream order; and
Reproducing the converted multi-channel stream and outputting it to an audio device
Including, stream conversion method. According to claim 1,
After outputting to an audio device,
Checking the buffer occupancy of each stereo stream buffer, and
Setting a data write cycle in which the audio HAL writes the audio data to each stereo stream buffer based on the confirmed buffer occupancy rate
Including, stream conversion method. According to claim 2,
The step of setting the data write cycle,
If the buffer occupancy rate corresponds to the first interval, setting the data write cycle to a preset value, the stream conversion method. According to claim 2,
The step of setting the data write cycle,
If the buffer occupancy rate corresponds to the second interval, setting the data write cycle to a value proportional to the buffer occupancy rate. According to claim 2,
The step of setting the data write cycle,
If the buffer occupancy corresponds to a third period, the data write cycle is set based on a frame size and a constant value for preventing overflow. As an Android automotive system,
An audio hardware abstraction layer (HAL) that stores a plurality of stereo audio streams transmitted from a plurality of media applications, and
A multi-channel plug-in module that converts a plurality of stereo audio streams transmitted from the audio HAL into a multi-channel stream, reproduces the converted multi-channel stream based on a library, and outputs the converted multi-channel stream to an audio device.
Including, Android automotive system. According to claim 6,
The multi-channel plug-in module,
A plurality of stereo stream buffers respectively storing audio data for a plurality of stereo audio streams transmitted from the audio HAL for each media application;
A multi-channel converter for converting the plurality of audio data into a multi-channel stream;
A multi-channel buffer for storing the multi-channel stream converted by the multi-channel converter, and
Playback module for reproducing the multi-channel stream stored in the multi-channel buffer based on the library
Including, Android automotive system. According to claim 7,
The library is a tinyALSA library, Android Automotive System. According to claim 8,
The multi-channel plug-in module,
A management module preventing underflow or overflow of the plurality of stereo stream buffers
Further comprising, the Android automotive system. According to claim 9,
The management module,
Checking the buffer occupancy of each stereo stream buffer, and setting a data write cycle in which the audio HAL writes the audio data to each stereo stream buffer based on the buffer occupancy. According to claim 10,
The management module,
If the buffer occupancy rate corresponds to the first interval, the Android automotive system prevents the underflow by setting the data write cycle to a preset value. According to claim 10,
The management module,
If the buffer occupancy corresponds to the second period, the Android automotive system sets the data write cycle to a value proportional to the buffer occupancy. According to claim 10,
If the buffer occupancy rate corresponds to the third period, the Android automotive system prevents the overflow by setting the data write cycle based on the frame size and a predetermined constant value.

Images