WO2017005010A1

WO2017005010A1 - Audio processing method and device, and computer storage medium

Info

Publication number: WO2017005010A1
Application number: PCT/CN2016/076802
Authority: WO
Inventors: 杨爱民; 于光海
Original assignee: 深圳市中兴微电子技术有限公司
Priority date: 2015-07-03
Filing date: 2016-03-18
Publication date: 2017-01-12
Also published as: CN106325804B; CN106325804A

Abstract

Disclosed are an audio processing method and device, and computer storage medium. The method comprises: allocating a memory space for a DMA buffer area, wherein the DMA buffer area comprises a mixed audio playing buffer area and a mixed audio recording buffer area; declaring a name of a soft interrupt for processing the audio mixing and a corresponding processing function; acquiring addresses of the playing buffer area and the recording buffer area in the soft interrupt for audio mixing, invoking the processing function for audio mixing to perform the audio mixing process, and storing data undergone the audio mixing process in the mixed audio playing buffer area.

Description

Audio processing method and device, computer storage medium

Technical field

The present invention relates to audio technologies in the field of embedded platforms, and in particular, to an audio processing method and device, and a computer storage medium.

Background technique

With the rise of the top (OTT, Over The Top) set-top box business, the installation of K-song software on the set-top box to double the demand for karaoke in the home. On Android-based devices, the ear-back time (the time from the input of the microphone to the time of output from the TV) is above 120 milliseconds (ms), compared to the 40-second ear-back time of the iOS system. The big ear return time affects the user experience.

There is no effective solution for related technologies to effectively reduce the ear-back time of Android-based devices.

Summary of the invention

The embodiment of the invention provides an audio processing method and device, and a computer storage medium, which can reduce the ear return time of the Android device.

The technical solution of the embodiment of the present invention is implemented as follows:

An embodiment of the present invention provides an audio processing method, where the method includes:

Allocating memory space for a direct memory read (DMA) buffer, the DMA buffer including a mix play buffer and a mix recording buffer;

Declare the name of the soft interrupt handling the mix and the corresponding handler function;

The play buffer address and the recording buffer address are obtained in the soft interrupt of the mix, the mix processing function is called to perform the mixing process, and the processed mixed data is saved in the mix play buffer.

Preferably, the method further includes:

Assigning the address of the mix play buffer to the source address of the DMA function pointer;

The address of the mix play buffer is used to support the DMA function from the address data of the mix play buffer to the first in first out (FIFO) queue of the virtual sound card.

Preferably, the method further includes:

In the interrupt callback function of the DMA function, the data of the sound mixing buffer is copied into the sound recording buffer, and the sound recording buffer is used to support the application layer to read the sound by opening the virtual sound card. After the audio data.

Preferably, before the allocation of the mix play buffer and the mix recording buffer in the memory space, the method further includes:

Register a virtual sound card driver and a virtual sound card;

Register platform devices and platform device drivers;

Registered codec (Codec) devices, Codec drivers and Codec Digital Audio Interface (DAI).

Preferably, the method further includes:

Allocating a memory space normal play buffer for the DMA buffer, wherein the normal play buffer is used to support the application layer to write audio data to be played by opening the virtual sound card;

A normal recording buffer is allocated to the DMA buffer, and the normal recording buffer is used to support the application layer to read the original audio data input by the microphone by opening the virtual sound card.

An embodiment of the present invention provides an audio processing device, where the device includes:

a buffer unit configured to allocate a memory space for the DMA buffer, where the DMA buffer includes a mix play buffer and a mix recording buffer;

a soft interrupt declaration unit configured to declare a name of a soft interrupt that handles the mix and a corresponding processing function;

The soft interrupt processing unit is configured to acquire the play buffer address and the recording buffer address in the soft interrupt of the mix, call the mix processing function for mixing processing, and process the data after the mixing There is the mix play buffer.

Preferably, the device further includes:

a mixing playback processing unit configured to assign an address of the sound mixing buffer to a source address of a DMA function pointer;

The address of the mix play buffer is configured to support the DMA function from the address data of the mix play buffer to the FIFO queue of the virtual sound card.

Preferably, the device further includes:

a sound recording processing unit configured to copy data of the sound mixing buffer into the sound recording buffer in an interrupt callback function of the DMA function, wherein the sound recording buffer is used to support an application layer Read the mixed audio data by turning on the virtual sound card.

Preferably, the device further includes:

a registration unit configured to perform the following operations before the allocation of the mix play buffer and the mix recording buffer in the buffer unit memory space; registering the virtual sound card driver and the virtual sound card; registering the platform device and the platform device driver; registering the codec Device, Codec driver and Codec DAI.

Preferably, the buffer unit is further configured to allocate a memory space normal play buffer for the DMA buffer, and the normal play buffer is configured to support the application layer to write audio data to be played by opening the virtual sound card;

The buffer unit is further configured to allocate a normal recording buffer to the DMA buffer, and the normal recording buffer is configured to support the application layer to read the original audio data input by the microphone by opening the virtual sound card.

In a third aspect, an embodiment of the present invention provides a computer storage medium, where the computer storage medium stores executable instructions, and the executable instructions are used to execute an audio processing method provided by an embodiment of the present invention.

In the embodiment of the present invention, the play buffer address and the recording buffer address are obtained through the soft interrupt of the mix, and the mix processing function is called to perform the mixing process, for example, when running on the ARM processor. In the Android operating system, the mixing processing at the Linux kernel level is realized (instead of the mixing processing implemented by the related technology in the application layer), and the mixing processing result is provided to the application layer, and the application layer does not need to perform the mixing processing by itself. The application layer of the related art directly performs the mixing process to greatly shorten the ear return delay.

DRAWINGS

1 is a schematic flowchart of an implementation process of an audio processing method according to an embodiment of the present invention;

2 is a flowchart of an ASoC-based audio processing method according to an embodiment of the present invention;

3 is a flowchart of a call of a soc_probe function according to an embodiment of the present invention;

4 is a flowchart of a soc_probe_dai_link() function according to an embodiment of the present invention;

FIG. 5 is a flowchart of a snd_soc_instantiate_card function call according to an embodiment of the present invention;

6 is a schematic diagram of a virtual sound card being mounted under a snd_card according to an embodiment of the present invention;

FIG. 7 is a flowchart of a tasklet mixing according to an embodiment of the present invention; FIG.

FIG. 8 is a first schematic structural diagram of an audio processing device according to an embodiment of the present invention; FIG.

FIG. 9 is a second schematic structural diagram of an audio processing device according to an embodiment of the present invention.

detailed description

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

Android devices have a relatively standard and robust architecture for processing audio. The architecture includes: application layer, Java framework service Audio Mananger, local service Audio Flinger, abstraction layer Alsa HAL, local library, external (external) Alsa-lib support library, The underlying coding (Codec) driver;

Because audio data has undergone too much processing in the above architecture, resulting in increased audio delay, such a complex system architecture cannot meet the needs of real-time scenarios, such as Network K songs need to send the mixed data to other on-demand users on the network, and the large delay affects the user experience.

The audio processing device described in the embodiment of the present invention is an Advanced Linux Sound Architecture (ALSA) based on Android, and is based on an Android device, an Advanced RISC Machine (ARM) device, and a Linux device running on an Android device. The virtual sound card is added to the device list of the kernel, and the virtual sound card directly intercepts the audio data (such as the audio data input by the user through the microphone) and the background audio data (the background sound of the song playing) for mixing processing, and the application layer can directly obtain the mixed sound. Sound data, no need to be mixed by the application layer itself, reducing the ear return time;

In order to achieve the above object, in the embodiment of the present invention, in view of the insufficiency of mixing in the existing Linux kernel, a virtual sound card device (referred to as a virtual sound card) is added to the device list of the Linux kernel, and the virtual sound card resource is registered into the Linux. The kernel is managed by the Linux kernel;

The audio processing device provided in this embodiment is based on the ALSA architecture of the system-on-a-chip (SOC), that is, ASOC, and the ASoC runs the Linux kernel for audio processing. The Linux kernel running in the ASoC is provided with a virtual sound card, a Platform, a virtual sound card Codec, and a virtual sound card Codec driver. The audio data is processed by the virtual sound card at the Linux kernel level (instead of the application layer), and the Linux kernel is added with a normal play buffer, a normal recording buffer, a mix play buffer, and a mix recording buffer for different The application scene stores audio data, and the two channels of data are mixed by soft interrupt when the recorded data and the play data arrive, and the application layer can directly read the mixed data.

The driver of the virtual sound card is set under the driver management and registration mechanism of the Linux, including: defining the platform device (platform_device) structure corresponding to the virtual sound card, registering the platform_device structure corresponding to the virtual sound card, and defining the platform driver (platform_driver) structure corresponding to the virtual sound card. Body, and the platform_driver structure of the registered virtual sound card; adding virtual sound card, Platform, virtual sound card Codec and virtual sound card Codec driver processing follows the Linux driver Management and registration mechanism;

The following describes the processing of setting the virtual sound card, Platform, virtual sound card Codec and virtual sound card Codec driver. For the details, please refer to the relevant specification document of Linux driver management and registration mechanism.

The Linux kernel running on the Android device includes a bus, that is, a virtual bus or platform bus, a platform device (platform_device), and a platform driver (platform_driver); the virtual bus binds the platform device and the platform driver: registering a platform driver At the time, the virtual bus looks for platform devices that match the platform drivers.

Virtual sound card resource registration and mixing soft interrupt (Tasklet) processing, if you want a virtual sound card to work properly, you need to complete three steps: 1, virtual sound card driver and virtual sound card registration; 2, platform equipment and platform device driver The registration of the platform device mainly implements the direct memory access (DMA) function. The name of the platform device driver is consistent with the name of the platform device: 3. Codec device (Codec), Codec driver and Codec Registration of Digital Audio Interface (DAI, Digital Aaudio Interface);

1. The registration of the virtual sound card driver and the virtual sound card device (referred to as the virtual sound card), the name of the virtual sound card driver must be the same as the name of the platform device:

1) Set the virtual sound card driver, the relevant code is as follows:

2) Register the virtual sound card driver, the relevant code is as follows:

3) Set the virtual sound card, including the start and end addresses of the physical address occupied by the virtual sound card in the memory, and related flag bits.

The resource information of the virtual sound card is described by the platform_device structure, which includes the name (name), the identifier (ID) of the virtual sound card, the codec resource of the defined virtual sound card, and the resource of the Codec of the virtual sound card. The number of bytes occupied.

The relevant code is as follows:

4) Register the virtual sound card resources (zx_devices is an array containing all defined virtual devices), the relevant code is as follows:

Platform_add_devices(zx_devices,ARRAY_SIZE(zx_devices))

5) Register the virtual sound card, the relevant code is as follows:

2. Platform device and platform driver registration, platform device implements DMA function, platform driver name and platform device name are the same, both are zx-audio-platform:

1) Platform driver settings, the relevant code is as follows:

2) Platform-driven registration, the relevant code is as follows:

3) The setting of the platform device, the relevant code is as follows:

4) Registration of platform equipment:

Platform_add_devices(zx_devices,ARRAY_SIZE(zx_devices));

3. Registration of Codec, Codec drivers and Codec DAI.

1) The definition of Codec driver, the relevant code is as follows:

2) Registration of the virtual Codec driver, the relevant code is as follows:

3) The definition of the virtual Codec device, the relevant code is as follows:

4) Registration of the virtual Codec device, the relevant code is as follows:

Platform_add_devices(zx_devices,ARRAY_SIZE(zx_devices));

5) Codec DAI definition, the relevant code is as follows:

6) Registration of Codec DAI, the relevant code is as follows:

In addition to setting the virtual sound card, Platform, virtual sound card Codec and virtual sound card Codec driver in the Linux kernel according to the above steps, it is also necessary to set the structure of the record buffer and the soft interrupt (tasklet) processing function for processing the mix; wherein the buffer contains The normal play buffer, the normal recording buffer, the mix play buffer, and the mix recording buffer are configured to store audio data for different application scenarios; the soft break (tasklet) processing function is configured to process audio data input by the user through the microphone and Mixing of background audio.

As shown in Figure 1, based on Android devices such as set-top boxes running the above Linux kernel, audio The processing method includes the following steps:

Step 101: Allocate a memory space for a direct memory read (DMA) buffer, where the DMA buffer includes a mix play buffer and a mix recording buffer.

Step 102, declaring the name of the soft interrupt handling the mix and the corresponding processing function.

Step 103: Acquire a play buffer address and a recording buffer address in the soft interrupt of the mix, call a mix processing function to perform a mixing process, and save the processed data in the mix play buffer.

When the mix play is required, the address of the mix play buffer is assigned a source address of the DMA function pointer; the DMA function is based on the address of the mix play buffer, from the address of the mix play buffer Data to the virtual sound card's first in first out (FIFO) queue.

When the sound recording is required, in the interrupt callback function of the DMA function, the data of the sound mixing buffer is copied into the sound recording buffer, and the sound recording buffer is used to support the application layer. Turn on the virtual sound card to read the audio data after the mix.

In addition, before performing the above operations, you need to set up a virtual sound card, register virtual sound card driver and virtual sound card; register platform device and platform device driver; register codec device, Codec driver and Codec digital audio interface (DAI) Registered platform devices and platform device drivers are used to implement DMA functions.

For scenes other than normal play (that is, non-mixed play), when normal play is required, a normal play buffer is allocated for the DMA buffer, and the normal play buffer is used to support the application layer by opening the virtual The sound card writes audio data that needs to be played;

When the application layer needs to obtain the original audio data input by the microphone, the normal recording buffer is allocated to the DMA buffer, and the normal recording buffer is used to support the application layer to read the original audio data input by the microphone by opening the virtual sound card. .

The following describes an audio processing method shown in FIG. 2 for an Android device such as a set top box running the above Linux kernel, including the following steps:

In step 201, the platform bus matches the device and driver whose names are virtual sound cards.

Subsequent step 202 also triggers the invocation of the soc_probe function, which is the entry for the entire ASoC driver initialization.

According to the Linux device model, if there is platform_device, there will be platform_driver. The name field in the AS_C platform_driver is soc-audio, which is exactly the same as the name in the platform_device defined in step 1. According to the Linux device model, the platform bus will Matching the two devices and drivers with the same name will also trigger the call to soc_probe, which is the entry point for the entire ASoC driver initialization.

Step 202: Call the soc_probe function.

As shown in Figure 3, the call to the soc_probe function includes the following steps:

In step 301, the snd_soc_register_card is called in the soc_probe function to apply for memory for the snd_soc_pcm_runtime array, and each dai_link corresponds to a unit of the snd_soc_pcm_runtime array.

Step 302: Copy the dai_link configuration in the snd_soc_card to the corresponding snd_soc_pcm_runtime.

In step 303, the snd_soc_instantiate_card function is called.

The code corresponding to the soc_probe function is as follows:

Snd_soc_card*card=platform_get_drvdata(pdev)

Snd_soc_register_card(card)

Card->rtd=kzalloc(sizeof(struct snd_soc_pcm_runtime)*(card->num_links+card->num_aux_devs), GFP_KERNEL)

For(i=0;i<card->num_links;i++)

Card->rtd[i].dai_link=

&card->dai_link[i]

Snd_soc_instantiate_cards()

Snd_soc_instantiate_card()

Step 203: The snd_soc_instantiate_card function call process, The snd_soc_instantiate_card function traverses each pair of dai_links, binds Codec, Platform, and DAI, and the snd_soc_instantiate_card function calls the soc_probe_dai_link function. The call flow of the soc_probe_dai_link function is shown in Figure 4. In the soc_probe_dai_link function, the Codec, DAI, and platform-driven probes are called. In addition to the function, the soc_new_pcm() function is also called at the end to create a virtual sound card.

The snd_soc_instantiate_card function call flow is shown in Figure 5 and includes the following steps:

In step 401, card->instantiated is used to determine whether the virtual sound card has been instantiated, and if it has been instantiated, it returns directly.

In step 402, each pair of dai_links is traversed, and the binding of codec, platform, and DAI is performed.

In step 403, the register buffer of Codec is initialized.

In step 404, a virtual sound card is created.

A schematic diagram of the virtual sound card being mounted under the snd_card is shown in FIG. 6.

In step 405, the probe function of each substructure is called in turn.

In step 406, the soc_probe_dai_link() function is executed.

Step 407, initialization and setting.

In step 408, the virtual sound card is registered by calling the sound card registration function driven by the standard ALSA.

Step 204: The audio data input by the user through the microphone and the audio data of the background music are processed and mixed by processing a soft break of the mix.

As shown in Figure 7, step 204 is implemented by the following steps:

Step 501: Define a structure of a record buffer, including a normal play buffer, a normal recording buffer, a mix play buffer, and a mix recording buffer, and store audio data for different application scenarios. The code is as follows:

Define a pointer to the record buffer:

In the function that allocates kernel space for the DMA buffer, the four pointers are assigned respectively.

Step 502, declaring the name of the tasklet that processes the mix and the corresponding processing function, that is, DECLARE_TASKLET (zx_i2s_spdif_mix_tasklet, zx_snd_soc_mix_taskelet, 0).

In step 503, the tasklet is initialized in the DMA interrupt callback function (only needs to be initialized once), and the tasklet_schedule is called to dispatch the tasklet.

Step 504: Obtain a mix play buffer address and a mix recording buffer address in the mixed tasklet, call a mix processing function to perform a mix process, and save the processed data in the mix play buffer.

In step 505a, when the mixing is played, the address of the mixing play buffer is assigned to the source address of the DMA pointer, and the DMA transfers data from the address to the first-in-first-out (FIFO) queue of the virtual sound card, thereby realizing the mixing play. .

Step 506b, when the sound recording is realized, in the DMA interrupt callback function, the data of the sound mixing buffer is copied into the sound recording buffer, and the application layer can read the mixed audio data by opening the virtual sound card. Normal play.

This can mix two audio data after each recorded data and playback data arrives; the tasklet mechanism is a relatively common mechanism, usually used to reduce the interrupt processing time, which should be in the interrupt service routine. The completed task is converted to a soft interrupt completion.

In order to avoid the interrupt processing time being too long (interruption, if the mixing operation will cause the interrupt processing time to be too long), the interrupt will be lost, and some tasks that are not very urgent in the interrupt processing will be executed later, and the interrupt will be interrupted. The handler returns as soon as possible.

In the tasklet processing, the addresses of the normal play buffer, the normal recording buffer, and the mix play buffer are recorded as parameters, and the mixing process is performed by the following mixing algorithm:

Use A and B to represent two audio signals, and Y to represent the audio signal after mixing.

For an n-bit audio sample signal, if both A and B are negative, then

Y=A+B-(A*B/(-(2pow(n-1)-1)))

otherwise:

Y=A+B-(A*B/(2pow(n-1)).

C language means:

In the DMA interrupt callback handler, the tasklet processing function tasklet_init(&zx_i2s_spdif_mix_tasklet, zx_snd_soc_mix_taskelet, (unsigned long) arg) of the mix is registered for the first time, and tasklet_schedule(&zx_i2s_spdif_mix_tasklet) is called;

In the DMA interrupt callback processing function, firstly assign the stream pointer of the recorded data and the playback data to the global variable pcm_substream, and then pass the two-way audio stream pointer to the tasklet of the mixing process through the global variable. Finally, in the task bar of the mixing process, the two pointers are respectively applied to the above-mentioned mixing algorithm, and after the algorithm is processed, the mixed data is obtained (the data is stored in the buffer pointed to by the mixing play pointer). If the data for DMA transfer is 1 KB, the size of the recorded data and the playback data is 1 KB each time the mix is mixed, and the data of the mix output is still 1 KB.

Since the mixed data is processed in two ways, the mixed data is played and the audio data is obtained through the virtual sound card. Therefore, in the DMA interrupt processing function, the physical address pointer of the DMA at the time of playback needs to be replaced with the mixed after the mixing processing. DMA physical address pointer for audio data playback, so After the change, the originally played sound data is replaced with the data after the mixing process, and the data of the sound mixing buffer is copied to the buffer pointed to by the sound recording buffer pointer, and snd_pcm_period_elapsed is called to notify the Linux kernel. The current data has been processed.

When the application layer needs to obtain the original audio data input by the user's microphone, the data can be read by opening the device dev/snd/pcmC0D0c. When the application layer wants to play the sound, it can be written by opening the dev/snd/pcmC1D0p device. Into the data. When the application layer wants to play the mix data directly, the function of mixing the device dev/snd/pcmC0D0c and the device dev/snd/pcmC1D0p can be realized by opening the dev/snd/pcmC2D0p device to write data. When the application layer wants to transmit the mixed data through the network, the dev/snd/pcmC2D0c device can be opened to read the mixed data and send it to other nodes on the network.

Since the physical address needs to be configured when configuring the DMA, the function pointer that needs to be transmitted in the function for processing the mix points to the mix play buffer, that is, the pointer records the virtual address, so it is necessary to allocate the record mix. In the buffer, the dma_alloc_coherent is called to record the virtual address and the physical address of the buffer of the mixing process, and the virtual address and the physical address of the buffer of the mixing process are in a one-to-one correspondence, so that the buffer is processed into the buffer. When the virtual address writes data, it can be handled by transferring the physical address of the buffer of the mixing process to the DMA.

The virtual sound card resource registration involved in the embodiment of the present invention mainly performs the following work:

Two sets of resources are defined, which describe the resources of a virtual sound card. The first group describes the bus address range occupied by the virtual sound card. IORESOURCE_MEM indicates that the first group describes the memory type resource information, and the second group describes this. The interrupt number of the mixing device, IORESOURCE_IRQ indicates that the second group describes the interrupt resource information, and the virtual sound card driver obtains the corresponding resource information according to the flags.

Based on the resource information, you can define platform_device. After defining the platform_device structure, you can call the function platform_add_devices to add a virtual sound card to the Linux system. The virtual sound card can be registered by calling platform_device_register(). For the virtual sound card Codec device, only the name and ID identifier need to be defined.

After calling the platform_add_devices to add the virtual sound card and the virtual sound card Codec device successfully, the virtual sound card and the virtual sound card Codec will be added to the Linux system's device list.

The soc_new_pcm function in the Linux kernel will call the function that requests the memory space for the virtual sound card. This address space is used to record the data after the mixing process. In the DMA interrupt callback function, the physics corresponding to the memory buffer processed by the mix is processed. The address is allocated to the DMA for mixing playback, and the mixing processing is completed based on the tasklet. The tasklet reads the DMA-transferred microphone data and the background music sent by the application layer for the mixing process, and processes the recorded and played data. The size is currently 1K of data per processing, and the data after mixing is also 1K.

When the above steps are completed, the user can play back the mix by reading and writing /dev/snd/pcmC2D0p, and the mix can be recorded by dev/snd/pcmC2D0c.

After the actual test, the ear return time of the mixing scheme based on the technical solution of the embodiment of the present invention is reduced to 60 ms, which can meet the demand of the product.

The embodiment of the invention further describes a computer storage medium, wherein the computer storage medium stores executable instructions, and the executable instructions are used to execute the audio processing method shown in FIG. 1 or FIG.

The embodiment of the invention describes an audio processing device based on the Linux kernel, which performs mixing processing at the kernel level based on the running Linux kernel. As shown in FIG. 8, the device includes:

The buffer unit 100 is configured to allocate a memory space for the direct memory read DMA buffer, where the DMA buffer includes a mix play buffer and a mix recording buffer;

a soft interrupt declaration unit 200 configured to declare a name of a soft interrupt for processing a mix and a corresponding processing function;

Soft interrupt processing unit 300, configured to acquire a play buffer address and a soft interrupt in the mix Record the buffer address, call the mix processing function to perform the mixing process, and save the processed data in the mix play buffer.

As an example, referring to FIG. 9, the device further includes:

The mixing playback processing unit 400 is connected to the soft interrupt processing unit 300, and configured to assign an address of the sound mixing buffer to a source address of the DMA function pointer;

The address of the mix play buffer is used to support the DMA function from the address data of the mix play buffer to the first in first out FIFO queue of the virtual sound card.

As an example, referring to FIG. 9, the device further includes:

The sound recording processing unit 500 is connected to the soft interrupt processing unit 300, and configured to copy the data of the sound mixing buffer into the sound recording buffer in the interrupt callback function of the DMA function, the mixing The tone recording buffer is used to support the application layer to read the mixed audio data by turning on the virtual sound card.

As an example, referring to FIG. 9, the device further includes:

The registration unit 600 is connected to the buffer unit 100, and configured to perform the following operations before registering the sound mixing buffer and the mixing recording buffer in the memory space of the buffer unit 100; registering the virtual sound card driver and the virtual sound card; Device and platform device drivers; registered codec Codec devices, Codec drivers and Codec digital audio interfaces DAI.

As an example, the buffer unit 100 is further configured to allocate a memory space normal play buffer for the DMA buffer, and the normal play buffer is used to support the application layer to write audio data to be played by opening the virtual sound card;

The buffer unit 100 is further configured to allocate a normal recording buffer to the DMA buffer, and the normal recording buffer is configured to support the application layer to read the original audio data input by the microphone by opening the virtual sound card.

Each of the above units is a virtual functional unit, and the actual application can be implemented by an ASoC such as a processor in a set top box (such as an ARM processor), a logic programmable gate array (FPGA), or a dedicated integrated power supply. Road (ASIC) implementation.

The embodiment of the invention further provides a computer storage medium, wherein the computer storage medium stores executable instructions, and the executable instructions are used to execute the audio processing method described in the embodiment of the invention.

A person skilled in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by using hardware related to the program instructions. The foregoing program may be stored in a computer readable storage medium, and the program is executed when executed. The foregoing storage medium includes: a mobile storage device, a random access memory (RAM), a read-only memory (ROM), a magnetic disk, or an optical disk. A medium that can store program code.

Alternatively, the above-described integrated unit of the present invention may be stored in a computer readable storage medium if it is implemented in the form of a software function module and sold or used as a standalone product. Based on such understanding, the technical solution of the embodiments of the present invention may be embodied in the form of a software product in essence or in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for making A computer device (which may be a personal computer, server, or network device, etc.) performs all or part of the methods described in various embodiments of the present invention. The foregoing storage medium includes various media that can store program codes, such as a mobile storage device, a RAM, a ROM, a magnetic disk, or an optical disk.

The above is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope of the present invention. It should be covered by the scope of the present invention. Therefore, the scope of the invention should be determined by the scope of the appended claims.

Claims

An audio processing method, the method comprising:

Allocating a memory space for the direct memory read DMA buffer, the DMA buffer including a mix play buffer and a mix recording buffer;

Declare the name of the soft interrupt handling the mix and the corresponding handler function;

The play buffer address and the recording buffer address are obtained in the soft interrupt of the mix, the mix processing function is called to perform the mixing process, and the processed mixed data is saved in the mix play buffer.
The method of claim 1 wherein the method further comprises:

Assigning the address of the mix play buffer to the source address of the DMA function pointer;

The address of the mix play buffer is configured to support the DMA function from the address data of the mix play buffer to the first in first out FIFO queue of the virtual sound card.
The method of claim 1 wherein the method further comprises:

In the interrupt callback function of the DMA function, the data of the sound mixing buffer is copied into the sound recording buffer, and the sound recording buffer is used to support the application layer to read the sound by opening the virtual sound card. After the audio data.
The method of claim 1 wherein prior to allocating the mix play buffer and the mix recording buffer in the memory space, the method further comprises:

Register a virtual sound card driver and a virtual sound card;

Register platform devices and platform device drivers;

Register codec Codec device, Codec driver and Codec digital audio interface DAI.
The method of any of claims 1 to 4, wherein the method further comprises:

Allocating a memory space normal play buffer for the DMA buffer, wherein the normal play buffer is used to support the application layer to write audio data to be played by opening the virtual sound card;

A normal recording buffer is allocated to the DMA buffer, and the normal recording buffer is used to support the application layer to read the original audio data input by the microphone by opening the virtual sound card.
An audio processing device, the device comprising:

a buffer unit configured to allocate a memory space for the direct memory read DMA buffer, where the DMA buffer includes a mix play buffer and a mix recording buffer;

a soft interrupt declaration unit that declares the name of the soft interrupt that handles the mix and the corresponding handler function;

The soft interrupt processing unit is configured to acquire a play buffer address and a recording buffer address in the soft interrupt of the mix, call a mix processing function to perform a mixing process, and save the processed data in the mix play buffer. Area.
The system of claim 6 wherein said device further comprises:

a mixing playback processing unit configured to assign an address of the sound mixing buffer to a source address of a DMA function pointer;

The address of the mix play buffer is used to support the DMA function from the address data of the mix play buffer to the first in first out FIFO queue of the virtual sound card.
The system of claim 6 wherein said device further comprises:

a sound recording processing unit configured to copy data of the sound mixing buffer into the sound recording buffer in an interrupt callback function of the DMA function, wherein the sound recording buffer is used to support an application layer Read the mixed audio data by turning on the virtual sound card.
The system of claim 6 wherein said device further comprises:

a registration unit configured to perform the following operations before the allocation of the mix play buffer and the mix recording buffer in the buffer unit memory space; registering the virtual sound card driver and the virtual sound card; registering the platform device and the platform device driver; registering the codec Codec device, Codec driver and Codec digital audio interface DAI.
A system according to any one of claims 6 to 9, wherein

The buffer unit is further configured to allocate a memory space normal play buffer for the DMA buffer, where the normal play buffer is used to support the application layer to write audio data to be played by opening the virtual sound card;

The buffer unit is further configured to allocate a normal recording buffer for the DMA buffer, and the normal recording buffer is configured to support the application layer to read the original audio data input by the microphone by opening the virtual sound card.
A computer storage medium having executable instructions stored therein, the executable instructions being configured to perform the audio processing method of any one of claims 1 to 5.