WO2019149017A1

WO2019149017A1 - Audio data processing method, apparatus, electronic device and storage medium

Info

Publication number: WO2019149017A1
Application number: PCT/CN2018/125945
Authority: WO
Inventors: 程乐
Original assignee: 北京金山云网络技术有限公司; 北京金山云科技有限公司; 北京金迅瑞博网络技术有限公司
Priority date: 2018-01-31
Filing date: 2018-12-30
Publication date: 2019-08-08
Also published as: CN110096250B; CN110096250A

Abstract

Provided are an audio data processing method, an apparatus, an electronic device and a storage medium. The method comprises: a client receives audio data and a first implementation pointer sent by an operating system; when the first implementation pointer is NULL, performing sound effect processing on the audio data to obtain the audio data of non-low latency sound effect; when the first implementation pointer is non-NULL, performing sound effect processing on the audio data to obtain the audio data of low latency sound effect; the client sends the audio data of non-low latency sound effect or the audio data of low latency sound effect and a second implementation pointer to the operating system; and the operating system plays the received audio data of non-low latency sound effect or audio data of low latency sound effect according to the second implementation pointer. The apparatus can respectively obtain the audio data of non-low latency sound effect or the audio data of low latency sound effect, and play the different audio data, thereby reducing the maintenance cost of software.

Description

Audio data processing method, device, electronic device and storage medium

This application claims the priority of the Chinese Patent Application entitled "Audio Data Processing Method, Apparatus, Electronic Device and Storage Medium" filed on January 31, 2018 by the Chinese Patent Office, Application No. 201810099469.3, the entire contents of which are incorporated by reference. The citations are incorporated herein by reference.

Technical field

The present application relates to the field of data processing technologies, and in particular, to an audio data processing method, apparatus, electronic device, and storage medium.

Background technique

At present, the development of mobile live broadcast is relatively rapid. In the mobile live broadcast scenario, the scene of the anchor and the song is very common. However, due to the delay or deterioration of the sound during the accompaniment, the time when the anchor hears the accompaniment is delayed or the accompaniment is not clear at all, which affects the live broadcast effect. Therefore, the anchor needs to wear the ear return and hear the accompaniment through the ear return. Your own voice, so that you can adjust your singing style, you can also add some sound effects (such as reverb, tone, etc.) to enhance the fun, get a better live broadcast effect.

The delay of the ear return is a very important parameter, and the delay will greatly reduce the experience of singing. On the Android platform, the general live app uses the non-low-latency ear-back mode to achieve the ear-return effect, and the delay is basically more than 200ms. Such a high delay is greatly reduced for the practicality of the ear return.

At present, in order to use the low-frequency ear-back mode on the Android platform to achieve low-latency ear-back effect, the low-latency ear-back mode is often used to play audio data. In the prior art, the audio is played in the low-delay ear-back mode. The data process is usually as follows: the operating system collects audio data in the C language layer module to obtain audio data, and the audio processing client processes the audio data collected by the operating system C language layer module through the callback function to obtain the processed audio data. The operating system then plays the processed audio data through a callback function.

However, since some types of terminal devices do not support low-latency characteristics, for such terminal devices, when the low-frequency ear return is implemented on the client using the above-mentioned prior art, there is a problem that the ear returns have a noise or the sound is abnormal. That is to say, some terminal equipments may not support low-latency characteristics. Therefore, in order to avoid the above problems, developers need to implement both non-low-latency ear-back mode and low-delay ear-back mode, while supporting low-latency characteristics. The low-latency ear-back mode is used on the terminal device, and the non-low-latency ear-back mode is used on the terminal device that does not support the low-latency feature.

However, since the low-latency ear-back mode requires that the audio data does not pass through the JAVA layer during acquisition, processing, and playback, the non-low-latency ear-back mode requires audio data to be transmitted through the JAVA layer, thereby achieving both low-latency ear-back mode and non-low Delayed ear-return mode will greatly increase the implementation complexity of the live app and the maintenance complexity and maintenance cost of the live app.

Summary of the invention

An object of the present application is to provide an audio data processing method, apparatus, electronic device, and computer readable storage medium to reduce the maintenance cost of software when a non-low delay ear return mode and a low delay ear return mode coexist. The specific technical solutions are as follows:

In a first aspect, an embodiment of the present application provides an audio data processing method, which is applied to a client, where the method includes: receiving a first implementation pointer and audio data sent by an operating system; wherein, if the currently used audio processing mode is non- The low-latency ear-back mode, the type of the first implementation pointer is a first type; if the currently used sound processing mode is a low-latency ear-back mode, the type of the first implementation pointer is a second type; The first implementation pointer is sent after the operating system collects the audio data; determining the type of the first implementation pointer; if it is determined that the type of the first implementation pointer is the first type, performing the received audio data Sound processing, obtaining non-low delay sound effect audio data; after obtaining the non-low delay sound effect audio data, generating a second implementation pointer of the first type, and the second implementation pointer and the non-low delay sound effect Transmitting audio data to the operating system, for triggering, by the operating system, the number of non-low delay audio effects according to the second implementation pointer Performing playback; if it is determined that the type of the first implementation pointer is the second type, the audio data is called from the operating system by using a first callback function, and audio processing is performed on the audio data to obtain low-latency audio audio data. After obtaining the low-latency audio audio data, generating a second implementation pointer of the second type, and transmitting the second implementation pointer and the low-latency audio audio data to the operating system for triggering The operating system plays the low-latency audio audio data according to the second implementation pointer.

In a second aspect, an embodiment of the present application provides an audio data processing method, which is applied to an operating system of a terminal, where the method includes: collecting audio data; determining from a non-low delay ear return mode and a low delay ear return mode. The currently used sound processing mode; according to the currently used sound processing mode, generating a first implementation pointer, when the sound processing mode is a non-low delay ear return mode, the first implementation pointer type is the first type, When the audio processing mode is the low delay ear return mode, the type of the first implementation pointer is the second type; the first implementation pointer and the audio data are sent to the client, so that the client determines the An implementation of the pointer type is a first type, obtaining the audio data sent by the operating system, performing audio processing on the audio data, to obtain non-low delay audio audio data, if the client determines the first implementation The type of the pointer is a second type, and the audio data is called by the first callback function, and the audio data is processed by the sound effect to obtain a low delay. Late sound effect audio data; after obtaining the non-low delay sound effect audio data or the low delay sound effect audio data, feeding back the obtained data to the operating system; receiving the non-low delay sound effect sent by the client The audio data or the low-latency audio audio data is played back to the received audio data.

In a third aspect, an embodiment of the present application provides an audio data processing apparatus, which is applied to a client, and includes: a data receiving module, configured to receive a first implementation pointer and audio data sent by an operating system; The sound processing mode is a non-low delay ear return mode, and the type of the first implementation pointer is a first type; if the currently used sound processing mode is a low delay ear return mode, the type of the first implementation pointer is a second type; the first implementation pointer is sent after the operating system collects audio data; the pointer type determining module is configured to determine a type of the first implementation pointer; and the first data processing module is configured to determine The first implementation pointer is of a first type, and performs sound processing on the received audio data to obtain non-low delay audio audio data; and the first data sending module is configured to obtain the non-low delay audio audio data. Thereafter, generating a second implementation pointer of the first type, and the second implementation pointer and the non-low latency sound effect Sending data to the operating system, for triggering the operating system to play the non-low delay audio audio data according to the second implementation pointer; and the second data processing module is configured to determine the first implementation The type of the pointer is a second type, the audio data is called from the operating system by a first callback function, and the audio data is subjected to sound processing to obtain low-latency audio audio data; and the second data sending module is set to After obtaining the low-latency audio audio data, generating a second implementation pointer of the second type, and transmitting the second implementation pointer and the low-latency audio audio data to the operating system, for triggering the The operating system plays the low-latency audio audio data according to the second implementation pointer.

In a fourth aspect, an embodiment of the present application provides an audio data processing apparatus, which is applied to an operating system of a terminal, where the apparatus includes: a data collection module configured to collect audio data; and a mode determining module configured to be non-low Determining the currently adopted sound effect processing mode in the delayed ear return mode and the low delay ear return mode; the pointer generating module is configured to generate a first implementation pointer according to the currently adopted sound effect processing mode, when the sound effect processing mode is non-low delay In the ear return mode, the type of the first implementation pointer is the first type, and when the sound processing mode is the low delay ear return mode, the type of the first implementation pointer is the second type; the data sending module is set to the client Transmitting the first implementation pointer and the audio data, so that if the client determines that the type of the first implementation pointer is a first type, obtaining the audio data sent by the operating system, the audio The data is subjected to sound processing to obtain non-low delay sound effect audio data, if the client determines that the type of the first implementation pointer is a second type, the audio data is called by the first callback function, and the audio data is subjected to sound processing to obtain low-latency audio audio data; after the non-low-latency audio audio data or the low-delay audio audio data is obtained And the received data is fed back to the operating system; the data receiving module is configured to receive the non-low delay sound effect audio data or the low delay sound effect audio data sent by the client, and the received audio data Play it.

In a fifth aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, a memory configured to store a computer program, and a processor configured to execute a program stored on the memory, to implement any of the foregoing The steps of the client's audio data processing method.

In a sixth aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, a memory configured to store a computer program, and a processor configured to execute the program stored on the memory to implement any of the foregoing applications. The steps of the audio data processing method of the operating system.

In a seventh aspect, the embodiment of the present application provides a computer readable storage medium, where the computer readable storage medium stores instructions, when executed on a computer, causing the computer to execute any of the above-mentioned audio data applied to the client. Approach.

In an eighth aspect, an embodiment of the present application provides a computer readable storage medium, where the computer readable storage medium stores an instruction, when it is run on a computer, causing the computer to perform any of the above applications to be set as a terminal operation. The audio data processing method of the system.

In a ninth aspect, the embodiment of the present application provides a computer program product comprising instructions, when executed on a computer, causing the computer to execute any of the above-described audio data processing methods applied to the client.

In a tenth aspect, the embodiment of the present application provides a computer program product comprising instructions, when executed on a computer, causing the computer to execute any of the above audio data processing methods applied to the terminal operating system.

In an eleventh aspect, the embodiment of the present application further provides a computer program that, when run on a computer, causes the computer to execute any of the above-described audio data processing methods applied to the client.

In a twelfth aspect, the embodiment of the present application further provides a computer program that, when run on a computer, causes the computer to execute any of the above-described audio data processing methods applied to the terminal operating system.

In the audio data processing method and apparatus provided by the embodiment of the present application, the client receives the audio data sent by the operating system and the first implementation pointer, and performs sound processing on the audio data when the type of the first implementation pointer is NULL. Obtaining non-low-latency audio audio data; when the first implementation pointer type is non-NULL, performing audio processing on the audio data to obtain low-latency audio audio data, and the client will perform non-low-latency audio data or low-latency sound effects. The audio data and the second implementation pointer are sent to the operating system, and the operating system plays the received non-low delay audio audio data or the low delay audio audio data according to the second implementation pointer. The present application can obtain non-low delay audio audio data and low delay audio audio data through a set of methods or devices, and play the above different audio data, thereby reducing the maintenance cost of the software.

Of course, implementing any of the products or methods of the present application does not necessarily require that all of the advantages described above be achieved at the same time.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings to be used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present application, and other drawings can be obtained according to the drawings without any creative work for those skilled in the art.

FIG. 1 is a schematic flowchart diagram of an audio data processing method according to an embodiment of the present application;

2 is a schematic flowchart diagram of an audio data processing method according to another embodiment of the present application;

FIG. 3 is a schematic flowchart diagram of an audio data processing method according to another embodiment of the present disclosure;

4 is a schematic structural diagram of an audio data processing apparatus according to an embodiment of the present application;

FIG. 5 is a schematic structural diagram of an audio data processing apparatus according to another embodiment of the present disclosure;

FIG. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

FIG. 7 is a schematic structural diagram of an electronic device according to another embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the drawings in the embodiments of the present application. It is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

In order to solve the technical problem that the maintenance cost of the software is high, the audio data processing method, the device, the electronic device, and the storage medium are provided.

In the first aspect, the audio data processing method applied to the client is described in detail in the embodiment of the present application. The audio data processing method may include the following steps A1-A6:

A1, receiving a first implementation pointer and audio data sent by the operating system; wherein, if the currently adopted sound processing mode is a non-low delay ear return mode, the type of the first implementation pointer is the first type; if the currently used sound processing The mode is a low-latency ear-back mode, and the type of the first implementation pointer is the second type; the first implementation pointer is sent after the operating system collects the audio data.

Wherein, in an embodiment, the first type may be null NULL and the second type may be non-NULL.

In an embodiment, the client includes a second JAVA language layer module; receiving the first implementation pointer and the audio data sent by the operating system, the second JAVA language layer module receiving the audio data sent by the operating system and the first Implement pointers.

As an implementation manner of the embodiment of the present application, the step of receiving, by the second JAVA language layer module, the audio data sent by the operating system and the first implementation pointer includes: receiving, by the second JAVA language layer module, the audio sent by the first JAVA language layer module Data and first implementation pointer.

A2, determining the type of the first implementation pointer.

A3. If it is determined that the type of the first implementation pointer is the first type, the received audio data is subjected to sound processing to obtain non-low delay audio audio data.

In an embodiment, the client further includes a second C language layer module; if it is determined that the type of the first implementation pointer is the first type, performing sound processing on the received audio data to obtain non-low delay audio audio data. The step may include: if it is determined that the type of the first implementation pointer is the first type, the second C language layer module receives the audio data sent by the second JAVA language layer module; and the second C language layer module performs sound processing on the audio data to obtain Non-low latency audio audio data.

A4, after obtaining the non-low delay audio audio data, generating a second implementation pointer of the first type, and sending the second implementation pointer and the non-low delay audio audio data to the operating system, for triggering the operating system according to the second Implement pointers to play non-low latency audio data.

In an embodiment, the client further includes a second C language layer module; if it is determined that the type of the first implementation pointer is the second type, the audio data is invoked from the operating system by the first callback function, and the audio data is processed. The step of obtaining low-latency audio audio data includes: if it is determined that the type of the first implementation pointer is the second type, the second C language layer module receives the first implementation pointer sent by the second JAVA language layer module; the second C language layer The module calls the audio data through the first callback function, and performs audio processing on the audio data to obtain low-latency audio audio data.

A5. If it is determined that the type of the first implementation pointer is the second type, the audio data is called from the operating system by the first callback function, and the audio data is subjected to sound processing to obtain low-latency audio audio data.

In an embodiment, the operating system further includes a first C language layer module; the second C language layer module calls the audio data through the first callback function, and performs audio processing on the audio data to obtain low-latency audio audio data, including The second C language layer module calls the audio data collected by the first C language layer module from the first C language layer module through the first callback function, and performs audio processing on the audio data to obtain low delay audio audio data.

A6, after obtaining the low-latency audio audio data, generating a second implementation pointer of the second type, and sending the second implementation pointer and the low-latency audio audio data to the operating system, for triggering the operating system according to the second implementation pointer Plays low-latency audio data.

For the sake of clarity of the scheme, the audio data processing method provided by the client from the perspective of the embodiment of the present application will be described later with reference to FIG. For details of the steps of the audio data processing method provided by the embodiment of the present application, refer to the introduction of the corresponding steps in the corresponding embodiment of FIG. An audio data processing method provided by the client from the perspective of the embodiment of the present application is described below with reference to FIG.

1 is a schematic flowchart of an audio data processing method according to an embodiment of the present disclosure. The audio data processing method is applied to a client including a second JAVA language layer module and a second C language layer module, and the audio data processing method may be Including the following steps:

Step 101: The second JAVA language layer module receives the audio data sent by the operating system and the first implementation pointer; wherein, if the currently adopted sound effect processing mode is a non-low delay ear return mode, the type of the first implementation pointer is null NULL; If the currently used sound processing mode is the low delay ear return mode, the type of the first implementation pointer is non-NULL.

When using the ear return for mobile live broadcast, the lower the delay of the ear return, the better the live broadcast effect. At present, compared with the traditional non-low-latency ear-back mode, the audio data is often used in the process of acquisition, processing, and playback without the low-latency ear-back mode of the JAVA language layer to reduce the ear-back delay, but for the low support Terminal devices with delay characteristics, when the ear mode is implemented in the above mode, may cause noise or abnormal sound.

In the solution of the embodiment, the non-low-latency ear-back mode and the low-latency ear-back mode may be separately selected according to the actual needs of the user or the attributes of the terminal device to perform mobile live broadcast: on the terminal device supporting low delay characteristics. Using the low-latency ear-back mode, the non-low-latency ear-back mode is used on terminal devices that do not support low-latency characteristics, reducing software maintenance costs.

In this step, when the ear return mode is specified as the non-low delay ear return mode, the second JAVA language layer module of the client may receive the audio data sent by the operating system and the first implementation pointer of the type NULL; when the ear return mode is specified In the low latency ear return mode, the client's second JAVA language layer module can receive the audio data sent by the operating system and the first implementation pointer of type non-NULL. The audio data sent by the operating system may be an audio data collected by an operating system, and the collected audio data may be audio data in a PCM (Pulse Code Modulation) format, or may be The audio data of other formats, the embodiment of the present application does not limit the format of the audio data.

In an implementation, the operating system may include a first JAVA language layer module; at this time, the second JAVA language layer module receives the audio data sent by the operating system and the first implementation pointer, and may include: receiving, by the second JAVA language layer module The audio data sent by the first JAVA language layer module and the first implementation pointer.

Step 102: Determine the type of the first implementation pointer; if it is determined that the type of the first implementation pointer is NULL, step 103 is performed; if it is determined that the type of the first implementation pointer is non-NULL, step 104 is performed.

Step 103: The second JAVA language layer module sends the audio data to the second C language layer module.

Corresponding to step 101, the type of the first implementation pointer is NULL, that is, when the conventional non-low delay ear return mode is adopted, the client sends the received audio data from the second JAVA language layer module to the second C language layer. The module is configured to perform audio processing on the audio data by the second C language layer module.

Step 104: The second JAVA language layer module sends the first implementation pointer to the second C language layer module.

Corresponding to step 101, the type of the first implementation pointer is non-NULL, that is, when the conventional low-latency ear-back mode is adopted, the client sends the non-NULL first implementation pointer to the second C language layer module.

Step 105: After receiving the audio data, the second C language layer module performs audio processing on the audio data to obtain non-low delay audio audio data, and sends the non-low delay audio audio data to the second JAVA language layer module.

Corresponding to step 103, after receiving the audio data, the second C language layer module of the client may perform sound processing on the audio data to obtain non-low delay audio audio data. The sound processing method may include: mixing audio data, recording, volume gain, climax interception, male and female voice change, rhythm speed adjustment, sound fade processing, etc., and those skilled in the art may understand the specific implementation manner of the sound effect processing. I will not repeat them here.

Step 106: After receiving the first implementation pointer of the non-NULL type, the second C language layer module calls the audio data through the first callback function, performs audio processing on the audio data, obtains low-latency audio audio data, and has low delay. The audio audio data is sent to the second JAVA language layer module.

Corresponding to step 104, after receiving the first implementation pointer of the non-NULL type, the second C language layer module of the client can directly call the audio data collected by the operating system through the callback function, and perform sound effect processing, for convenience. Description, the callback function can be the first callback function. In this way, the problem of increased delay caused by the audio data passing through the second JAVA language layer module can be avoided, and the time taken for the audio data to be processed from the acquisition to the sound effect processing is reduced. Similarly, the audio processing method in this step may include: mixing audio data, recording, volume gain, climax interception, male and female voice change, rhythm speed adjustment, sound fade processing, etc., and those skilled in the art can understand the sound effect processing. The specific implementation manner is not described here.

In an embodiment, the operating system includes a first C language layer module, and after receiving the first implementation pointer that is not NULL, the second C language layer module calls the audio data through the first callback function to perform sound effects on the audio data. Processing, obtaining low-latency audio audio data, may include: after receiving the first implementation pointer that is not NULL, the second C language layer module invokes the first C language layer from the first C language layer module by using the first callback function The audio data collected by the module is subjected to sound processing of the audio data to obtain low-latency audio data.

Step 107: After receiving the non-low delay audio audio data, the second JAVA language layer module generates a second implementation pointer of type NULL, and sends the second implementation pointer and the received non-low delay audio audio data to the operating system. And for triggering the operating system to play the non-low delay audio audio data according to the second implementation pointer.

Step 108: After receiving the low-latency audio audio data, the second JAVA language layer module generates a second implementation pointer of a non-NULL type, and sends the second implementation pointer and the received low-latency audio audio data to the operating system. The triggering operating system plays the low-latency audio data according to the second implementation pointer.

In step 107 and step 108, the client sends the audio data processed by the sound effect to the operating system for the non-low delay ear return mode and the low delay ear return mode, respectively, so that the operating system plays the processed audio data.

In the audio data processing method provided by the embodiment of the present application, after the second JAVA language layer module of the client receives the audio data sent by the operating system and the first implementation pointer, determining the type of the first implementation pointer, when the type of the first implementation pointer When NULL, the second JAVA language layer module sends the audio data to the second C language layer module; when the first implementation pointer type is non-NULL, the second JAVA language layer module sends the first implementation pointer to the second C a language layer module; after receiving the audio data, the second C language layer module performs sound processing on the audio data to obtain non-low delay sound effect audio data, and sends the non-low delay sound effect audio data to the second JAVA language layer module; After receiving the first implementation pointer of the non-NULL type, the second C language layer module calls the audio data through the first callback function, performs audio processing on the audio data, obtains low-latency audio audio data, and transmits the low-latency audio data. To the second JAVA language layer module; the second JAVA language layer module generates a type after receiving non-low latency audio audio data a second implementation pointer of NULL, and sending the second implementation pointer and the received non-low delay audio audio data to the operating system, for triggering the operating system to play the non-low delay audio audio data according to the second implementation pointer; After receiving the low-latency audio data, the second JAVA language layer module generates a second implementation pointer of a non-NULL type, and sends the second implementation pointer and the received low-latency audio audio data to the operating system for triggering the operation. The system plays the low-latency audio audio data according to the second implementation pointer. The present application can obtain non-low-latency audio audio data and low-latency audio audio data by a set of methods, and play the above different audio data, thereby reducing the maintenance cost of the software.

In a second aspect, the audio data processing method applied to the operating system of the terminal is described in detail in the embodiment of the present application. The audio data processing method may include the following steps B1-B5:

B1, collecting audio data;

In an embodiment, the operating system includes a first C language layer module and a first JAVA language layer module; when the sound effect processing mode is a low delay ear return mode, the step of collecting audio data includes: a first C language layer module Acquire audio data. When the sound processing mode is a non-low delay ear return mode, the step of collecting audio data includes: acquiring, by the first JAVA language layer module, audio data.

B2, determining a currently used sound processing mode from the non-low delay ear return mode and the low delay ear return mode;

B3, according to the currently used sound processing mode, generating a first implementation pointer. When the sound processing mode is a non-low delay ear return mode, the first implementation pointer type is a first type, and the sound processing mode is a low delay ear return mode. At the time, the type of the first implementation pointer is the second type.

In an embodiment, when the sound processing mode is a low-latency ear-back mode, the step may include: after receiving the audio data collected by the first C language layer module, the first JAVA language layer module generates a second type The first implementation pointer of the type; when the sound processing mode is the non-low delay ear return mode, the step may include: after acquiring the audio data, the first JAVA language layer module generates a first implementation pointer of the first type.

B4, sending the first implementation pointer and the audio data to the client, so that if the client determines that the type of the first implementation pointer is the first type, obtaining audio data sent by the operating system, performing audio processing on the audio data, and obtaining non-low delay Audio audio data, if the client determines that the type of the first implementation pointer is the second type, the audio data is called by the first callback function, the audio data is processed by the sound effect, and the low-latency audio audio data is obtained; and the non-low-latency audio audio data is obtained. Or after low-latency audio data, the resulting data is fed back to the operating system.

In an embodiment, when the sound effect processing mode is the low delay ear return mode, the step may include: the first JAVA language layer module sends the audio data and the first implementation pointer of the second type to the client; When the sound processing mode is the low delay ear return mode, the step may include the first JAVA language layer module transmitting the audio data and the first implementation pointer of the first type to the client.

As an implementation of the embodiment of the present application, the client includes a second JAVA language layer module. The step of the first JAVA language layer module sending the audio data and the first implementation pointer of the second type to the client may include: The first JAVA language layer module sends the audio data and the first implementation pointer of the second type to the second JAVA language layer module.

As another implementation manner of the embodiment of the present application, the client includes a second JAVA language layer module; the first JAVA language layer module sends the audio data and the first implementation pointer of the first type to the client, which may include The first JAVA language layer module sends the audio data and the first implementation pointer of the first type to the second JAVA language layer module.

B5. Receive non-low delay audio audio data or low delay audio audio data sent by the client, and play the received audio data.

In an embodiment, when the sound effect processing mode is a low-latency ear-back mode, the step may include: the first JAVA language layer module receiving the low-latency sound effect audio data sent by the client and the second implementation of the second type being the second type a pointer, and transmitting a second implementation pointer of the second type to the first C language layer module; the first C language layer module plays the low delay audio audio data according to the second implementation pointer of the second type; when the sound effect When the processing mode is the non-low delay ear return mode, the step may include: the first JAVA language layer module receives the low delay sound effect audio data sent by the client and the second implementation pointer of the second type type, and the type is the second A second implementation pointer of the type is sent to the first C language layer module; the first C language layer module plays the non-low latency audio audio data according to the second implementation pointer of the first type.

For the sake of clarity, the audio data processing method provided by the operating system of the terminal in the embodiment of the present application will be described later with reference to FIG. 2 . For details of the steps of the audio data processing method provided by the embodiment of the present application, refer to the introduction of the corresponding steps in the corresponding embodiment of FIG. 2 . An audio data processing method provided by the operating system of the terminal in the embodiment of the present application is described below with reference to FIG.

2 is a schematic flowchart of an audio data processing method according to another embodiment of the present application. The audio data processing method is applied to an operating system including a first JAVA language layer module and a first C language layer module, and the audio data processing method is Can include:

Step 201: Determine a currently used sound effect processing mode from the non-low delay ear return mode and the low delay ear return mode.

In this step, the non-low delay ear return mode or the low delay ear return mode may be selected as the current sound processing mode for mobile live broadcast according to the actual needs of the user or the attributes of the terminal device: low delay is used on the terminal device supporting the low delay characteristic. The ear-back mode uses a non-low-latency ear-back mode on a terminal device that does not support low-latency characteristics.

Step 202: When the sound effect processing mode is the low delay ear return mode, the first C language layer module collects the audio data, and sends the audio data to the first JAVA language layer module.

The part of the operating system used to complete the audio data collection may be referred to as an audio collection unit. The manner in which the audio collection unit collects audio data is different for different sound processing modes. In this step, the sound processing mode is a low-latency ear-back mode. Correspondingly, the audio collecting unit performs the following steps to complete the audio data collection: first, obtain the nominal sampling rate and the atomic buffer size of the operating system, and then follow the name. Sampling rate, using OpenSL ES (a call interface in the operating system with audio data acquisition and playback) for audio data collection. The collected audio data may be audio data in the PCM format, or may be audio data in other formats, which is not limited in this embodiment of the present application.

Step 203: When the sound effect processing mode is a non-low delay ear return mode, the first JAVA language layer module collects audio data.

Corresponding to step 202, the sound effect processing mode is a non-low delay ear return mode, and the audio collecting unit can complete the collection of the audio data by first acquiring the nominal sampling rate and the buffer size of the operating system, according to the nominal sampling rate. Audio data acquisition is performed using Audio Record (a call interface in the operating system with audio data acquisition). The collected audio data may be audio data in the PCM format, or may be audio data in other formats, which is not limited in this embodiment of the present application.

Step 204: After receiving the audio data collected by the first C language layer module, the first JAVA language layer module generates a first implementation pointer of a non-NULL type, and sends the audio data and the first implementation pointer of the non-NULL type. To the client, used to trigger the client to perform low-latency sound processing on the audio data, obtain low-latency audio data, generate a non-NULL second implementation pointer, and generate low-latency audio data and a second implementation of non-NULL type The pointer returns to the first JAVA language layer module.

In the low-latency ear-back mode, after receiving the audio data collected by the first C language layer module, the first JAVA language layer module of the operating system may send the audio data and the first implementation pointer of the non-NULL type to the client. To enable the client to perform low-latency sound processing on the audio data. Similarly, the sound processing method in this step may include: mixing audio data, recording, volume gain, climax interception, male and female voice change, rhythm speed adjustment, The sound fade processing and the like can be understood, and those skilled in the art can understand the specific implementation of the sound processing, and details are not described herein.

In this step, the audio data is collected by using the first C language layer module of the operating system, so that the operating system can avoid the passage of the first JAVA language layer module when transmitting the audio data to the client for sound processing, thereby shortening the audio data transmission. The length of time required for the process.

Step 205: After acquiring the audio data, the first JAVA language layer module generates a first implementation pointer of type NULL, and sends the audio data and the first implementation pointer of type NULL to the client, and is used to trigger the client pair. The audio data is subjected to non-low delay sound processing to obtain non-low delay sound effect audio data, generating a second implementation pointer of type NULL, and returning non-low delay sound effect audio data and a second implementation pointer of type NULL to the first JAVA language Layer module.

In the non-low delay ear return mode, the operating system can send the audio data collected by the first JAVA language layer module and the first implementation pointer of the type NULL to the client, so that the client performs non-low delay sound on the audio data. Processing, in the same manner, the audio processing method in this step may include: mixing audio data, recording, volume gain, climax interception, male and female voice change, rhythm speed adjustment, sound fade processing, etc., and those skilled in the art can understand the sound effect. The specific implementation of the processing is not described here.

Step 206: After receiving the low-latency audio audio data sent by the client and the second implementation pointer of the non-NULL type, the first JAVA language layer module sends the second implementation pointer of the non-NULL type to the first C language layer. Module.

In the low-latency ear-back mode, the first JAVA language layer module of the operating system may send the second implementation pointer of the non-NULL type sent by the client to the first C language layer module.

Step 207: The first JAVA language layer module sends the second implementation pointer of type NULL and the non-low delay audio audio data after receiving the non-low delay audio audio data sent by the client and the second implementation pointer of type NULL. To the first C language layer module.

In the non-low delay ear return mode, the first JAVA language layer module of the operating system may send the non-low delay sound effect audio data sent by the client and the second implementation pointer of the type NULL to the first C language layer module.

Step 208: After receiving the second implementation pointer of the non-NULL type, the first C language layer module plays the low-latency audio audio data according to the second implementation pointer of the non-NULL type.

Step 209: After receiving the second implementation pointer of the type NULL and the non-low delay audio audio data, the first C language layer module plays the non-low delay audio audio data according to the second implementation pointer of the type NULL.

In

steps

208 and 209, the first C language layer module of the operating system may use OpenSL ES for non-low-latency audio data or low-latency audio data according to a second implementation pointer of type NULL or non-NULL. Play.

In the audio data processing method shown in FIG. 2 provided by the embodiment of the present application, the currently adopted sound effect processing mode is first determined. When the sound processing mode is the low delay ear return mode, the first C language layer module collects the audio data, and sends the audio data to the first JAVA language layer module; when the sound effect processing mode is the non-low delay ear return mode, the first The JAVA language layer module collects the audio data; after receiving the audio data collected by the first C language layer module, the first JAVA language layer module generates a first implementation pointer of a non-NULL type, and the audio data and the type are non-NULL. The first implementation pointer is sent to the client, and is used to trigger the client to perform low-latency sound processing on the audio data, obtain low-latency audio audio data, generate a second implementation pointer of a non-NULL type, and generate low-latency audio data and non- The second implementation pointer of NULL is returned to the first JAVA language layer module; after acquiring the audio data, the first JAVA language layer module generates a first implementation pointer of type NULL, and firstly sets the audio data and the type to NULL. The implementation pointer is sent to the client, which is used to trigger the client to perform non-low delay sound processing on the audio data, and obtain non-low delay sound effect audio data. Forming a second implementation pointer of type NULL, and returning the non-low latency audio audio data and the second implementation pointer of type NULL to the first JAVA language layer module; the first JAVA language layer module receives the low sent by the client After delaying the audio audio data and the second implementation pointer of the non-NULL type, sending the second implementation pointer of the non-NULL type to the first C language layer module; the first JAVA language layer module receiving the non-lower sent by the client After delaying the audio audio data and the second implementation pointer of type NULL, transmitting the second implementation pointer of type NULL and the non-low delay audio audio data to the first C language layer module; the first C language layer module receives the type After the non-NULL second implementation pointer, the low-latency audio data is played according to the second implementation pointer of type non-NULL; the first C language layer module receives the second implementation pointer of type NULL and the non-low delay sound effect. After the audio data, non-low-latency audio data is played according to the second implementation pointer of type NULL. The present application can obtain non-low-latency audio audio data and low-latency audio audio data by a set of methods, and play the above different audio data, thereby reducing the maintenance cost of the software.

The audio data processing method, device, electronic device and storage medium provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings. As shown in FIG. 3, an audio data processing method provided by an embodiment of the present application is applied to a user terminal, and an operating system including a first JAVA language layer module and a first C language layer module is run on the user terminal. The user terminal further includes a client that includes a second JAVA language layer module and a second C language layer module. The audio data processing method may specifically include the following steps:

Step 301: Determine a currently adopted sound effect processing mode from the non-low delay ear return mode and the low delay ear return mode. If the currently used sound processing mode is the low delay ear return mode, the process proceeds to step 302; if the currently used sound effect processing mode is the non-low delay ear return mode, the process proceeds to step 309.

In this embodiment, normal non-low delay audio data and low delay audio data can be obtained by a set of methods, and if the currently used sound processing mode is a non-low delay ear return mode, non-low delay sound can be obtained. Audio data; if the currently used sound processing mode is the low-latency ear-back mode, low-latency audio data can be obtained.

Step 302: When the sound effect processing mode is the low delay ear return mode, the first C language layer module of the operating system collects the audio data, and synchronizes the audio data to the first JAVA language layer module of the operating system.

The part of the operating system used to complete the audio data collection may be referred to as an audio collection unit. The manner in which the audio collection unit collects audio data is different for different sound processing modes. In this step, the sound processing mode is a low-latency ear-back mode. Accordingly, the audio collecting unit can complete the collection of the audio data by first obtaining the nominal sampling rate and the atomic buffer size of the operating system, and then according to the name. Sampling rate, using OpenSL ES (a call interface in the operating system with audio data acquisition and playback) for audio data collection. The collected audio data may be audio data in the PCM format, or may be audio data in other formats, which is not limited in this embodiment of the present application.

Step 303: After receiving the audio data collected by the first C language layer module, the first JAVA language layer module generates a first implementation pointer of a non-NULL type, and sends the audio data and the first implementation pointer of the non-NULL type. The second JAVA language layer module to the client.

Step 304: After receiving the audio data and the first implementation pointer of the non-NULL type, the second JAVA language layer module sends the implementation pointer of the non-NULL type to the second C language layer module.

Step 305: After receiving the first implementation pointer of the non-NULL type, the second C language layer module calls the audio data through the first callback function, performs audio processing on the audio data, obtains low-latency audio audio data, and has low delay. The audio audio data is sent to the second JAVA language layer module.

In this step, after receiving the non-NULL first implementation pointer, the second C language layer module adopts the first callback function, and adopts the interface of the first callback function set in the first C language layer module of the operating system. Directly calling the audio data collected by the first C language layer module of the operating system, and performing audio processing on the audio data according to the user's requirements, and obtaining low-latency audio audio data, and the specific sound processing method may include: mixing audio data. The recording, the volume gain, the climax interception, the male and female voice change, the rhythm speed adjustment, the sound fade processing, and the like, and the embodiment of the present application does not limit this. Here, the part of the client for completing the sound effect processing may be referred to as a sound effect processing unit, and the sound effect processing unit may be a single one for completing one of the above sound effect processing modes; meanwhile, the sound effect processing unit may also There are a plurality of, for example, a first sound processing unit, a second sound processing unit, a third sound processing unit, and the like, wherein each of the sound processing units is used to complete one of the sound processing methods. When there are multiple sound processing units, the sound effects generated by the plurality of sound processing units may be superimposed on each other, that is, the output result of the first sound processing unit may be used as an input of the second sound processing unit; The output of the unit, as an input to the third sound processing unit, and so on, results in the final low-latency audio audio data.

Step 306: After receiving the low-latency audio audio data, the second JAVA language layer module generates a second implementation pointer of a non-NULL type, and sends the second implementation pointer and the received low-latency audio audio data to the first JAVA. Language layer module.

Step 307: After receiving the low-latency audio audio data sent by the second JAVA language layer module and the second implementation pointer of the non-NULL type, the first JAVA language layer module sends the second implementation pointer of the non-NULL type to the first A C language layer module.

Step 308, after receiving the second implementation pointer of the non-NULL type, the first C language layer module plays the low delay audio audio data according to the second implementation pointer of the non-NULL type.

The part of the operating system for completing the audio data playing may be referred to as an audio playing unit. In this step, the second implementation pointer type is non-NULL, and the audio playing module may complete the playing of the audio data by the following steps: After the second implementation pointer of the non-NULL type, the second callback function is used to directly call the low-latency audio audio data processed by the audio processing unit, and the low-latency audio audio data is played by using OpenSL ES.

Step 309, when the sound effect processing mode is the non-low delay ear return mode, the first JAVA language layer module of the operating system collects the audio data.

In step 302, the portion of the operating system used to complete the audio data collection may be referred to as an audio collection unit. The manner in which the audio collection unit collects audio data is different for different audio processing modes. In this step, the sound processing mode is a non-low delay ear return mode. Correspondingly, the audio collecting unit can complete the collection of the audio data by first obtaining the nominal sampling rate and the buffer size of the operating system, and sampling according to the nominal Rate, using Audio Record (a call interface in the operating system with audio data acquisition) for audio data collection.

The collected audio data may be audio data in the PCM format, or may be audio data in other formats, which is not limited in this embodiment of the present application.

Step 3010: After acquiring the audio data, the first JAVA language layer module generates a first implementation pointer of type NULL, and sends the audio data and the first implementation pointer of the type NULL to the second JAVA language layer module of the client. .

Step 3011: After receiving the audio data and the first implementation pointer of the type NULL, the second JAVA language layer module sends the audio data to the second C language layer module.

Step 3012: After receiving the audio data, the second C language layer module performs audio processing on the audio data to obtain non-low delay audio audio data, and sends the non-low delay audio audio data to the second JAVA language layer module.

In this step, when the first implementation pointer type is NULL, after receiving the audio data, the second C language layer module performs sound processing on the audio data according to the user's requirements to obtain non-low delay audio audio data. Specific sound processing methods may include: mixing audio data, recording, volume gain, climax interception, male and female voice change, rhythm speed adjustment, sound fade processing, etc., without limitation. The same as step 305, this step can also be performed by the sound effect processing unit, and the sound effect processing unit can be a single one for completing one of the above-mentioned sound effect processing modes; at the same time, the sound effect processing unit can also have multiple, such as: It can be named as the first sound processing unit, the second sound processing unit, the third sound processing unit, and the like, wherein each of the sound processing units is used to complete one of the above sound processing methods. When there are multiple sound processing units, the sound effects generated by the plurality of sound processing units may be superimposed on each other, that is, the output result of the first sound processing unit may be used as an input of the second sound processing unit; The output result of the unit, as an input of the third sound processing unit, and so on, obtains the final non-low delay sound effect audio data.

The second C language layer module can send the non-low delay sound effect audio data obtained by the sound processing to the second JAVA language layer module through a local interface (JNI) call.

Step 3013: After receiving the non-low delay audio audio data, the second JAVA language layer module generates a second implementation pointer of type NULL, and sends the second implementation pointer and the received non-low delay audio audio data to the operating system. The first JAVA language layer module.

Step 3014: After receiving the non-low delay audio audio data sent by the client and the second implementation pointer of type NULL, the first JAVA language layer module sends the second implementation pointer of type NULL and the non-low delay audio audio data. To the first C language layer module.

Step 3015: After receiving the second implementation pointer of type NULL and the non-low delay audio audio data, the first C language layer module plays the non-low delay audio audio data according to the second implementation pointer of type NULL.

In step 308, the portion of the operating system for completing audio data playback may be referred to as an audio playback unit. In this step, the second implementation pointer type is NULL, and the audio playback unit may perform audio data playback by the following steps. : After receiving the second implementation pointer of non-NULL type and non-low latency audio audio data, the non-low latency audio audio data is played using OpenSL ES.

In the embodiment of the present application, the client receives the audio data sent by the operating system and the first implementation pointer. When the first implementation pointer type is NULL, the audio data is processed by the audio data to obtain non-low delay audio audio data; When the pointer type is non-NULL, the audio data is processed to obtain low-latency audio data, and the client sends the non-low-latency audio data or the low-delay audio data and the second implementation pointer to the operating system, and the operating system according to the operating system The second implementation pointer plays the received non-low delay audio audio data or low delay audio audio data. The present application can obtain non-low-latency audio audio data and low-latency audio audio data by a set of methods, and play the above different audio data, thereby reducing the maintenance cost of the software.

In a third aspect, the embodiment of the present application provides an audio data processing apparatus applied to a client, where the audio data processing apparatus may include:

a data receiving module, configured to receive a first implementation pointer and audio data sent by the operating system; wherein, if the currently adopted sound processing mode is a non-low delay ear return mode, the type of the first implementation pointer is a first type; If the currently used sound processing mode is a low-latency ear-back mode, the type of the first implementation pointer is a second type; the first implementation pointer is sent after the operating system collects audio data;

a pointer type determining module, configured to determine a type of the first implementation pointer;

The first data processing module is configured to: if it is determined that the type of the first implementation pointer is the first type, performing sound effect processing on the received audio data to obtain non-low delay audio effect audio data;

a first data sending module, configured to generate a second implementation pointer of a first type after the non-low delay audio audio data is obtained, and send the second implementation pointer and the non-low delay audio audio data Up to the operating system, configured to trigger the operating system to play the non-low delay audio audio data according to the second implementation pointer;

a second data processing module, configured to: if it is determined that the type of the first implementation pointer is a second type, invoke the audio data from the operating system by using a first callback function, and perform sound processing on the audio data to obtain Low latency audio audio data;

a second data sending module, configured to generate a second implementation pointer of a second type after the low-latency audio audio data is obtained, and send the second implementation pointer and the low-latency audio audio data to the The operating system is configured to trigger the operating system to play the low-latency audio audio data according to the second implementation pointer.

In an embodiment, the client includes a second JAVA language layer module, and the data receiving module is configured to: trigger the second JAVA language layer module to receive the audio data sent by the operating system and the first implementation pointer.

In an embodiment, the operating system includes a first JAVA language layer module, and the first data sending module is configured to: trigger the second JAVA language layer module to receive the first JAVA language layer module. The transmitted audio data and the first implementation pointer.

In an embodiment, the client further includes a second C language layer module; the first data processing module includes: a first data receiving unit, configured to determine that the type of the first implementation pointer is a type, the second C language layer module is triggered to receive the audio data sent by the second JAVA language layer module; the first data processing unit is configured to trigger the second C language layer module to the audio data Performing sound processing to obtain non-low-latency audio audio data; the first data processing module further includes: a second data receiving unit, configured to trigger the first if the type of the first implementation pointer is determined to be the second type The second C language layer module receives the first implementation pointer sent by the second JAVA language layer module, and the second data processing unit is configured to trigger the second C language layer module to invoke the audio data by using a first callback function. Performing sound processing on the audio data to obtain low-latency audio audio data.

In an embodiment, the operating system further includes a first C language layer module, and the second C language layer module is configured to: invoke the first C language layer module from the first C language layer module by using a first callback function The audio data collected by the first C language layer module performs sound processing on the audio data to obtain low-latency audio audio data.

In one embodiment, the first type is null NULL and the second type is non-NULL.

An audio data processing method is applied to a client that includes a second JAVA language layer module and a second C language layer module according to the above embodiments of the present application. Accordingly, an embodiment of the present application further provides an audio data processing apparatus. The client is located in the second JAVA language layer module and the second C language layer module. The schematic diagram of the client is as shown in FIG. 4, and includes: a second JAVA language layer module 401 and a second C language layer module 402.

The second JAVA language layer module 401 is configured to receive the audio data sent by the operating system and the first implementation pointer; wherein, if the currently adopted sound effect processing mode is a non-low delay ear return mode, the type of the first implementation pointer is null NULL If the currently used sound processing mode is a low-latency ear-back mode, the type of the first implementation pointer is non-NULL; and the type of the first implementation pointer is determined; if the type of the first implementation pointer is determined to be NULL, Sending the audio data to the second C language layer module 402; if it is determined that the type of the first implementation pointer is non-NULL, the first implementation pointer is sent to the second C language layer module 402;

The second C language layer module 402 is configured to perform audio processing on the audio data after receiving the audio data, to obtain non-low delay audio audio data, and send the non-low delay audio audio data to the second JAVA language layer module 401; And, after receiving the first implementation pointer of the non-NULL type, calling the audio data through the first callback function, performing audio processing on the audio data, obtaining low-latency audio audio data, and transmitting the low-latency audio audio data to the second JAVA language layer module 401;

The second JAVA language layer module 401 is further configured to generate a second implementation pointer that is NULL after receiving the non-low delay audio audio data, and send the second implementation pointer and the received non-low delay audio audio data to the operation. a system for triggering an operating system to play non-low-latency audio data according to a second implementation pointer; and, after receiving low-latency audio audio data, generating a second implementation pointer of a non-NULL type, and implementing the second implementation The pointer and the received low-latency audio audio data are sent to the operating system for triggering the operating system to play the low-latency audio audio data according to the second implementation pointer.

Further, the operating system includes a first JAVA language layer module; the second JAVA language layer module 401 is configured to receive the audio data and the first implementation pointer sent by the first JAVA language layer module.

Further, the operating system includes a first C language layer module, and the second C language layer module 402 is specifically configured to: after receiving the first implementation pointer of the non-NULL type, pass the first callback function from the first C The language layer module calls the audio data collected by the first C language layer module, and performs audio processing on the audio data to obtain low-latency audio audio data. The functions of the above modules may correspond to the corresponding processing steps in the processes shown in FIG. 1 and FIG. 3, and details are not described herein again.

In the embodiment of the present application, the second JAVA language layer module 401 receives the audio data sent by the operating system and the first implementation pointer. When the first implementation pointer type is NULL, the audio data is sent to the second C language layer module 402. When the first implementation pointer type is non-NULL, the first implementation pointer is sent to the second C language layer module 402; after receiving the audio data, the second C language layer module 402 performs audio processing on the audio data to obtain a non- Low-latency audio audio data, and transmitting non-low-latency audio audio data to the second JAVA language layer module 401; and, after receiving the first implementation pointer of the non-NULL type, calling the audio data through the first callback function, The audio data is subjected to sound processing to obtain low-latency audio audio data, and the low-latency audio audio data is sent to the second JAVA language layer module 401; finally, after receiving the non-low-latency audio audio data, the second JAVA language layer module 401 Generating a second implementation pointer of type NULL and sending the second implementation pointer and the received non-low latency audio audio data to the operating system The triggering operating system plays the non-low delay audio audio data according to the second implementation pointer; after receiving the low delay audio audio data, generates a second implementation pointer of a non-NULL type, and the second implementation pointer and the receiving The low-latency audio audio data that is sent to the operating system is used to trigger the operating system to play the low-latency audio audio data according to the second implementation pointer. In this embodiment, non-low-latency audio audio data and low-latency audio audio data can be respectively obtained through a set of devices, and the different audio data is played, thereby reducing the maintenance cost of the software.

In a fourth aspect, an embodiment of the present application provides an audio data processing apparatus, which is applied to an operating system of a terminal, where the apparatus includes:

a data acquisition module configured to collect audio data;

a mode determining module configured to determine a currently used sound effect processing mode from the non-low delay ear return mode and the low delay ear return mode;

a pointer generating module, configured to generate a first implementation pointer according to the currently adopted sound processing mode, and when the sound processing mode is a non-low delay earback mode, the first implementation pointer type is a first type, when the sound effect When the processing mode is the low delay ear return mode, the type of the first implementation pointer is the second type;

a data sending module, configured to send the first implementation pointer and the audio data to a client, so that if the client determines that the type of the first implementation pointer is a first type, obtaining, sent by the operating system The audio data is subjected to sound processing on the audio data to obtain non-low delay sound effect audio data. If the client determines that the type of the first implementation pointer is the second type, the audio is called by the first callback function. Data, performing audio processing on the audio data to obtain low-latency audio audio data; after obtaining the non-low-latency audio audio data or the low-latency audio audio data, feeding the obtained data to the operating system;

The data receiving module is configured to receive the non-low delay sound effect audio data or the low delay sound effect audio data sent by the client, and play the received audio data.

In an embodiment, the operating system includes a first C language layer module and a first JAVA language layer module; when the sound effect processing mode is a low delay ear return mode, the data acquisition module is configured to: trigger The first C language layer module collects audio data; the pointer generating module is configured to: trigger the first JAVA language layer module to generate the audio data after receiving the audio data collected by the first C language layer module, and generate a type a first implementation pointer of the second type; the data sending module is configured to: trigger the first JAVA language layer module to send the audio data and the first implementation pointer of the second type to the client; The data receiving module is configured to: trigger the first JAVA language layer module to receive the low-latency sound effect audio data sent by the client, and a second implementation pointer of a second type, and the type is Sending a second implementation pointer of the second type to the first C language layer module; triggering the first C language layer module to play the second implementation pointer according to the type Low-latency sound audio data.

In an embodiment, the operating system includes a first C language layer module and a first JAVA language layer module; when the sound effect processing mode is a non-low delay ear return mode, the data collection module is configured to: Triggering the first JAVA language layer module to collect audio data; the pointer generating module is configured to: trigger the first JAVA language layer module to generate a first implementation pointer of a first type after acquiring audio data; The data sending module is configured to: trigger the first JAVA language layer module to send the audio data and the first implementation pointer of the type to the client; the data receiving module is configured to: Trimming the first JAVA language layer module to receive the low-latency sound effect audio data sent by the client and a second implementation pointer of a second type, and send the second implementation pointer of the second type Up to the first C language layer module; triggering the first C language layer module to play the non-low delay sound effect audio according to the second implementation pointer of the type being the first type It is.

In an embodiment, the client includes a second JAVA language layer module; the data sending module is configured to: the first JAVA language layer module sets the audio data and the type to a second type The first implementation pointer is sent to the second JAVA language layer module.

In an embodiment, the client includes a second JAVA language layer module; the data sending module is configured to: the first JAVA language layer module sets the audio data and the type to a first type The first implementation pointer is sent to the second JAVA language layer module.

The audio data processing method applied to the terminal operating system including the first C language layer module and the first JAVA language layer module is provided according to the above embodiment of the present application. Accordingly, an embodiment of the present application further provides an audio data processing. The device is located in the operating system of the terminal, and its structure is shown in FIG. 5, including:

The mode determination module 501, the first JAVA language layer module 502, and the first C language layer module 503.

The mode determining module 501 is configured to determine a currently adopted sound effect processing mode from the non-low delay ear return mode and the low delay ear return mode;

The first C language layer module 503 is configured to: when the sound effect processing mode is the low delay ear return mode, collect audio data, and send the audio data to the first JAVA language layer module;

The first JAVA language layer module 502 is configured to collect audio data when the sound processing mode is a non-low delay ear return mode; and after receiving the audio data collected by a C language layer module, generate a first implementation of a non-NULL type Pointer, and send the audio data and the first implementation pointer of the non-NULL type to the client, to trigger the client to perform low-latency sound processing on the audio data, obtain low-latency audio data, and generate a second type of non-NULL Implementing a pointer, and returning low-latency audio data and a second implementation pointer of a non-NULL type to the first JAVA language layer module; after acquiring the audio data, generating a first implementation pointer of type NULL, and the audio data And transmitting a first implementation pointer of the type NULL to the client, for triggering the client to perform non-low delay sound processing on the audio data, obtaining non-low delay audio audio data, generating a second implementation pointer of type NULL, and Non-low latency audio audio data and a second implementation pointer of type NULL are returned to the first JAVA language layer module; and upon receipt of the client After sending the low-latency audio audio data and the second implementation pointer of the non-NULL type, the second implementation pointer of the non-NULL type is sent to the first C language layer module; after receiving the non-low delay audio audio sent by the client After the data and the second implementation pointer of type NULL, the second implementation pointer of type NULL and the non-low delay audio audio data are sent to the first C language layer module;

The first C language layer module 503 is further configured to: after receiving the second implementation pointer of the non-NULL type, play the low-latency audio audio data according to the second implementation pointer of the non-NULL type; and receive the type NULL. After the second implementation of the pointer and the non-low delay audio data, the non-low latency audio audio data is played according to the second implementation pointer of type NULL.

Further, the client includes a second JAVA language layer module and a second C language layer module; the first JAVA language layer module 502 is configured to generate a non-NULL type after receiving the audio data collected by the C language layer module 503. The first implementation pointer, and the audio data and the first implementation pointer of type non-NULL are sent to the second JAVA language layer module.

Further, the first JAVA language layer module 502 is configured to generate a first implementation pointer of type NULL after the audio data is collected, and send the audio data and the first implementation pointer of the type NULL to the second JAVA language. Layer module. The functions of the above modules may correspond to the corresponding processing steps in the processes shown in FIG. 2 and FIG. 3, and details are not described herein again.

In the embodiment of the present application, the mode determining module 501 first determines the currently adopted sound effect processing mode. When the sound effect processing mode is the low delay ear return mode, the first C language layer module 503 is used to collect audio data, and the audio data is collected. Sending to the first JAVA language layer module 502; when the sound effect processing mode is the non-low delay ear return mode, the first JAVA language layer module 502 collects audio data; the first JAVA language layer module 502 receives a C language layer module to collect After the audio data, the first implementation pointer of the non-NULL type is generated, and the audio data and the first implementation pointer of the non-NULL type are sent to the client, which is used to trigger the client to perform low-latency sound processing on the audio data, Low-latency audio data, generating a second implementation pointer of type non-NULL, and returning low-latency audio data and a second implementation pointer of non-NULL type to the first JAVA language layer module; after acquiring the audio data, Generating a first implementation pointer of type NULL and sending the audio data and the first implementation pointer of type NULL to the client for The client performs non-low delay sound processing on the audio data, obtains non-low delay sound audio data, generates a second implementation pointer of type NULL, and returns non-low delay sound audio data and a second implementation pointer of type NULL to a first JAVA language layer module; and after receiving the low-latency audio audio data sent by the client and the second implementation pointer of the non-NULL type, sending the second implementation pointer of the non-NULL type to the first C language layer module Transmitting the second implementation pointer of the type NULL and the non-low delay audio audio data to the first C language layer module after receiving the non-low delay audio audio data sent by the client and the second implementation pointer of the type NULL; Finally, after receiving the second implementation pointer of the non-NULL type, the first C language layer module 503 plays the low-latency audio audio data according to the second implementation pointer of the non-NULL type; and receives the second type of the type NULL. After the pointer and non-low latency audio data are implemented, the non-low latency audio data is played according to the second implementation pointer of type NULL. In this embodiment, non-low-latency audio audio data and low-latency audio audio data can be respectively obtained through a set of devices, and the different audio data is played, thereby reducing the maintenance cost of the software.

The audio data processing method applied to the client including the second JAVA language layer module and the second C language layer module is provided according to the above embodiment of the present application. Accordingly, the embodiment of the present application further provides an electronic device, as shown in FIG. 6 . Included, including processor 601 and memory 602,

a memory 602, configured to store a computer program;

The processor 601 is configured to implement the audio data processing method applied to the client according to the first aspect of the present application when the program stored in the memory 602 is executed.

In the electronic device provided by the embodiment of the present application, the second JAVA language layer module of the client receives the audio data sent by the operating system and the first implementation pointer. When the first implementation pointer type is NULL, the second JAVA language layer module will audio. The data is sent to the second C language layer module; when the first implementation pointer type is non-NULL, the second JAVA language layer module sends the first implementation pointer to the second C language layer module; the second C language layer module receives After the audio data, audio processing is performed on the audio data to obtain non-low delay audio audio data, and the non-low delay audio audio data is sent to the second JAVA language layer module; the second C language layer module receives the non-NULL type. After the first implementation of the pointer, the audio data is called by the first callback function, the audio data is subjected to sound processing, the low-latency audio audio data is obtained, and the low-latency audio audio data is sent to the second JAVA language layer module; the second JAVA language layer After receiving the non-low delay audio audio data, the module generates a second implementation pointer of type NULL, and the second implementation pointer and The received non-low delay audio audio data is sent to the operating system for triggering the operating system to play the non-low delay audio audio data according to the second implementation pointer; the second JAVA language layer module after receiving the low delay audio audio data Generating a second implementation pointer of a non-NULL type, and transmitting the second implementation pointer and the received low-latency audio audio data to the operating system, for triggering the operating system to perform low-latency audio audio data according to the second implementation pointer Play. In this embodiment, non-low delay audio audio data and low delay audio audio data can be respectively obtained through a set of electronic devices, and the different audio data is played, which reduces the maintenance cost of the software.

The memory mentioned in the above electronic device may include a random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. Further, the memory may also be at least one storage device located away from the aforementioned processor.

The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (Network Processor, NP for short), or a digital signal processor (DSP). , Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component.

The audio data processing method applied to the terminal operating system including the first C language layer module and the first JAVA language layer module is provided according to the above embodiment of the present application. Accordingly, the embodiment of the present application further provides an electronic device, such as As shown in FIG. 7, the processor 701 and the memory 702 are included.

a memory 702, configured to store a computer program;

The processor 701 is configured to execute the program stored in the memory 702, and implement the audio data applied to the terminal operating system including the first C language layer module and the first JAVA language layer module according to the second aspect of the present application. Approach.

In the electronic device provided by the embodiment of the present application, the currently adopted sound effect processing mode is first determined. When the sound processing mode is the low delay ear return mode, the first C language layer module collects the audio data, and sends the audio data to the first JAVA language layer module; when the sound effect processing mode is the non-low delay ear return mode, the first The JAVA language layer module collects the audio data; after receiving the audio data collected by the first C language layer module, the first JAVA language layer module generates a first implementation pointer of a non-NULL type, and the audio data and the type are non-NULL. The first implementation pointer is sent to the client, and is used to trigger the client to perform low-latency sound processing on the audio data, obtain low-latency audio audio data, generate a second implementation pointer of a non-NULL type, and generate low-delay audio data and types. Returning the non-NULL second implementation pointer to the first JAVA language layer module; after acquiring the audio data, the first JAVA language layer module generates a first implementation pointer of type NULL, and the audio data and the type are null NULL The first implementation pointer is sent to the client, and is used to trigger the client to perform non-low delay sound processing on the audio data to obtain a non-low delay audio effect number. According to the second implementation pointer of type NULL, the non-low delay audio audio data and the second implementation pointer of type NULL are returned to the first JAVA language layer module; the first JAVA language layer module is sent by the receiving client After the low latency audio audio data and the second implementation pointer of the non-NULL type, the second implementation pointer of the non-NULL type is sent to the first C language layer module; the first JAVA language layer module receives the sending by the client After the non-low-latency audio audio data and the second implementation pointer of type NULL, the second implementation pointer of type NULL and the non-low delay audio audio data are sent to the first C language layer module; the first C language layer module is receiving After the second implementation pointer of the non-NULL type, the low-definition audio audio data is played according to the second implementation pointer of the non-NULL type; the first C language layer module receives the second implementation pointer of the type NULL and is non-low After delaying the audio data, non-low-latency audio data is played according to the second implementation pointer of type NULL. In this embodiment, non-low delay audio audio data and low delay audio audio data can be respectively obtained through a set of electronic devices, and the different audio data is played, which reduces the maintenance cost of the software.

In still another embodiment provided by the present application, there is provided a computer readable storage medium having stored therein instructions that, when run on a computer, cause the computer to perform any of the above embodiments The above is applied to an audio data processing method of a client including a second JAVA language layer module and a second C language layer module.

In the computer readable storage medium provided by the embodiment of the present application, the second JAVA language layer module of the client receives the audio data sent by the operating system and the first implementation pointer. When the type of the first implementation pointer is NULL, the second JAVA language The layer module sends the audio data to the second C language layer module; when the type of the first implementation pointer is non-NULL, the second JAVA language layer module sends the first implementation pointer to the second C language layer module; the second C language After receiving the audio data, the layer module performs audio processing on the audio data to obtain non-low delay audio audio data, and sends the non-low delay audio audio data to the second JAVA language layer module; the second C language layer module receives the After the first implementation pointer of the non-NULL type, the audio data is called by the first callback function, the audio data is subjected to sound processing, the low-latency audio audio data is obtained, and the low-latency audio audio data is sent to the second JAVA language layer module; The second JAVA language layer module generates a second implementation pointer of type NULL after receiving non-low latency audio audio data, and The second implementation pointer and the received non-low delay audio audio data are sent to the operating system for triggering the operating system to play the non-low delay audio audio data according to the second implementation pointer; the second JAVA language layer module receives the low delay After the audio audio data is generated, a second implementation pointer of a non-NULL type is generated, and the second implementation pointer and the received low-latency audio audio data are sent to the operating system for triggering the operating system according to the second implementation pointer, and the low delay Audio audio data is played. In this embodiment, non-low-latency audio audio data and low-latency audio audio data can be respectively obtained through a set of computer readable storage media, and the different audio data is played, which reduces the maintenance cost of the software.

In still another embodiment provided by the present application, there is provided a computer readable storage medium having stored therein instructions that, when run on a computer, cause the computer to perform any of the above embodiments The above is applied to an audio data processing method of a terminal operating system including a first C language layer module and a first JAVA language layer module.

In the computer readable storage medium provided by the embodiment of the present application, the currently adopted sound effect processing mode is first determined. When the sound processing mode is the low delay ear return mode, the first C language layer module collects the audio data, and sends the audio data to the first JAVA language layer module; when the sound effect processing mode is the non-low delay ear return mode, the first The JAVA language layer module collects the audio data; after receiving the audio data collected by the first C language layer module, the first JAVA language layer module generates a first implementation pointer of a non-NULL type, and the audio data and the type are non-NULL. The first implementation pointer is sent to the client, and is set to trigger the client to perform low-latency sound processing on the audio data, obtain low-latency audio audio data, generate a second implementation pointer of a non-NULL type, and generate low-latency audio data and types. Returning the non-NULL second implementation pointer to the first JAVA language layer module; after acquiring the audio data, the first JAVA language layer module generates a first implementation pointer of type NULL, and the audio data and the type are null NULL The first implementation pointer is sent to the client, and is used to trigger the client to perform non-low delay sound processing on the audio data to obtain non-low delay audio audio. Data, generating a second implementation pointer of type NULL, and returning non-low latency audio audio data and a second implementation pointer of type NULL to the first JAVA language layer module; the first JAVA language layer module is sent at the receiving client After the low-latency audio audio data and the non-NULL second implementation pointer, the second implementation pointer of the non-NULL type is sent to the first C language layer module; the first JAVA language layer module receives the non-lower sent by the client. After delaying the audio audio data and the second implementation pointer of type NULL, transmitting the second implementation pointer of type NULL and the non-low delay audio audio data to the first C language layer module; the first C language layer module receives the type After the non-NULL second implementation pointer, the low-latency audio data is played according to the second implementation pointer of type non-NULL; the first C language layer module receives the second implementation pointer of type NULL and the non-low delay sound effect. After the audio data, non-low-latency audio data is played according to the second implementation pointer of type NULL. In this embodiment, non-low-latency audio audio data and low-latency audio audio data can be respectively obtained through a set of computer readable storage media, and the different audio data is played, which reduces the maintenance cost of the software.

In still another embodiment provided by the present application, there is also provided a computer program product comprising instructions, when executed on a computer, causing the computer to perform any of the above embodiments to be set to include a second JAVA language layer The audio data processing method of the module and the client of the second C language layer module.

The computer program product of the embodiment provided by the embodiment of the present application adopts the method that: the second JAVA language layer module of the client receives the audio data sent by the operating system and the first implementation pointer, when the first implementation pointer type is NULL, The second JAVA language layer module sends the audio data to the second C language layer module; when the first implementation pointer type is non-NULL, the second JAVA language layer module sends the first implementation pointer to the second C language layer module; After receiving the audio data, the second C language layer module performs sound processing on the audio data to obtain non-low delay sound effect audio data, and sends the non-low delay sound effect audio data to the second JAVA language layer module; the second C language layer module After receiving the first implementation pointer of the non-NULL type, the audio data is called by the first callback function, the audio data is processed by the sound, the low delay audio audio data is obtained, and the low delay audio audio data is sent to the second JAVA language. Layer module; the second JAVA language layer module generates a second type of NULL after receiving non-low latency audio audio data The pointer is forwarded, and the second implementation pointer and the received non-low delay audio audio data are sent to the operating system for triggering the operating system to play the non-low delay audio audio data according to the second implementation pointer; the second JAVA language layer After receiving the low-latency audio audio data, the module generates a second implementation pointer of a non-NULL type, and sends the second implementation pointer and the received low-latency audio audio data to the operating system for triggering the operating system according to the second Implement pointers to play low-latency audio data. In this embodiment, non-low-latency audio audio data and low-latency audio audio data can be respectively obtained through a set of computer program products including instructions, and the different audio data is played, thereby reducing the maintenance cost of the software.

In still another embodiment provided by the present application, there is also provided a computer program product comprising instructions, when executed on a computer, causing a computer to perform any of the above-described embodiments for applying to include a first C language layer module And an audio data processing method of the terminal operating system of the first JAVA language layer module.

The computer program product including the instruction provided by the embodiment of the present application adopts the method of first determining the currently adopted sound effect processing mode. When the sound processing mode is the low delay ear return mode, the first C language layer module collects the audio data, and sends the audio data to the first JAVA language layer module; when the sound effect processing mode is the non-low delay ear return mode, the first The JAVA language layer module collects the audio data; after receiving the audio data collected by the first C language layer module, the first JAVA language layer module generates a first implementation pointer of a non-NULL type, and the audio data and the type are non-NULL. The first implementation pointer is sent to the client, and is used to trigger the client to perform low-latency sound processing on the audio data, obtain low-latency audio audio data, generate a second implementation pointer of a non-NULL type, and generate low-delay audio data and types. Returning the non-NULL second implementation pointer to the first JAVA language layer module; after acquiring the audio data, the first JAVA language layer module generates a first implementation pointer of type NULL, and the audio data and the type are null NULL The first implementation pointer is sent to the client, and is used to trigger the client to perform non-low delay sound processing on the audio data to obtain non-low delay audio audio. According to the second implementation pointer of type NULL, the non-low delay audio audio data and the second implementation pointer of type NULL are returned to the first JAVA language layer module; the first JAVA language layer module is sent by the receiving client After the low latency audio audio data and the second implementation pointer of the non-NULL type, the second implementation pointer of the non-NULL type is sent to the first C language layer module; the first JAVA language layer module receives the sending by the client After the non-low-latency audio audio data and the second implementation pointer of type NULL, the second implementation pointer of type NULL and the non-low delay audio audio data are sent to the first C language layer module; the first C language layer module is receiving After the second implementation pointer of the non-NULL type, the low-definition audio audio data is played according to the second implementation pointer of the non-NULL type; the first C language layer module receives the second implementation pointer of the type NULL and is non-low After delaying the audio data, non-low-latency audio data is played according to the second implementation pointer of type NULL. In this embodiment, non-low-latency audio audio data and low-latency audio audio data can be respectively obtained through a set of computer program products including instructions, and the different audio data is played, thereby reducing the maintenance cost of the software.

In still another embodiment provided by the present application, there is also provided a computer program comprising instructions, when executed on a computer, causing the computer to perform any of the above embodiments to be set to include the first C language layer module And an audio data processing method of the terminal operating system of the first JAVA language layer module.

The computer program including the instruction provided by the embodiment of the present application adopts the method of first determining the currently adopted sound effect processing mode. When the sound processing mode is the low delay ear return mode, the first C language layer module collects the audio data, and sends the audio data to the first JAVA language layer module; when the sound effect processing mode is the non-low delay ear return mode, the first The JAVA language layer module collects the audio data; after receiving the audio data collected by the first C language layer module, the first JAVA language layer module generates a first implementation pointer of a non-NULL type, and the audio data and the type are non-NULL. The first implementation pointer is sent to the client, and is used to trigger the client to perform low-latency sound processing on the audio data, obtain low-latency audio audio data, generate a second implementation pointer of a non-NULL type, and generate low-delay audio data and types. Returning the non-NULL second implementation pointer to the first JAVA language layer module; after acquiring the audio data, the first JAVA language layer module generates a first implementation pointer of type NULL, and the audio data and the type are null NULL The first implementation pointer is sent to the client, and is used to trigger the client to perform non-low delay sound processing on the audio data to obtain non-low delay audio audio. According to the second implementation pointer of type NULL, the non-low delay audio audio data and the second implementation pointer of type NULL are returned to the first JAVA language layer module; the first JAVA language layer module is sent by the receiving client After the low latency audio audio data and the second implementation pointer of the non-NULL type, the second implementation pointer of the non-NULL type is sent to the first C language layer module; the first JAVA language layer module receives the sending by the client After the non-low-latency audio audio data and the second implementation pointer of type NULL, the second implementation pointer of type NULL and the non-low delay audio audio data are sent to the first C language layer module; the first C language layer module is receiving After the second implementation pointer of the non-NULL type, the low-definition audio audio data is played according to the second implementation pointer of the non-NULL type; the first C language layer module receives the second implementation pointer of the type NULL and is non-low After delaying the audio data, non-low-latency audio data is played according to the second implementation pointer of type NULL. In this embodiment, non-low-latency audio audio data and low-latency audio audio data can be respectively obtained through a set of computer program products including instructions, and the different audio data is played, thereby reducing the maintenance cost of the software.

In still another embodiment provided by the present application, there is also provided a computer program comprising instructions, when executed on a computer, causing the computer to perform any of the above embodiments to be set to include a second JAVA language layer module And the audio data processing method of the client of the second C language layer module.

The computer program including the instruction provided by the embodiment of the present application adopts the method that: the second JAVA language layer module of the client receives the audio data sent by the operating system and the first implementation pointer, when the first implementation pointer type is NULL, The second JAVA language layer module sends the audio data to the second C language layer module; when the first implementation pointer type is non-NULL, the second JAVA language layer module sends the first implementation pointer to the second C language layer module; After receiving the audio data, the C language layer module performs audio processing on the audio data to obtain non-low delay audio audio data, and sends the non-low delay audio audio data to the second JAVA language layer module; the second C language layer module is After receiving the first implementation pointer of type non-NULL, the audio data is called by the first callback function, the audio data is processed, the low-latency audio data is obtained, and the low-latency audio data is sent to the second JAVA language. a layer module; after receiving the non-low delay audio audio data, the second JAVA language layer module generates a second real type of NULL a pointer, and sending the second implementation pointer and the received non-low delay audio audio data to the operating system, for triggering the operating system to play the non-low delay audio audio data according to the second implementation pointer; the second JAVA language layer module After receiving the low-latency audio audio data, generating a second implementation pointer of a non-NULL type, and transmitting the second implementation pointer and the received low-latency audio audio data to the operating system for triggering the operating system according to the second implementation Pointer for playing low-latency audio data. In this embodiment, non-low-latency audio audio data and low-latency audio audio data can be respectively obtained through a set of computer program products including instructions, and the different audio data is played, thereby reducing the maintenance cost of the software.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, it may be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions. When the above computer program instructions are loaded and executed on a computer, the above described processes or functions in accordance with embodiments of the present application are generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, computer instructions can be wired from a website site, computer, server or data center (eg Coax, fiber, digital subscriber line (DSL) or wireless (eg, infrared, wireless, microwave, etc.) is transmitted to another website, computer, server, or data center. The computer readable storage medium described above can be any available media that can be accessed by a computer or a data storage device such as a server, data center, or the like that includes one or more available media. The above usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).

It should be noted that, in this context, relational terms such as first and second are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply such entities or operations. There is any such actual relationship or order between them. Furthermore, the term "comprises" or "comprises" or "comprises" or any other variations thereof is intended to encompass a non-exclusive inclusion, such that a process, method, article, or device that comprises a plurality of elements includes not only those elements but also Other elements, or elements that are inherent to such a process, method, item, or device. An element defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device that comprises the element, without further limitation. The various embodiments in the present specification are described in a related manner, and the same or similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the device and the electronic device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

The above is only the preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application are included in the scope of the present application.

Industrial applicability

Based on the foregoing audio data processing method, device, electronic device and storage medium provided by the embodiment of the present application, the client receives the audio data sent by the operating system and the first implementation pointer. When the type of the first implementation pointer is NULL, Performing audio processing on the audio data to obtain non-low-latency audio audio data; when the first implementation pointer type is non-NULL, performing audio processing on the audio data to obtain low-latency audio audio data, and the client will have non-low delay The audio audio data or the low-latency audio data and the second implementation pointer are sent to the operating system, and the operating system plays the received non-low-latency audio data or the low-delay audio data according to the second implementation pointer. The present application can obtain non-low delay audio audio data and low delay audio audio data through a set of methods or devices, and play the above different audio data, thereby reducing the maintenance cost of the software.

Claims

An audio data processing method is applied to a client, which includes:

Receiving a first implementation pointer and audio data sent by the operating system; wherein, if the currently adopted sound processing mode is a non-low delay ear return mode, the type of the first implementation pointer is a first type; if the currently used sound processing The mode is a low-latency ear-back mode, and the type of the first implementation pointer is a second type; the first implementation pointer is sent after the operating system collects audio data;

Determining a type of the first implementation pointer;

If it is determined that the type of the first implementation pointer is the first type, performing sound processing on the received audio data to obtain non-low delay audio effect audio data;

After obtaining the non-low delay sound effect audio data, generating a second implementation pointer of a first type, and transmitting the second implementation pointer and the non-low delay audio audio data to the operating system for Triggering the operating system to play the non-low delay audio audio data according to the second implementation pointer;

If it is determined that the type of the first implementation pointer is the second type, the audio data is invoked from the operating system by using a first callback function, and audio processing is performed on the audio data to obtain low-latency audio audio data;

After obtaining the low-latency audio audio data, generating a second implementation pointer of the second type, and transmitting the second implementation pointer and the low-latency audio audio data to the operating system for triggering The operating system plays the low-latency audio audio data according to the second implementation pointer.
The method of claim 1 wherein said client comprises a second JAVA language layer module;

The step of receiving the first implementation pointer and the audio data sent by the operating system includes:

The second JAVA language layer module receives the audio data sent by the operating system and the first implementation pointer.
The method of claim 2, wherein the operating system comprises a first JAVA language layer module;

The step of receiving, by the second JAVA language layer module, the audio data sent by the operating system and the first implementation pointer includes:

The second JAVA language layer module receives the audio data and the first implementation pointer sent by the first JAVA language layer module.
The method of claim 2 wherein said client further comprises a second C language layer module;

If the type of the first implementation pointer is determined to be the first type, the step of performing sound processing on the received audio data to obtain non-low delay audio audio data includes:

If the type of the first implementation pointer is determined to be the first type, the second C language layer module receives the audio data sent by the second JAVA language layer module;

The second C language layer module performs sound processing on the audio data to obtain non-low delay sound effect audio data;

If it is determined that the type of the first implementation pointer is the second type, the audio data is called from the operating system by using a first callback function, and the audio data is subjected to sound processing to obtain low-delay audio audio data. Steps, including:

If the type of the first implementation pointer is determined to be the second type, the second C language layer module receives the first implementation pointer sent by the second JAVA language layer module;

The second C language layer module calls the audio data through a first callback function, and performs audio processing on the audio data to obtain low-latency audio audio data.
The method of claim 4, wherein the operating system further comprises a first C language layer module;

The step of the second C language layer module calling the audio data by using a first callback function, performing sound processing on the audio data, and obtaining low-latency audio audio data, including:

The second C language layer module calls the audio data collected by the first C language layer module from the first C language layer module by using a first callback function, and performs sound processing on the audio data to obtain a low delay sound effect. Audio data.
The method according to any one of claims 1 to 5, wherein the first type is null NULL and the second type is non-NULL.
An audio data processing method is applied to an operating system of a terminal, where the method includes:

Collecting audio data;

Determining the currently used sound processing mode from the non-low delay ear return mode and the low delay ear return mode;

Generating a first implementation pointer according to the currently adopted sound processing mode. When the sound processing mode is a non-low delay ear return mode, the type of the first implementation pointer is a first type, and when the sound processing mode is a low delay ear When returning to the mode, the type of the first implementation pointer is the second type;

Sending the first implementation pointer and the audio data to the client, so that if the client determines that the type of the first implementation pointer is the first type, obtaining the audio data sent by the operating system, Performing audio processing on the audio data to obtain non-low delay audio audio data. If the client determines that the type of the first implementation pointer is the second type, the audio data is called by the first callback function, and the audio is The data is subjected to sound processing to obtain low-latency audio audio data; after the non-low-latency audio audio data or the low-delay audio audio data is obtained, the obtained data is fed back to the operating system;

Receiving the non-low delay sound effect audio data or the low delay sound effect audio data sent by the client, and playing the received audio data.
The method of claim 7, wherein the operating system comprises a first C language layer module and a first JAVA language layer module;

When the sound processing mode is a low-latency ear-back mode, the step of collecting audio data includes:

The first C language layer module collects audio data;

The step of generating a first implementation pointer according to the currently adopted sound processing mode includes:

After receiving the audio data collected by the first C language layer module, the first JAVA language layer module generates a first implementation pointer of a second type;

The step of sending the first implementation pointer and the audio data to a client includes:

Transmitting, by the first JAVA language layer module, the audio data and the first implementation pointer of the second type to a client;

The step of receiving the non-low delay audio effect audio data or the low delay audio effect audio data sent by the client, and playing the received audio data, includes:

The first JAVA language layer module receives the low-latency sound effect audio data sent by the client and a second implementation pointer of a second type, and sends the second implementation pointer of the second type to The first C language layer module;

The first C language layer module plays the low-latency audio audio data according to the second implementation pointer of the second type.
The method of claim 7, wherein the operating system comprises a first C language layer module and a first JAVA language layer module;

When the audio processing mode is a non-low delay ear return mode, the step of collecting audio data includes: the first JAVA language layer module collecting audio data;

The step of generating a first implementation pointer according to the currently adopted sound processing mode includes:

After acquiring the audio data, the first JAVA language layer module generates a first implementation pointer of a first type;

The step of sending the first implementation pointer and the audio data to a client includes:

The first JAVA language layer module sends the audio data and the first implementation pointer of the type of the first type to a client;

The step of receiving the non-low delay audio effect audio data or the low delay audio effect audio data sent by the client, and playing the received audio data, includes:

The first JAVA language layer module receives the low-latency sound effect audio data sent by the client and a second implementation pointer of a second type, and sends the second implementation pointer of the second type to The first C language layer module;

The first C language layer module plays the non-low delay audio audio data according to the second implementation pointer of the first type.
The method of claim 8 wherein said client comprises a second JAVA language layer module;

The step of the first JAVA language layer module sending the audio data and the first implementation pointer of the second type to the client, including:

The first JAVA language layer module sends the audio data and the first implementation pointer of the second type to the second JAVA language layer module.
The method of claim 9, wherein the client comprises a second JAVA language layer module;

The step of the first JAVA language layer module sending the audio data and the first implementation pointer of the first type to the client, including:

The first JAVA language layer module sends the audio data and the first implementation pointer of the type of the first type to a second JAVA language layer module.
An audio data processing device is applied to a client, wherein:

a data receiving module, configured to receive a first implementation pointer and audio data sent by the operating system; wherein, if the currently adopted sound processing mode is a non-low delay ear return mode, the type of the first implementation pointer is a first type; If the currently used sound processing mode is a low-latency ear-back mode, the type of the first implementation pointer is a second type; the first implementation pointer is sent after the operating system collects audio data;

a pointer type determining module, configured to determine a type of the first implementation pointer;

The first data processing module is configured to: if it is determined that the type of the first implementation pointer is the first type, performing sound effect processing on the received audio data to obtain non-low delay audio effect audio data;

a first data sending module, configured to generate a second implementation pointer of a first type after the non-low delay audio audio data is obtained, and send the second implementation pointer and the non-low delay audio audio data Up to the operating system, configured to trigger the operating system to play the non-low delay audio audio data according to the second implementation pointer;

a second data processing module, configured to: if it is determined that the type of the first implementation pointer is a second type, invoke the audio data from the operating system by using a first callback function, and perform sound processing on the audio data to obtain Low latency audio audio data;

a second data sending module, configured to generate a second implementation pointer of a second type after the low-latency audio audio data is obtained, and send the second implementation pointer and the low-latency audio audio data to the The operating system is configured to trigger the operating system to play the low-latency audio audio data according to the second implementation pointer.
The apparatus of claim 12, wherein the client comprises a second JAVA language layer module;

The data receiving module is configured to:

The second JAVA language layer module is triggered to receive the audio data sent by the operating system and the first implementation pointer.
The apparatus of claim 13, wherein the operating system comprises a first JAVA language layer module;

The first data sending module is configured to:

And the triggering, by the second JAVA language layer module, the audio data and the first implementation pointer sent by the first JAVA language layer module.
The apparatus of claim 13, wherein the client further comprises a second C language layer module;

The first data processing module includes:

The first data receiving unit is configured to: if it is determined that the type of the first implementation pointer is the first type, triggering the second C language layer module to receive the audio data sent by the second JAVA language layer module;

a first data processing unit, configured to trigger the second C language layer module to perform sound processing on the audio data to obtain non-low delay audio audio data;

The first data processing module further includes:

a second data receiving unit, configured to: if it is determined that the type of the first implementation pointer is a second type, triggering the second C language layer module to receive the first implementation pointer sent by the second JAVA language layer module;

The second data processing unit is configured to trigger the second C language layer module to call the audio data through the first callback function, and perform sound effect processing on the audio data to obtain low-latency audio audio data.
The apparatus of claim 15, wherein the operating system further comprises a first C language layer module;

The second C language layer module is set to:

The audio data collected by the first C language layer module is invoked from the first C language layer module by the first callback function, and the audio data is subjected to sound processing to obtain low-latency audio audio data.
Apparatus according to any one of claims 12 to 16, wherein said first type is null NULL and said second type is non-NULL.
An audio data processing apparatus is applied to an operating system of a terminal, wherein the apparatus comprises:

a data acquisition module configured to collect audio data;

a mode determining module configured to determine a currently used sound effect processing mode from the non-low delay ear return mode and the low delay ear return mode;

a pointer generating module, configured to generate a first implementation pointer according to the currently adopted sound processing mode, and when the sound processing mode is a non-low delay earback mode, the first implementation pointer type is a first type, when the sound effect When the processing mode is the low delay ear return mode, the type of the first implementation pointer is the second type;

a data sending module, configured to send the first implementation pointer and the audio data to a client, so that if the client determines that the type of the first implementation pointer is a first type, obtaining the sending by the operating system The audio data is subjected to sound processing on the audio data to obtain non-low delay sound effect audio data. If the client determines that the type of the first implementation pointer is the second type, the audio is called by the first callback function. Data, performing audio processing on the audio data to obtain low-latency audio audio data; after obtaining the non-low-latency audio audio data or the low-latency audio audio data, feeding the obtained data to the operating system;

The data receiving module is configured to receive the non-low delay sound effect audio data or the low delay sound effect audio data sent by the client, and play the received audio data.
The apparatus of claim 18, wherein the operating system comprises a first C language layer module and a first JAVA language layer module;

The data acquisition module is configured to: trigger the first C language layer module to collect audio data, when the sound processing mode is a low-latency ear-back mode;

The pointer generating module is configured to: trigger the first JAVA language layer module to generate a first implementation pointer of a second type after receiving the audio data collected by the first C language layer module;

The data sending module is configured to: trigger the first JAVA language layer module to send the audio data and the first implementation pointer of the second type to the client;

The data receiving module is configured to: trigger the first JAVA language layer module to receive the low-latency sound effect audio data sent by the client, and a second implementation pointer of a second type, and the type is Sending, by the first C language layer module, the second implementation pointer of the second type to the first C language layer module, according to the second implementation pointer of the second type, playing the low delay audio audio data .
The apparatus of claim 18, wherein the operating system comprises a first C language layer module and a first JAVA language layer module;

The data acquisition module is configured to: trigger the first JAVA language layer module to collect audio data, when the sound processing mode is a non-low delay ear return mode;

The pointer generating module is configured to: trigger the first JAVA language layer module to generate a first implementation pointer of a first type after acquiring audio data;

The data sending module is configured to: trigger the first JAVA language layer module to send the audio data and the first implementation pointer of the type to the first type to the client;

The data receiving module is configured to: trigger the first JAVA language layer module to receive the low-latency sound effect audio data sent by the client, and a second implementation pointer of a second type, and the type is Transmitting the second implementation pointer of the second type to the first C language layer module; triggering the first C language layer module to play the non-low delay audio audio according to the second implementation pointer of the type being the first type data.
The apparatus of claim 19, wherein the client comprises a second JAVA language layer module;

The data sending module is configured to: the first JAVA language layer module sends the audio data and the first implementation pointer of the second type to the second JAVA language layer module.
The apparatus of claim 20, wherein the client comprises a second JAVA language layer module; the data sending module is configured to: the first JAVA language layer module sets the audio data and the type to A first implementation pointer of the first type is sent to the second JAVA language layer module.
An electronic device including a processor and a memory;

a memory, configured to store a computer program;

The method steps of any of claims 1-6 are implemented when the processor is configured to execute a program stored on the memory.
An electronic device including a processor and a memory;

a memory, configured to store a computer program;

The method steps of any of claims 7-11 are implemented when the processor is configured to execute a program stored on the memory.
A computer readable storage medium, wherein the computer readable storage medium stores a computer program, the computer program being executed by a processor to implement the method steps of any of claims 1-6.
A computer readable storage medium, wherein the computer readable storage medium stores a computer program, the computer program being executed by a processor to implement the method steps of any of claims 7-11.
A computer program for performing the method steps of any of claims 1-6 at runtime.
A computer program for performing the method steps of any of claims 7-11 at runtime.