CN115437611A - Audio coding and decoding frame and development method based on audio coding and decoding frame - Google Patents
Audio coding and decoding frame and development method based on audio coding and decoding frame Download PDFInfo
- Publication number
- CN115437611A CN115437611A CN202211122264.5A CN202211122264A CN115437611A CN 115437611 A CN115437611 A CN 115437611A CN 202211122264 A CN202211122264 A CN 202211122264A CN 115437611 A CN115437611 A CN 115437611A
- Authority
- CN
- China
- Prior art keywords
- audio
- decoding
- coding
- component
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/20—Software design
- G06F8/24—Object-oriented
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The application discloses an audio coding and decoding framework and a development method based on the audio coding and decoding framework. The frame includes: the application interface layer comprises a developer interface, the audio processing layer comprises an audio coding and decoding interface, the audio coding and decoding component layer comprises a plurality of audio components, and the application interface layer comprises an audio coding and decoding component layer and an operating system abstraction layer application interface, wherein: the application interface layer is used for receiving the audio data and generating a calling instruction corresponding to the audio data; the audio processing layer is used for receiving the audio data and the calling instruction and calling a target audio component corresponding to the calling instruction in the audio coding and decoding component layer through the audio coding and decoding interface; the audio coding and decoding component layer is used for coding and decoding the audio data through the target audio component and returning the coded and decoded audio data to the application interface layer; and the developer interface is used for the developer to access the audio coding and decoding framework into the application program to be developed.
Description
Technical Field
The application relates to the technical field of software development, in particular to an audio coding and decoding framework and a development method based on the audio coding and decoding framework.
Background
With the rapid development of microelectronic technology and multimedia technology, embedded products are widely applied in many fields such as homes, workplaces, entertainment venues, etc.
The smart watch develops rapidly in recent years, and particularly, the smart watch is added with audio functions such as music playing and recording, so that the requirements on audio coding and decoding are greatly improved. However, most smartwatches use a Microprogrammed Control Unit (MCU) and a Central Processing Unit (CPU) with low power consumption, and have the characteristics of imperfect software ecosphere, limited system resources, and the need to rapidly release new audio related functions, so that the audio coding and decoding requirements are simple and the component type access with functions increased and decreased at any time.
However, in the current embedded field, especially in the field of non-operating systems, no audio codec framework is available, which results in continuous and repeated wheel building and low development efficiency.
Disclosure of Invention
The embodiment of the application provides an audio coding and decoding frame and a development method based on the audio coding and decoding frame, and development efficiency can be improved.
In a first aspect, an embodiment of the present application provides an audio coding and decoding framework, including: the device comprises an application interface layer, an audio processing layer, an audio coding and decoding component layer and an operating system abstraction layer application interface, wherein the application interface layer comprises a developer interface, the audio processing layer comprises an audio coding and decoding interface, the audio coding and decoding component layer comprises a plurality of audio components, and the application interface layer comprises a developer interface, the audio processing layer comprises an audio coding and decoding interface, and the audio coding and decoding component layer comprises a plurality of audio components, wherein:
the application interface layer is used for receiving audio data, generating a calling instruction corresponding to the audio data, and sending the audio data and the calling instruction to the audio processing layer;
the audio processing layer is used for receiving the audio data and the calling instruction, and calling a target audio component corresponding to the calling instruction in the audio coding and decoding component layer through the audio coding and decoding interface under the condition of receiving the calling instruction;
the audio coding and decoding component layer is used for coding and decoding the audio data through the target audio component and returning the coded and decoded audio data to the application interface layer;
the operating system abstract layer application interface is used for shielding a lower operating system;
the developer interface is used for a developer to access the audio coding and decoding framework into an application program to be developed.
In a second aspect, an embodiment of the present application further provides a development method based on an audio codec framework, where the development method is applied to a computer device, the computer device is used to develop an application program to be developed, the audio codec framework is accessed to the application program to be developed, the audio codec framework includes an application interface layer, an audio processing layer, an audio codec component layer, and an operating system abstraction layer application interface, the application interface layer includes a developer interface, the audio processing layer includes an audio codec interface, the audio codec component layer includes multiple audio components, and the method includes:
the application interface layer receives audio data and generates a calling instruction corresponding to the audio data, and sends the audio data and the calling instruction to the audio processing layer;
the audio processing layer calls a target audio component corresponding to the calling instruction in the audio coding and decoding component layer through the audio coding and decoding interface under the condition that the calling instruction is received;
the target audio component carries out coding and decoding processing on the audio data and returns the audio data subjected to coding and decoding processing to the application interface layer;
and the to-be-developed application program calls the audio data subjected to the coding and decoding processing in the application interface layer to realize the coding and decoding functions of the to-be-developed application on the audio data.
The embodiment of the application provides an audio coding and decoding framework and a development method based on the audio coding and decoding framework. Wherein, this frame includes: the device comprises an application interface layer, an audio processing layer, an audio coding and decoding component layer and an operating system abstraction layer application interface, wherein the application interface layer comprises a developer interface, the audio processing layer comprises an audio coding and decoding interface, the audio coding and decoding component layer comprises a plurality of audio components, and the application interface layer comprises a developer interface, the audio processing layer comprises an audio coding and decoding interface, and the audio coding and decoding component layer comprises a plurality of audio components, wherein: the application interface layer is used for receiving audio data, generating a calling instruction corresponding to the audio data, and sending the audio data and the calling instruction to the audio processing layer; the audio processing layer is used for receiving the audio data and the calling instruction and calling a target audio component corresponding to the calling instruction in the audio coding and decoding component layer through the audio coding and decoding interface; the audio coding and decoding component layer is used for coding and decoding the audio data through the target audio component and returning the coded and decoded audio data to the application interface layer; the operating system abstract layer application interface is used for shielding a lower operating system; the developer interface is used for a developer to access the audio coding and decoding framework into an application program to be developed. In the scheme, a developer can access the audio coding and decoding framework into the application program to be developed through a developer interface, the audio coding and decoding function is realized through the audio coding and decoding framework, and the developer does not need to develop the audio coding and decoding function when developing the application program, so that the development efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic structural diagram of an audio codec framework according to an embodiment of the present application;
FIG. 2 is an interaction diagram of an audio processing layer and an audio encoding component layer according to an embodiment of the present application;
fig. 3 is a schematic view of an application scenario of a development method based on an audio coding/decoding framework according to an embodiment of the present application;
FIG. 4 is a flow chart of a command response of the audio codec framework according to the embodiment of the present application;
fig. 5 is a schematic flowchart of a development method based on an audio codec framework according to an embodiment of the present application;
fig. 6 is a schematic overall flowchart of a development method based on an audio codec framework according to an embodiment of the present application;
fig. 7 is a decoding application scenario diagram of a development method based on an audio coding/decoding framework according to an embodiment of the present application;
fig. 8 is a hybrid application scenario diagram of a development method based on an audio codec framework according to an embodiment of the present application;
fig. 9 is a coding application scenario diagram of a development method based on an audio coding/decoding framework according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification of the present application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items and includes such combinations.
The embodiment of the application provides an audio coding and decoding framework and a development method based on the audio coding and decoding framework.
The following first describes an audio codec framework provided in the present application:
referring to fig. 1, fig. 1 is a schematic structural diagram of an audio codec framework provided in an embodiment of the present Application, where the audio codec framework includes an Application Interface (API) 40, an audio processing Layer 20, an audio codec component Layer 30, and an Operating System Abstraction Layer (Operating System Abstraction Layer Application Interface, OSAL) Interface (API), the audio processing Layer 20 includes an audio codec API21, the Application Interface Layer 10 includes a developer API11, and the audio codec component Layer 30 includes a plurality of audio components 31, where:
application interface layer 10:
the application interface layer 10 is mainly used for receiving audio data, generating a call instruction corresponding to the audio data, and sending the audio data and the call instruction to the audio processing layer 20;
specifically, the application interface layer 10 is responsible for establishing and maintaining an audio processing chain according to application requirements, there is no actual audio processing in this layer, and it is an external call interface, and the application interface layer 10 is mainly used for sending commands and receiving replies of the underlying audio processing layer 20.
In some embodiments, the application interface layer 10 also creates an IPC (InterProcess Connect) thread through which commands are sent to the audio processing layer 20 and responses are received by the audio processing layer 20.
Further, the application interface layer 10 provides APIs such as a start interface, a pause interface, a continue interface, a stop interface, a volume adjustment interface, a data input interface, a mix interface, a rendering hardware interface, and a sound (microphone) capture interface, and when the application to be developed calls these interfaces, a conversion command message is sent to the audio processing layer 20 through IPC.
Audio processing layer 20:
the audio processing layer 20 is mainly configured to receive audio data and a call instruction, and call a target audio component corresponding to the call instruction in the audio codec component layer 30 through the audio codec API21 when the call instruction is received;
further, the audio processing layer 20 further includes a control module 22 and a processing module 23, where the control module 22 includes an agent interface and a message interface, the agent interface is used to obtain audio data, the message interface is used to enable the control module 22 to be matched with the processing module 23, the processing module 23 includes a memory management module and a processing scheduling module, the memory management module is used to uniformly manage a used memory of the audio codec frame, and the processing scheduling module is used to schedule the target audio component according to the call instruction.
Audio codec component layer 30:
the audio coding and decoding component layer 30 is mainly used for coding and decoding the audio data through the target audio component, and returning the coded and decoded audio data to the application interface layer 10;
the audio components included in the audio codec component layer 30 include various functional audio algorithms, and are, for example, an MP3 decoding component, an AAC decoding component, an MP3 encoding component, an AAC decoding component, and so on. It can perform actual audio processing according to the call instruction received by the application interface layer 10, and return the result to the application interface layer 10 after completing the command.
The audio codec framework further includes a Real Time Operating System (RTOS) 50, where the RTOS50 is used to implement IPC (Inter Process connection) communication between the application interface layer 10 and the audio processing layer 20, where the IPC is a connection between the application interface layer 10 and the audio processing layer 20, and is used to transmit commands and receive responses.
Further, RTOS50 is capable of creating multiple threads, with data communication between threads being accomplished through RTOS50 message queues and mutex locks. Typically RTOS50 comprises a FreeRTOS or UCOS.
OSALAPI11：
Operating system abstraction layer application interfaces are used to mask the underlying operating system, and there are many other RTOS kernels in the embedded domain such as: UCOS, freeRTOS, RT-Thread, liteos, aliOS, and the like. Because the functions and system call interfaces of different operating systems are often different, a unified standard does not exist at present, so that the transplanting cost of the embedded application program is too high, and on the other hand, programmers often need to be familiar with a plurality of different development environments, so that the development cost of the embedded software is greatly increased. This necessitates the ability to shield the underlying disparate operating systems.
Developer API11:
the audio codec framework is used for a developer to access the audio codec framework to an application program to be developed, the developer API11 is an interface between the application program to be developed and the audio codec framework, and the developer can establish (pull) the audio codec framework through the developer API11, configure parameters for the framework, and start the framework, so as to use the codec and other functions of the framework.
It should be noted that, when pulling the audio decoding framework, the developer can selectively load the audio components in the audio codec component layer 30, and the audio codec component has scalability.
When new audio components need to be added or deleted, the components can be directly added or deleted through the audio codec API 21.
Referring to fig. 2, fig. 2 is an interactive schematic diagram of an audio processing layer 20 and an audio encoding component layer in the embodiment:
the audio processing layer 20 employs an object-oriented class architecture to manage, schedule, and execute the respective components.
In FIG. 2, the generic base class provides all component generic functions, such as memory allocation, memory release;
audio codec class: a codec component for MP3 (a file in an audio coding format), AAC (a file in an audio coding format);
and (3) mixed type: synthesizing one audio data for a plurality of audio data;
rendering type: the voice is output to equipment such as a loudspeaker and the like, and the voice is rendered to enable the loudspeaker to sound;
capture class: for capturing microphone audio data.
Each class is used to schedule components for a corresponding function.
In summary, the audio codec framework provided by the present solution includes an application interface layer 10, an audio processing layer 20, an audio codec component layer 30 and an operating system abstraction layer application interface, where the application interface layer 10 includes a developer API11, the audio processing layer 20 includes an audio codec API21, and the audio codec component layer 30 includes a plurality of audio components, where: the application interface layer 10 is configured to receive audio data, generate a call instruction corresponding to the audio data, and send the audio data and the call instruction to the audio processing layer 20; the audio processing layer 20 is configured to receive audio data and a call instruction, and call a target audio component corresponding to the call instruction in the audio coding and decoding component layer 30 through the audio coding and decoding API 21; the audio coding and decoding component layer 30 is configured to perform coding and decoding processing on the audio data through the target audio component, and return the coded and decoded audio data to the application interface layer 10; the operating system abstract layer application interface is used for shielding a lower operating system; and the developer API11 is used for the developer to access the audio coding and decoding framework into the application program to be developed. In the scheme, a developer can access the audio coding and decoding framework into the application program to be developed through the developer API11, the audio coding and decoding function is realized through the audio coding and decoding framework, and the developer does not need to develop the audio coding and decoding function when developing the application program, so that the development efficiency is improved.
Referring to fig. 3, fig. 3 is a schematic view of an application scenario of a development method based on an audio codec framework according to an embodiment of the present application, where an application in the diagram is an application program, such as an application program to be developed or a developed application program, and the specific details are not limited herein.
As shown in fig. 4, fig. 4 is a command response flowchart of the audio codec framework in the embodiment of the present application, a command message needs to be sent to the audio codec framework each time an application to be developed needs a certain function, the application interface layer 10 transfers the command to the audio processing layer 20 through IPC, the audio processing layer 20 finishes processing and generates a response message, and the response message is also returned to the application interface layer 10 through IPC, so that the application to be developed receives the response message.
Fig. 5 is a flowchart of the development method based on an audio codec framework according to an embodiment of the present application, where the development method based on an audio codec framework is applied to a computer device, the computer device may be a terminal or a server, the computer device may be used to develop an application program to be developed, the application program to be developed has an audio codec framework connected thereto, the audio codec framework includes an application interface layer 10, an audio processing layer 20, an audio codec component layer 30, and an operating system abstraction layer application interface, the application interface layer 10 includes a developer API11, the audio processing layer 20 includes an audio codec API21, and the audio codec component layer 30 includes multiple audio components, as shown in fig. 5, the method includes the following steps S110 to S150.
S110, the application interface layer receives the audio data and generates a calling instruction corresponding to the audio data, and sends the audio data and the calling instruction to the audio processing layer.
Wherein the audio data comprises MP3 file audio data, AAC file audio data, PCM file audio data and/or microphone audio data.
Specifically, when determining the call instruction, the application interface layer 10 needs to first determine the data type of the audio data; and then generating a calling instruction according to the data type.
Further, when the call instruction is generated according to the data type, the method comprises the following steps:
if the data type is an MP3 file or an AAC file, determining a decoding instruction and a rendering instruction as calling instructions; if the data type is a PCM file, determining the encoding instruction as a calling instruction; and if the data type is an MP3 file and microphone audio or the data type is an AAC file and microphone audio, determining the decoding instruction, the mixing instruction and the rendering instruction as calling instructions.
S120, under the condition that the audio processing layer receives the calling instruction, calling a target audio component corresponding to the calling instruction in the audio coding and decoding component layer through an audio coding and decoding API.
In some embodiments, step S120 includes the steps of:
judging whether the calling instruction comprises a decoding instruction or not; if the decoding instruction is not contained, determining the coding component in the audio coding and decoding component layer 30 as a target audio component; if the mixed instruction contains the decoding instruction, judging whether the calling instruction contains the mixed instruction or not; if the mixing instruction is included, the decoding component, the mixing component, and the rendering component in the audio codec component layer 30 are determined as the target audio component, where the execution order of the decoding component, the mixing component, and the rendering component is: firstly, executing a decoding component, then mixing the components and finally rendering the components; if the mixing instruction is not included, determining the decoding component and the rendering component as target audio components, wherein the execution sequence of the decoding component and the rendering component is that the decoding component is executed first and then the rendering component is executed; the target audio component is called in the audio codec component through the audio codec API 21.
S130, the target audio component carries out coding and decoding processing on the audio data and returns the audio data subjected to coding and decoding processing to the application interface layer.
In some embodiments, step S130 includes: if the coding component is determined to be the target audio component, coding the audio data according to the coding component to obtain audio data subjected to coding and decoding; if the decoding assembly, the mixing assembly and the rendering assembly are determined to be the target audio assembly, decoding the audio data according to the decoding assembly to obtain decoded audio data; mixing the decoded audio data according to the mixing component to obtain mixed audio data; rendering the mixed audio data according to the rendering component to obtain audio data subjected to coding and decoding; if the decoding assembly and the rendering assembly are determined to be the target audio assembly, decoding the audio data according to the decoding assembly to obtain decoded audio data; and rendering the decoded audio data according to the rendering component to obtain the audio data subjected to coding and decoding.
S140, the to-be-developed application program calls the audio data subjected to encoding and decoding processing in the application interface layer, and the encoding and decoding functions of the to-be-developed application on the audio data are achieved.
In this embodiment, the application program to be developed only needs to call the audio data obtained through the audio codec framework after the codec processing, and a developer does not need to pay attention to the development of the audio codec when developing the application program to be developed, thereby improving the development efficiency.
It can be seen that, in the present application, by designing a set of audio codec framework having an application interface layer 10, an audio processing layer 20, and an audio codec component layer 30, a set of simplified "developer API" is defined, and communication between the application layer and the audio processing layer 20IPC is realized by using the RTOS, so that a final application program can be quickly created, the difficulty of using the audio codec by the developer is reduced, and the codec library components are flexibly increased and decreased, so that the developer can concentrate on developing business logic more, and does not need to spend too much time on researching development and use of components.
Referring to fig. 6, fig. 6 is a schematic overall flowchart of the development method based on the audio codec frame provided in this embodiment.
In fig. 6, the solid line part is the step executed inside the audio codec frame, and the dotted line is the step executed outside the audio codec frame. From FIG. 6, it can be seen that:
firstly, inputting audio file and audio stream data into an application interface layer 10 by a microphone, converting the application interface layer 10 into command messages according to requirements and sending the command messages to an audio processing layer 20;
the audio processing layer 20 adopts different modes according to whether different files need to be decoded, calls a corresponding decoding component to decode if the files need to be decoded, calls a coding component to code if the files do not need to be decoded, and outputs coded data to return to the application by the coding component.
The decoded audio data is generally mixed for the case that two audio data need to be mixed according to whether the audio data is mixed according to the application requirement, for example, the audio data is required only when the microphone audio is added to the background music, or the two audio data are mixed.
It should be noted that, the decoded data, whether mixed or not, may enter the rendering component for processing, and the rendering component is a driver of the hardware device, and functions to convert the digital signal into an analog signal and drive the speaker corresponding to the application to be developed to sound. The speaker can only receive analog signals to produce sound.
For further understanding the development method based on the audio codec framework in this embodiment, please refer to the application scenarios in fig. 7 to 9, where the application in the figures is an application program, such as an application program to be developed or a developed application program, and the details are not limited herein.
Referring to fig. 7, fig. 7 is a typical decoding application scenario, first, an MP3 file or an AAC file to be decoded and displayed is input into a frame through an application program, the application program calls an application interface layer 10 of an audio codec frame to process the input data into corresponding commands and data, the application interface layer 10 transfers the data and the commands to a control module 22 of an audio processing layer 20 in an IPC manner, and the audio processing layer 20 transfers the data to corresponding decoding components through an interface of a call processing module 23, for example: and after the MP3 or AAC decoding component decodes the audio waveform, the audio waveform can be output to a loudspeaker for playing through the audio waveform of the rendering component, so that the whole decoding process is completed.
Fig. 8 is a typical hybrid application scenario. Firstly, audio files such as MP3 files or AAC files and microphone data are input into an application interface layer 10 through an application program, the application program calls the application interface layer 10 of an audio coding and decoding framework to process the audio files and the microphone data into corresponding commands and data, the application interface layer 10 transmits the data and the commands to a control module 22 of an audio processing layer 20 in an IPC mode, the audio processing layer 20 transmits the audio data to a corresponding decoding component (for example, an MP3 or AAC decoding component, the decoded data are PCM audio data) through an interface of a calling processing module 23, mixed PCM audio data are output after the audio data and the microphone data are subjected to mixed component processing, the mixed PCM audio data are mixed to form audio waveforms through a rendering component, and the audio waveforms can be output to a loudspeaker to be played, so that the whole mixed processing flow is completed.
Fig. 9 is a typical audio coding application scenario. Firstly, a PCM original audio file is input into an application interface layer 10, an application program calls the application interface layer 10 of an audio coding and decoding framework to process the PCM original audio file into corresponding commands and data, the application interface layer 10 transmits the data and the commands to a control module 22 of an audio processing layer 20 in an IPC mode, the audio processing layer 20 transmits the audio data to a corresponding coding component (for example, an MP3 or AAC coding component, the coded data is the coded audio data) through a calling processing module 23 interface, and the audio data is output to an application to form a file with a corresponding format, so that the whole mixing processing flow is completed.
The audio coding and decoding framework provided by the embodiment can quickly create the final application program, reduces the difficulty of using audio coding and decoding by developers, flexibly increases and decreases the components of the coding and decoding library, enables the developers to concentrate on developing business logic, does not need to spend too much time on researching the components for use, and saves the time for developing audio functions of products. And the scalable audio coding and decoding architecture is convenient for adding functions of a new coding and decoding algorithm.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the technical solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative. For example, the division of each unit is only one logic function division, and there may be another division manner in actual implementation. For example, various elements or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented.
The steps in the method of the embodiment of the application can be sequentially adjusted, combined and deleted according to actual needs. The units in the device of the embodiment of the application can be combined, divided and deleted according to actual needs. In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a storage medium. Based on such understanding, the technical solution of the present application may be substantially or partially implemented in the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a terminal, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application.
While the invention has been described with reference to specific embodiments, the scope of the invention is not limited thereto, and those skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the invention. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. An audio codec framework, comprising: the device comprises an application interface layer, an audio processing layer, an audio coding and decoding component layer and an operating system abstraction layer application interface, wherein the application interface layer comprises a developer interface, the audio processing layer comprises an audio coding and decoding interface, the audio coding and decoding component layer comprises a plurality of audio components, and the application interface layer comprises a developer interface, the audio processing layer comprises an audio coding and decoding interface, and the audio coding and decoding component layer comprises a plurality of audio components, wherein:
the application interface layer is used for receiving audio data, generating a calling instruction corresponding to the audio data, and sending the audio data and the calling instruction to the audio processing layer;
the audio processing layer is used for receiving the audio data and the calling instruction, and calling a target audio component corresponding to the calling instruction in the audio coding and decoding component layer through the audio coding and decoding interface under the condition of receiving the calling instruction;
the audio coding and decoding component layer is used for coding and decoding the audio data through the target audio component and returning the coded and decoded audio data to the application interface layer;
the operating system abstract layer application interface is used for shielding a lower operating system;
the developer interface is used for a developer to access the audio coding and decoding framework into an application program to be developed.
2. The audio codec framework of claim 1, wherein the application interface layer further comprises a start interface, a pause interface, a continue interface, a stop interface, a volume adjustment interface, a data input interface, a mix interface, a rendering hardware interface, and a sound capture interface.
3. The audio codec frame of claim 1, wherein the audio processing layer comprises a control module and a processing module, the control module comprises an agent interface and a message interface, the agent interface is configured to obtain the audio data, the message interface is configured to enable the control module to cooperate with the processing module, the processing module comprises a memory management module and a processing scheduling module, the memory management module is configured to uniformly manage used memory of the audio codec frame, and the processing scheduling module is configured to schedule the target audio component.
4. The audio codec framework of any one of claims 1 to 3, further comprising a real-time operating system for enabling IPC communication of the application interface layer and the audio processing layer.
5. A development method based on an audio coding and decoding framework, which is applied to a computer device, wherein the computer device is used for developing an application program to be developed, and the application program to be developed is accessed to the audio coding and decoding framework of any one of claims 1 to 4, and the method comprises the following steps:
the application interface layer receives audio data and generates a calling instruction corresponding to the audio data, and sends the audio data and the calling instruction to the audio processing layer;
the audio processing layer calls a target audio component corresponding to the calling instruction in the audio coding and decoding component layer through the audio coding and decoding interface under the condition that the calling instruction is received;
the target audio component carries out coding and decoding processing on the audio data and returns the audio data subjected to coding and decoding processing to the application interface layer;
and the to-be-developed application program calls the audio data subjected to the coding and decoding processing in the application interface layer to realize the coding and decoding functions of the to-be-developed application on the audio data.
6. The method according to claim 5, wherein the calling, in the audio codec component layer through the audio codec interface, the target audio component corresponding to the call instruction comprises:
judging whether the calling instruction comprises a decoding instruction or not;
if the decoding instruction is not contained, determining the coding component in the audio coding and decoding component layer as the target audio component;
if the decoding instruction is contained, judging whether the calling instruction contains a mixed instruction or not;
if the mixing instruction is contained, determining a decoding component, a mixing component and a rendering component in the audio coding and decoding component layer as the target audio component;
if the mixing instruction is not included, determining the decoding component and the rendering component as the target audio component;
and calling the target audio component in the audio coding and decoding component through the audio coding and decoding interface.
7. The method of claim 6, wherein the encoding and decoding the audio data comprises:
if the coding component is determined to be the target audio component, coding the audio data according to the coding component to obtain the audio data after coding and decoding;
if the decoding assembly, the mixing assembly and the rendering assembly are determined to be the target audio assembly, decoding the audio data according to the decoding assembly to obtain decoded audio data;
mixing the decoded audio data according to the mixing component to obtain mixed audio data;
rendering the mixed audio data according to the rendering component to obtain the audio data after the coding and decoding;
if the decoding assembly and the rendering assembly are determined to be the target audio assembly, decoding the audio data according to the decoding assembly to obtain the decoded audio data;
and rendering the decoded audio data according to the rendering component to obtain the audio data subjected to encoding and decoding.
8. The method according to claim 6 or 7, wherein the audio data comprises MP3 file audio data, AAC file audio data, PCM file audio data and/or microphone audio data.
9. The method of claim 8, wherein the generating the call instruction corresponding to the audio data comprises:
determining a data type of the audio data;
and generating the calling instruction according to the data type.
10. The method of claim 9, wherein the generating the call instruction according to the data type comprises:
if the data type is an MP3 file or an AAC file, determining a decoding instruction and a rendering instruction as the calling instruction;
if the data type is a PCM file, determining the coding instruction as the calling instruction;
and if the data types are the MP3 file and the microphone audio, or the data types are the AAC file and the microphone audio, determining a decoding instruction, a mixing instruction and a rendering instruction as the calling instruction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211122264.5A CN115437611A (en) | 2022-09-15 | 2022-09-15 | Audio coding and decoding frame and development method based on audio coding and decoding frame |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211122264.5A CN115437611A (en) | 2022-09-15 | 2022-09-15 | Audio coding and decoding frame and development method based on audio coding and decoding frame |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115437611A true CN115437611A (en) | 2022-12-06 |
Family
ID=84248670
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211122264.5A Pending CN115437611A (en) | 2022-09-15 | 2022-09-15 | Audio coding and decoding frame and development method based on audio coding and decoding frame |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115437611A (en) |
-
2022
- 2022-09-15 CN CN202211122264.5A patent/CN115437611A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111634188B (en) | Method and device for projecting screen | |
| CN102427465B (en) | Voice service proxy method and device and system for integrating voice application through proxy | |
| CN108932948B (en) | Audio data processing method and device, computer equipment and computer readable storage medium | |
| EP2561439B1 (en) | Unified framework and method for call control and media control | |
| US8027839B2 (en) | Using an automated speech application environment to automatically provide text exchange services | |
| US11626140B2 (en) | Audio data processing method, electronic device, and storage medium | |
| CN112289307A (en) | Method, system and medium for realizing Kaldi voice recognition service based on GRPC | |
| CN106155625A (en) | A kind of audio frequency adaptation method and system | |
| WO2023160385A1 (en) | Method and apparatus for configuring audio and video conference, and server and storage medium | |
| KR20220109373A (en) | Method for providing speech video | |
| CN102473096A (en) | A method, apparatus and computer program for creating software components for computing devices | |
| CN115437611A (en) | Audio coding and decoding frame and development method based on audio coding and decoding frame | |
| WO2023142494A1 (en) | Resource data processing method and system, editor, electronic device, and storage medium | |
| KR100882381B1 (en) | Method and equipment for managing interactions in the mpeg-4 standard | |
| US20110135113A1 (en) | Apparatus and method for increasing volumn in portable terminal | |
| US20050262256A1 (en) | Method and device for multimedia processing | |
| CN114679550A (en) | Universal recording device and method | |
| Scheirer et al. | Algorithmic and wavetable synthesis in the MPEG-4 multimedia standard | |
| US11830120B2 (en) | Speech image providing method and computing device for performing the same | |
| CN111899764B (en) | Audio monitoring method and device, computer equipment and storage medium | |
| US20240046540A1 (en) | Speech image providing method and computing device for performing the same | |
| CN1533137A (en) | Access audio frequency output through capturing service | |
| CN116566963B (en) | Audio processing method and device, electronic equipment and storage medium | |
| CN114760389B (en) | Voice communication method and device, computer storage medium and electronic equipment | |
| US20220050702A1 (en) | Virtualization for audio capture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |