CN111757168B - Audio decoding method, device, storage medium and equipment - Google Patents
Audio decoding method, device, storage medium and equipment Download PDFInfo
- Publication number
- CN111757168B CN111757168B CN201910247138.4A CN201910247138A CN111757168B CN 111757168 B CN111757168 B CN 111757168B CN 201910247138 A CN201910247138 A CN 201910247138A CN 111757168 B CN111757168 B CN 111757168B
- Authority
- CN
- China
- Prior art keywords
- decoding
- audio data
- data
- audio
- decoded
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4398—Processing of audio elementary streams involving reformatting operations of audio signals
Abstract
The application discloses an audio decoding method, an audio decoding device, a storage medium and equipment, and belongs to the technical field of multimedia. The method comprises the following steps: after the multimedia file is started to play, audio data to be decoded in a first time length after the playing starts are obtained; decoding the audio data based on a software decoding mode, and verifying the obtained first decoded audio data to obtain first verification data; decoding processing is performed on the audio data based on a hardware decoding mode, and verification processing is performed on the obtained second decoded audio data to obtain second verification data; and when the first check data is matched with the second check data, decoding processing is performed on the audio data to be decoded after the first time length based on a hardware decoding mode. The method and the device for decoding the audio signals further avoid the problem of audio noise caused by decoding errors on the basis of improving decoding efficiency.
Description
Technical Field
The present application relates to the field of multimedia technologies, and in particular, to an audio decoding method, apparatus, storage medium, and device.
Background
With the increasing demand for images and sound, high-definition digital televisions and digital movies should have not only high-quality video images but also high-quality stereo audio. In view of this demand, audio is often recorded by digital methods at present instead of the original analog recording method. However, for digital audio, the data volume is usually very large, which not only needs to occupy a large amount of storage space, but also needs to occupy a large bandwidth when transmitting. In order to solve the above two problems, an audio compression technique is usually adopted to compress the original audio data, which is also called compression encoding; accordingly, in the playing process, data restoration, also called audio decoding, is performed on the compressed audio data, wherein the decoding process corresponds to the encoding process, and the decoding process and the encoding process are inverse processes to each other.
Disclosure of Invention
The embodiment of the application provides an audio decoding method, an audio decoding device, a storage medium and audio decoding equipment, and the problem of audio noise caused by decoding errors is solved. The technical scheme is as follows:
in one aspect, an audio decoding method is provided, the method including:
after the multimedia file is started to play, audio data to be decoded in a first time length after the playing starts are obtained;
decoding the audio data based on a software decoding mode, and verifying the obtained first decoded audio data to obtain first verification data;
decoding the audio data based on a hardware decoding mode, and verifying the obtained second decoded audio data to obtain second verification data;
and when the first check data is matched with the second check data, decoding the audio data to be decoded after the first time length based on the hardware decoding mode.
In another aspect, an audio decoding apparatus is provided, where the apparatus includes:
the first acquisition module is used for acquiring audio data to be decoded within a first time length after the start of playing after the multimedia file is started to be played;
the first processing module is used for executing decoding processing on the audio data based on a software decoding mode;
the verification module is used for performing verification processing on the obtained first decoding audio data to obtain first verification data;
the second processing module is used for executing decoding processing on the audio data based on a hardware decoding mode;
the check module is further configured to perform check processing on the obtained second decoded audio data to obtain second check data;
and the second processing module is further configured to, when the first check data matches the second check data, perform decoding processing on the audio data to be decoded after the first duration based on the hardware decoding manner.
In a possible implementation manner, the second processing module is further configured to, when the third check data matches the fourth check data, continue to perform, based on the hardware decoding manner, decoding processing on the audio data to be decoded after the current decoding progress.
In a possible implementation manner, the first obtaining module is further configured to perform audio-video data separation on the multimedia file to obtain audio data of the multimedia file; and acquiring the audio data to be decoded in the first time length from the audio data of the multimedia file.
In a possible implementation manner, the first processing module is further configured to, when the first check data and the second check data do not match, perform, based on the software decoding manner, decoding processing on the audio data to be decoded after the first duration.
In another aspect, a storage medium is provided, where at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement the above-mentioned audio decoding method.
In another aspect, an audio decoding apparatus is provided, which includes a processor and a memory, where at least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor to implement the audio decoding method described above.
The technical scheme provided by the embodiment of the application has the following beneficial effects:
in the embodiment of the application, the audio decoding device can flexibly adjust the decoding mode, for example, at the beginning of playing, the audio decoding device can acquire audio data to be decoded within a period of time, and respectively adopt a software decoding mode and a hardware decoding mode to perform decoding processing on the part of the audio data, check the obtained two pieces of decoded audio data, and if the check is correct, indicate that the hardware decoding mode is correct currently, so that the hardware decoding mode is adopted to perform decoding processing on the subsequent audio data to be decoded.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic diagram of an implementation environment related to an audio decoding method provided by an embodiment of the present application;
fig. 2 is a schematic structural diagram of an audio decoding apparatus according to an embodiment of the present application;
fig. 3 is a schematic diagram of an audio decoding process provided in an embodiment of the present application;
fig. 4 is a flowchart of an audio decoding method provided in an embodiment of the present application;
fig. 5 is a flowchart of another audio decoding method provided in an embodiment of the present application;
fig. 6 is a flowchart of another audio decoding method provided in an embodiment of the present application;
fig. 7 is a flowchart of another audio decoding method provided in an embodiment of the present application;
FIG. 8 is a diagram of a video frame according to an embodiment of the present application;
FIG. 9 is a diagram of a PCM data according to an embodiment of the present application;
FIG. 10 is a waveform diagram of an output of PCM data according to an embodiment of the present application;
fig. 11 is a schematic structural diagram of an audio decoding apparatus according to an embodiment of the present application;
fig. 12 is a schematic structural diagram of an audio decoding apparatus according to an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, the following detailed description of the embodiments of the present application will be made with reference to the accompanying drawings.
Before explaining embodiments of the present application in detail, some terms that may be referred to in the embodiments of the present application are explained.
Audio frame: a frame, also called a data frame.
An audio frame is a segment of fixed-length or unfixed-length data block generated after audio data with a certain length is compressed by a certain specific compression algorithm. Such a block of data is called a frame.
As an example, the playing duration of an audio frame may be 23.32ms (millisecond), 46.64ms, or 128ms, etc., which is not specifically limited in the embodiment of the present application.
The above-mentioned audio data before compression processing is referred to herein as PCM (Pulse Code modulation) data.
In addition, for an audio file, the audio file is composed of a plurality of audio frames and frame headers after being encoded and compressed. The frame header includes some descriptive information of the audio frame, such as coding type, channel number, and the like. Illustratively, for an Audio-video file, such as an MP3(MPEG Audio Layer3, Audio dynamic compression Layer three) file or an MP4(MPEG Audio Layer4, Audio dynamic compression Layer four) file, the Audio portion thereof includes a large number of Audio frames.
Based on the above description, an audio frame is a basic unit in encoding and decoding in the audio and video technology.
And (3) software decoding, namely decoding the compressed audio data by running a software program by means of a Central Processing Unit (CPU), namely a decoding algorithm part is realized by running the CPU. Therefore, when software decodes, operations such as addition, subtraction, multiplication, division and the like included in the decoding algorithm are realized by the capability of the CPU alone. Compared with hardware decoding, when the audio frame with certain complexity is decoded by software decoding, the load and power consumption of a CPU (central processing unit) are high, and the time consumption for decoding one frame is relatively long; however, the software decoding mode is flexible, codes are visible, and if the decoding is abnormal, the problem can be solved in time through software upgrading easily.
Hardware decoding: the decoding algorithm part which is time-consuming is operated by a non-CPU form hardware decoding unit which is integrated on a chip so as to realize the decoding of the audio data. The CPU typically only assists in providing some necessary description information to the hardware decoding unit and in controlling the decoding process. Compared with software decoding, the decoding efficiency of hardware decoding is higher, the operation speed is higher, the power consumption is lower, and the occupancy rate of a CPU is lower; however, the hardware decoding method is not flexible enough, and generally, the hardware decoding method is not scalable, and once decoding errors caused by hardware design limitations or bugs occur, the decoding is difficult to remedy.
In another expression mode, hardware decoding is performed through hardware, so that the workload of a CPU (Central processing Unit) can be reduced, and the power consumption can be reduced; the software decoding is performed by the CPU occupied by the software program itself, so that the workload of the CPU is increased compared with the hardware decoding.
MD 5: message Digest Algorithm 5, fifth edition of the Message Digest Algorithm.
The MD5 is a hash function widely used in the field of computer security, and is used to provide integrity protection for files, and to operate files into a fixed length value, which is the basic principle of hash algorithm. MD5 has the advantages of compressibility, easy calculation, modification resistance, strong impact resistance and the like.
As an example, MD5, a widely used hash function, may generate a 128-bit (16-byte) hash value (hash value) to ensure that information is transmitted in a complete and consistent manner. Alternatively, MD5 may generate a unique digital fingerprint for any file (regardless of size, format, or number), and if someone modifies the file, the corresponding MD5 value, i.e., the corresponding digital fingerprint, will change. A typical application of MD5 is to generate a message digest for a piece of information to prevent tampering.
Illustratively, the server provides a MD5 value in advance for a file, and after the user finishes downloading the file, the MD5 algorithm recalculates the MD5 value of the downloaded file, and by comparing whether the two MD values are the same, it can be determined whether the downloaded file is erroneous or not, or whether the downloaded file is tampered.
Audio noise: in the embodiment of the present application, the audio noise is a generic term for noise and abnormal audio. After the PCM data generated by normal decoding is subjected to digital-to-analog conversion, if the PCM data is heard by human ears, no abnormal sound is generated, namely audio noise is proved not to be generated; on the contrary, if an error occurs in the decoding process, abnormal data is generated, and after the abnormal data is analog-to-digital converted, noise or erroneous sound information can be heard by human ears, i.e. audio noise is generated.
A passage: i.e., pipeline, in the embodiments of the present application, a path refers to a data or message flow in an audio decoding process.
In order to improve the decoding efficiency, a hardware decoding method is generally used to perform decoding processing on compressed audio data. However, when a hardware decoding method is used to perform decoding, decoding errors may occur, and once decoding errors occur, abnormal data may be generated, and the abnormal data may form audio noise after being output through digital-to-analog conversion. There is currently no effective solution for audio noise due to decoding errors. Therefore, how to perform audio decoding to radically solve the problem of audio noise caused by decoding errors becomes a problem to be solved by those skilled in the art.
Stated another way, the related art has no effective solution to audio noise generated due to decoding errors, and relies on either peripheral filtering hardware or software noise reduction algorithms to remove audio noise during audio decoding. However, in any of the above-mentioned noise reduction methods, it is difficult to completely eliminate the audio noise, and the original sound effect is often affected to some extent.
In order to solve the problem of audio noise caused by decoding errors, if hardware decoding is started in the playing process of a multimedia file, a parallel software decoding daemon thread is also started. As one example, the multimedia file refers herein to an audio-video file. The daemon thread starts the decoding function within a period of time after the playing starts, and periodically starts the decoding function during the playing process.
When software decoding is started, performing hardware decoding and software decoding on audio data to be decoded; then, the decoded audio data generated by hardware decoding and software decoding are respectively checked; then, comparing the two check data, if the two check data are consistent, determining that a hardware decoding mode is not abnormal, and decoding all subsequent audio data to be decoded in a hardware decoding mode; if the two check data are not consistent, the embodiment of the application judges that the hardware decoding mode has errors, the hardware decoding needs to be stopped, and the hardware decoding mode is switched to the software decoding mode, so that the problem of audio noise possibly generated is avoided.
Based on the above description, it can be seen that, in the embodiments of the present application, by flexibly performing decoding mode adjustment and performing checksum determination on decoded audio data obtained by performing decoding processing, the problem of decoding errors that may occur is fundamentally avoided with low CPU overhead, so that audio noise is further avoided.
The following describes an implementation environment related to an audio decoding method provided in an embodiment of the present application.
Referring to fig. 1, the audio and video technology often involves an encoding and decoding process, and the audio data in a compressed form is obtained after the acquired original audio data in an analog form is encoded by the audio encoding device 1. After the audio data in the compressed form is decoded by the audio decoding device 2, PCM data is output to complete the restoration of the data.
The audio data output by the audio decoding device 2 in fig. 1 is in digital form. The collected original audio data in analog form is usually subjected to sampling, quantization, pulse code modulation, compression coding and other processing in sequence by the audio coding apparatus 1, and the audio data in compressed form can be obtained after the coding processing. The audio data output by the audio decoding device 2 can be heard by human ears after being processed by digital-to-analog conversion and the like.
The audio decoding method provided in the embodiment of the present application is applied to the audio decoding device 2, where the audio decoding device 2 is a computer device, and the computer device may be a mobile device such as a smart phone, a tablet computer, a notebook computer, or a fixed device such as a personal computer and a server, and this is not particularly limited in the embodiment of the present application.
Referring to fig. 2, the audio decoding apparatus 2 in fig. 1 includes the following modules: the device comprises a data input module 201, a software decoding module 202, a hardware decoding module 203, a verification module 204 and a data output module 205.
Wherein, the data input module 201 is configured to:
1. in the beginning stage of playing the multimedia file, the software decoding module 202 and the hardware decoding module 203 are started simultaneously.
2. And simultaneously sending the audio data to be decoded within a period of time after the playing starts to the software decoding module 202 and the hardware decoding module 203, and checking whether the hardware decoding mode of the hardware decoding module 203 is wrong or not based on the decoding result output by the software decoding module 202 and the decoding result output by the hardware decoding module 203.
3. If the verification is finished and the hardware decoding mode is correct, all the subsequent audio data to be decoded are sent to the hardware decoding module 203, and the software decoding module 202 stops working temporarily, that is, the software decoding daemon thread is suspended temporarily.
Thereafter, the data input module 201 periodically selects a portion of the audio frame to send to the software decoding module 202, and starts a software decoding daemon thread.
4. If the data input module 201 determines that the hardware decoding mode is incorrect, the hardware decoding is directly stopped, and all subsequent audio data to be decoded are decoded in a software decoding mode.
If the hardware decoding mode is correct, the temporary suspension software decoding daemon thread of the point 3 above is repeatedly executed, a part of audio frames are periodically selected to be sent to the software decoding module 202, and the software decoding module 202 is started to decode.
The lifecycle of the software decoding daemon is accompanied by the whole playing process of the multimedia file, and the software decoding module 202 is used for providing a software decoding function in time according to needs. Wherein, the meaning in due time is: when the check judgment is needed or the hardware decoding is determined to be wrong, the audio decoding mode needs to be switched to the software decoding mode.
In the embodiment of the present application, the hardware decoding module 203 is used for providing a hardware decoding function. As an example, taking an audio decoding device as an intelligent mobile terminal as an example, for the intelligent mobile terminal installed with different types of operating systems, the hardware decoding modules of the intelligent mobile terminal are also different, for example, for the intelligent mobile terminal installed with an Android operating system, a mediacodec is generally used as the hardware decoding module; for an intelligent mobile terminal installed with an ios operating system, an AudioToolBox is generally used as a hardware decoding module, which is not specifically limited in this embodiment of the present application.
As an example, referring to fig. 3, the verification module 204 includes an MD5 generation module 2041 and an MD5 verification module 2042.
The arrow labeled "path 1" in fig. 3 represents the data or message flow during software decoding, and indicates the flow of the audio frame after being input into the software decoding module 202 to the decoding completion and outputting the decoding result (PCM data). The arrow marked with "path 2" in fig. 3 represents the data or message flow during hardware decoding, and indicates the flow of the audio frame after being input into the hardware decoding module 203 to the decoding result.
When the audio data is only sent to the software decoding module 202 or the hardware decoding module 203, no check is needed, and the MD5 generation and check steps are skipped, i.e., the MD5 generation module 2041 and the MD5 check module 2042 do not work.
When the audio data is sent to the software decoding module 202 and the hardware decoding module 203 at the same time, the MD5 generation module 2041 and the MD5 verification module 2042 operate; the MD5 generating module 2041 is configured to generate MD5 values based on decoding results generated by the software decoding module 202 and the hardware decoding module 203, respectively, and the MD5 verifying module 2042 is configured to perform verification according to the two generated MD5 values.
As shown in fig. 3, the MD5 generation module 2041 has as its input PCM data decoded by the software decoding module 202 or the hardware decoding module 203, i.e. corresponding to PCM data 1 and PCM data 2 in fig. 3, and has as its output 16 bytes (128 bits) generated based on the corresponding PCM data and suffixed with check data of MD5, i.e. corresponding to pcm1.md5 and pcm2.md5 in fig. 3.
In the embodiment of the present application, for the data output module 205, if only one path is in operation, it is not necessary to perform MD5 checksum and feed back the checksum to the data input module 201, and the decoded PCM data is directly output.
If the two paths work simultaneously and it is determined that the hardware decoding mode is incorrect after the verification is completed, the verification result is fed back to the data input module 201. In addition, when two same paths work simultaneously, the embodiment of the present application defaults to output the PCM data generated by the software decoding module 202.
Fig. 4 is a flowchart of an audio decoding method according to an embodiment of the present application. The execution subject of the method may be the audio decoding device shown in fig. 1 and fig. 2, and referring to fig. 4, the method flow provided by the embodiment of the present application includes:
401. after the multimedia file is started to play, the audio decoding equipment acquires audio data to be decoded within a first time length after the playing starts.
As an example, the multimedia file refers to an audio and video file in this embodiment, that is, the multimedia file may include both an audio portion and a video portion, or may include only an audio portion, which is not specifically limited in this embodiment.
Taking the multimedia file as an example of a video file, in order to decode the audio part contained in the multimedia file, as shown in fig. 3, the audio part and the video part of the multimedia file need to be separated (demux). The separated audio part includes the audio frame, and the separated video part includes the video frame.
In another expression, after the multimedia file is started to play, audio and video data of the multimedia file are separated to obtain audio data and video data of the multimedia file, respectively. The audio data is used for processing by adopting the audio decoding method provided by the embodiment of the application. And the video data may also be processed in a decoding manner or the like to present video pictures to the user.
The audio data mentioned herein refers to audio data to be decoded, also referred to as compressed audio data, and includes audio frames and frame headers.
As an example, the value of the first time period may be 2s (seconds) or 3s, which is not specifically limited in this embodiment of the application.
This step may be performed by the data input module 201 shown in fig. 3.
402. The audio decoding equipment performs decoding processing on audio data to be decoded in a first time length after the start of playing based on a software decoding mode, and performs verification processing on the obtained first decoded audio data to obtain first verification data.
In this embodiment, referring to fig. 3, after the audio data to be decoded in the first duration is acquired, the data input module 201 sends the portion of the audio data to be decoded to the software decoding module 202 and the hardware decoding module 203, respectively.
Wherein decoded audio data refers to PCM data in embodiments of the present application. In addition, the PCM data obtained by software decoding is also referred to as first decoded audio data herein.
As an example, the decoding process is performed on the audio data based on a software decoding manner, including but not limited to the following: acquiring the data format of the audio data; based on the target software program matching the data format, a decoding process is performed on the audio data.
That is, the software decoding system also needs to perform decoding processing by means of a decoding software program installed in the audio decoding apparatus. Wherein, for audio data of different data formats, the decoding software program required for performing the decoding process may also be different. For example, for audio data in MPEG4 format, it may be necessary to perform decoding processing using a decoding plug-in ffdshow; for Audio data in the AC3(Audio Coding3 Audio Coding 3) format, a decoding plug-in AC3fliter may be required to perform decoding processing, which is not specifically limited in this embodiment of the present application.
In a possible implementation manner, since the software decoding module 202 and the hardware decoding module 203 work in parallel, after the software decoding module 202 obtains the decoded audio data, as shown in fig. 3, the decoded audio data is sent to the MD5 generating module 2041.
Wherein, the check processing is performed on the first decoded audio data, including but not limited to the following: first check data of the first decoded audio data is generated based on the target hash function, wherein the first check data includes a hash value of the first decoded audio data. As an example, the target hash function may be the MD5 algorithm, i.e. the MD5 check is performed in the embodiment of the present application. Accordingly, for the first decoded audio data, a 128-bit (16-byte) hash value (hash value) is generated as the first check data.
403. The audio decoding device performs decoding processing on the audio data based on a hardware decoding mode, and performs verification processing on the obtained second decoded audio data to obtain second verification data.
The PCM data obtained by the hardware decoding is referred to as second decoded audio data herein.
As an example, the decoding process is performed on the audio data based on a hardware decoding manner, including but not limited to: acquiring the operating system type of the audio decoding equipment; when the type of the operating system is a first type, accessing a hardware decoding unit based on a first interface matched with the first type, and executing decoding processing on the audio data; and when the operating system type is a second type, accessing the hardware decoding unit based on a second interface matched with the second type, and executing decoding processing on the audio data.
That is, the hardware decoding method usually adopted is different according to the type of the operating system installed in the audio decoding device. For example, taking the first type as an Android operating system as an example, the first interface may be a Mediacodec, taking the second type as an ios operating system as an example, the second interface may be an AudioToolBox, which is not specifically limited in this embodiment of the present application.
In another expression, for an audio decoding device with an Android operating system installed, Mediacodec may be used to perform hardware decoding. The MediaCodec can be used to access the multimedia codec of the Android bottom layer, i.e. the above mentioned hardware decoding unit, which is part of the multimedia support infrastructure of the Android bottom layer. In addition, MediaCodec processes data in an asynchronous manner and uses a set of input-output buffers. After requesting or receiving an empty input buffer, it is filled with data and passed to the codec for processing. The codec finishes processing the data and outputs the processing result to an empty output buffer. Then, an output buffer full of result data is requested or received, the data in the output buffer is used up, and the output buffer is released to the codec for reuse.
For the audio decoding device installed with the ios operating system, the AudioToolBox may be used to perform hardware decoding, which is not specifically limited in this embodiment of the present application.
In a possible implementation manner, since the software decoding module 202 and the hardware decoding module 203 work in parallel, after the hardware decoding module 203 obtains the decoded audio data, as shown in fig. 3, the decoded audio data is sent to the MD5 generating module 2041.
In one possible implementation, the check processing is performed on the second decoded audio data, including but not limited to taking the following: generating second parity data based on the target hash function, wherein the second parity data includes a hash value of the second decoded audio data. Similarly to the first check data, a 128-bit (16-byte) hash value is also generated for the second decoded audio data.
404. And when the first check data is matched with the second check data, the audio decoding equipment executes decoding processing on the audio data to be decoded after the first time length based on a hardware decoding mode.
As shown in fig. 3, after the MD5 generating module 2041 generates the first check data and the second check data, the first check data and the second check data are sent to the MD5 checking module 2042 for checking.
In this embodiment, when the MD5 checks that there is no error, that is, the first check data matches the second check data, it indicates that the decoding result obtained by the software decoding mode matches the decoding result obtained by the hardware decoding mode, and all the subsequent audio data to be decoded may adopt the hardware decoding mode, that is, the data input module 201 may send all the subsequent audio data to be decoded to the hardware decoding module 203 shown in fig. 2 and 3 for decoding.
And when the first check data and the second check data are not matched, decoding processing is performed on the audio data to be decoded after the first time length based on a software decoding mode.
According to the method provided by the embodiment of the application, the audio decoding device can flexibly adjust the decoding mode, for example, at the beginning stage of playing, the audio decoding device can obtain the audio data to be decoded within a period of time, and respectively adopt the software decoding mode and the hardware decoding mode to perform decoding processing on the part of the audio data, and check the obtained two pieces of decoded audio data, if the check is correct, the current decoding of the hardware decoding mode is correct, so that the hardware decoding mode is adopted to perform decoding processing on the subsequent audio data to be decoded.
In another embodiment, referring to fig. 5, the embodiment of the present application further includes a step of outputting the decoding result.
405. The audio decoding apparatus outputs the obtained decoding result.
In the embodiment of the present application, the output decoding result is divided into the following two cases:
in the first case, when audio decoding is performed only based on the hardware decoding method or the software decoding method, decoded audio data obtained based on the hardware decoding method or the software decoding method is directly output as a decoding result.
That is, if only one of the path 1 and the path 2 shown in fig. 3 is in an operating state, the decoding result of the path is directly used as the output of the audio decoding apparatus.
In case two, when audio decoding is performed in parallel based on the hardware decoding method and the software decoding method, decoded audio data obtained based on the software decoding method is output as a decoding result.
That is, when both of the path 1 and the path 2 shown in fig. 3 are in the operating state, the audio decoding apparatus selects to output the decoded audio data generated by the software decoding module 202.
Depending on the type of operating system installed in the audio decoding apparatus, the manner in which the decoded result is output may vary, which generally depends on the capabilities provided by the operating system.
As an example, if the audio decoding device is installed with an Android operating system, the audio decoding device generally uses audiorack, opensl to output the decoding result; if the ios operating system is installed in the audio decoding apparatus, the decoding result is generally output by using audioqueue, audiooutlit, and the like, which is not particularly limited in this embodiment of the present application.
According to the method provided by the embodiment of the application, the decoding result output can be flexibly adjusted according to the working states of the software decoding module and the software decoding module, for example, when only one channel is in the working state, the decoding result of the channel is directly used as the output of the audio decoding equipment, and when both the channels are in the working state, the audio decoding equipment selects and outputs the decoded audio data generated by the software decoding module, so that the accuracy of the decoding result is ensured.
In another embodiment, the embodiment of the present application may also periodically detect whether the hardware decoding module is decoding normally, that is, the embodiment of the present application after step 404 and before step 405, referring to fig. 6, further includes the following processing steps:
406. after the first duration, the audio decoding apparatus periodically acquires a target amount of audio data based on a current decoding progress.
As an example, the audio decoding apparatus may perform the following determination once every 10s intervals, which is not particularly limited in the embodiment of the present application. That is, after the first time period, a certain number of audio frames are selected at intervals of 10s and sent to the software decoding module shown in fig. 2 and 3, and the software decoding daemon thread is started.
In one possible implementation, the target amount of audio data may be 10% of audio frames after the current decoding progress, which is not specifically limited in the embodiment of the present application.
407. And the audio decoding equipment performs decoding processing on the audio data of the target quantity based on a software decoding mode, and performs verification processing on the obtained third decoded audio data to obtain third verification data.
This step is similar to step 402 described above.
408. The audio decoding device performs decoding processing on the target number of audio data based on a hardware decoding mode, and performs verification processing on the obtained fourth decoded audio data to obtain fourth verification data.
This step is similar to step 403 described above.
409. And when the third check data is not matched with the fourth check data, the audio decoding equipment performs decoding processing on the audio data to be decoded after the current decoding progress based on a software decoding mode.
In the embodiment of the present application, if the MD5 check fails, that is, the third check data and the fourth check data do not match, it indicates that the current hardware decoding module is abnormal in decoding, and for such a case, the audio decoding device further needs to directly close the path 2 shown in fig. 3, and send all the audio data to be decoded after the current decoding progress to the path 1 shown in fig. 3 to execute software decoding.
In addition, if the MD5 checks that there is no error, that is, the third check data matches the fourth check data, it indicates that the current hardware decoding module is decoding normally, and for this situation, the audio decoding device continues to perform decoding processing on the audio data to be decoded after the current decoding progress based on the hardware decoding mode, and the software decoding daemon thread continues to be suspended.
For the case that MD5 checks to see if there is no error, a certain number of audio frames are selected periodically to execute the software decoding process to continue detecting whether the hardware decoding has errors.
According to the method provided by the embodiment of the application, the audio decoding device can be flexibly adjusted in a decoding mode, for example, at the beginning stage of playing, the audio decoding device can acquire audio data to be decoded within a period of time, and respectively adopt a software decoding mode and a hardware decoding mode to perform decoding processing on the part of audio data, and check the two obtained decoded audio data, if the check is correct, the current decoding of the hardware decoding mode is correct, so that the hardware decoding mode is adopted to perform decoding processing on the subsequent audio data to be decoded, and on the basis of improving the decoding efficiency, the problem of audio noise caused by decoding errors is also avoided.
In addition, in the subsequent process, the audio decoding equipment also can periodically select part of audio data to be decoded by adopting a software decoding mode, and check and compare the part of audio data with the hardware decoding result of the part of audio data, if the part of audio data is checked to be wrong, the hardware decoding mode is indicated to have decoding errors, so that the subsequent un-decoded audio data is switched to adopt the software decoding mode to execute decoding processing.
The embodiment of the application fundamentally avoids the problem of decoding errors possibly occurring with lower CPU expense, thereby further avoiding audio noise, flexibly adjusting the decoding strategy, and simultaneously considering the respective advantages of a software decoding mode and a hardware decoding mode, and the decoding mode has better effect.
In another embodiment, taking an audio and video file as an example, the following combines with fig. 3 to comb the sorting execution flow of the audio decoding method provided in the embodiment of the present application. Referring to fig. 7, the overall process includes:
701. and separating the audio part and the video part of the audio and video file to obtain an audio frame and a video frame.
702. In the playing starting stage, the data input module acquires audio frames within a period of time after the playing starts and respectively sends the audio frames into the software decoding module and the hardware decoding module.
703. The software decoding module performs decoding processing on the part of the audio frame by adopting a software decoding mode to obtain PCM data 1.
704. The hardware decoding module performs decoding processing on the part of the audio frame by adopting a hardware decoding mode to obtain PCM data 2.
705. The software decoding module sends PCM data 1 to the MD5 generation module.
706. The hardware decode module sends PCM data 2 to the MD5 generation module.
707. The MD5 generation module generates check data pcm1.md5 for PCM data 1 and generates check data pcm2.md5 for PCM data 1.
708. The MD5 generation module sends PCM1.MD5 and PCM2.MD5 to the MD5 verification module.
709. After the MD5 verification module completes verification, the MD5 verification module feeds back a verification result to the data input module, and outputs a decoding result through the data output module.
710. When pcm1.md5 and pcm2.md5 are consistent, the data input module sends all the subsequent audio frames into the hardware decoding module, and the software decoding module stops working temporarily.
711. And the data input module periodically selects part of the audio frames to be sent to the software decoding module, and starts the software decoding module to decode so as to check whether the hardware decoding mode is wrong.
In another embodiment, an application scenario of the embodiment of the present application is described below.
The audio decoding method provided by the embodiment of the disclosure can be applied to any scene needing audio decoding.
In an example, the method may be applied to a pure audio playing scenario, for example, a user downloads audio data of a song from a server through an intelligent mobile terminal, and the intelligent mobile terminal serving as an audio decoding device may perform audio playing by executing the audio decoding method provided in the embodiment of the present application.
In example two, the method can be applied to a video playing scene shown in fig. 8, for example, a user needs to play a segment of video through an intelligent mobile terminal, and after an audio part and a video part in a video file are separated, the intelligent mobile terminal serving as an audio decoding device can play audio in a video playing process by executing the audio decoding method provided in the embodiment of the present application.
Referring to fig. 8 to 10, in fig. 8, a frame of picture in a video playing process is shown, taking an example that an Android platform is built in an audio decoding device, when an audio is decoded by adopting a hardware decoding method of Mediacodec on the Android platform, a problem that some slice sources generate abnormal PCM data after hardware decoding may occur, for example, 4 bytes of PCM data have a periodic error every 8192 bytes, and the periodic error can be usually avoided when software decoding is executed.
Fig. 9 is a comparison between the PCM data generated by the software decoding method and the hardware decoding method, wherein the left side of fig. 9 is the normal PCM data output by the software decoding, and the right side of fig. 9 is the abnormal PCM data output by the hardware decoding. In addition, as shown in fig. 10, the two pairs of output waveforms corresponding to the PCM data have noise points in the PCM data output by hardware decoding, as can be seen from fig. 10.
Fig. 11 is a schematic structural diagram of an audio decoding apparatus according to an embodiment of the present application. Referring to fig. 11, the apparatus includes:
the first obtaining module 1101 is configured to, after the multimedia file is started to be played, obtain audio data to be decoded within a first duration after the playing starts;
a first processing module 1102, configured to perform decoding processing on the audio data based on a software decoding manner;
a checking module 1103, configured to perform checking processing on the obtained first decoded audio data to obtain first checking data;
a second processing module 1104, configured to perform decoding processing on the audio data based on a hardware decoding manner;
the checking module 1103 is further configured to perform checking processing on the obtained second decoded audio data to obtain second checking data;
the second processing module 1104 is further configured to, when the first parity data matches the second parity data, perform decoding processing on the audio data to be decoded after the first duration based on the hardware decoding manner.
The device provided by the embodiment of the application can be adjusted by flexibly performing a decoding mode, for example, at the beginning of playing, the audio decoding equipment can acquire audio data to be decoded within a period of time, and respectively perform decoding processing on the part of the audio data by adopting a software decoding mode and a hardware decoding mode, and check the obtained two pieces of decoded audio data, if the check is correct, the current decoding mode of the hardware decoding mode is indicated, so that the decoding processing is performed on the subsequent audio data to be decoded by adopting the hardware decoding mode.
In one possible implementation, the apparatus further includes:
a second obtaining module 1105, configured to obtain a target amount of audio data periodically based on the current decoding progress after the first duration;
the first processing module 1102 is further configured to perform decoding processing on the target amount of audio data based on the software decoding manner;
the verifying module 1103 is further configured to perform verification processing on the obtained third decoded audio data to obtain third verification data;
the second processing module 1104 is further configured to perform decoding processing on the target amount of audio data based on the hardware decoding manner;
the checking module 1103 is further configured to perform checking processing on the obtained fourth decoded audio data to obtain fourth checking data;
the first processing module 1102 is further configured to, when the third check data does not match the fourth check data, perform decoding processing on the audio data to be decoded after the current decoding progress based on the software decoding manner.
In a possible implementation manner, the second processing module 1104 is further configured to, when the third check data matches the fourth check data, continue to perform, based on the hardware decoding manner, decoding processing on the audio data to be decoded after the current decoding progress.
In a possible implementation manner, the first processing module 1102 is further configured to obtain a data format of the audio data; and executing decoding processing on the audio data based on a target software program matched with the data format.
In a possible implementation manner, the second processing module 1104 is further configured to obtain an operating system type of the audio decoding apparatus; when the operating system type is a first type, accessing a hardware decoding unit based on a first interface matched with the first type, and executing decoding processing on the audio data; and when the operating system type is a second type, accessing the hardware decoding unit based on a second interface matched with the second type, and executing decoding processing on the audio data.
In one possible implementation, the apparatus further includes:
an output module 1106, configured to output, as a decoding result, decoded audio data obtained based on the software decoding manner when decoding processing is executed in parallel based on the hardware decoding manner and the software decoding manner.
In a possible implementation manner, the first obtaining module 1101 is further configured to perform audio-video data separation on the multimedia file to obtain audio data of the multimedia file; and acquiring the audio data to be decoded in the first time length from the audio data of the multimedia file.
All the above optional technical solutions may be combined arbitrarily to form the optional embodiments of the present disclosure, and are not described herein again.
In the audio decoding apparatus provided in the above embodiment, only the division of the above functional modules is used for illustration when decoding the audio, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the above described functions. In addition, the audio decoding apparatus and the audio decoding method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments and are not described herein again.
Fig. 12 shows a block diagram of an audio decoding apparatus according to an exemplary embodiment of the present application.
The device 1200 may be a portable mobile terminal, such as: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, motion video Experts compression standard Audio Layer 3), an MP4 player (Moving Picture Experts Group Audio Layer IV, motion video Experts compression standard Audio Layer 4), a notebook computer, or a desktop computer. Device 1200 may also be referred to by other names as user equipment, portable terminals, laptop terminals, desktop terminals, and the like.
In general, the apparatus 1200 includes: a processor 1201 and a memory 1202.
The processor 1201 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 1201 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 1201 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 1201 may be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing content that the display screen needs to display. In some embodiments, the processor 1201 may further include an AI (Artificial Intelligence) processor for processing a computing operation related to machine learning.
In some embodiments, a non-transitory computer readable storage medium in the memory 1202 is used to store at least one instruction for execution by the processor 1201 to implement the audio decoding methods provided by the method embodiments herein.
Those skilled in the art will appreciate that the configuration shown in FIG. 12 is not intended to be limiting of the apparatus 1200, and that the apparatus 1200 may include more components than shown, or some components may be combined, or a different arrangement of components may be used.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only a preferred embodiment of the present application and should not be taken as limiting the present application, and any modifications, equivalents, improvements and the like that are made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (12)
1. An audio decoding method, characterized in that the method comprises:
after the multimedia file is started to be played, audio data to be decoded within a first time length after the playing starts are obtained;
decoding the audio data based on a software decoding mode, and verifying the obtained first decoded audio data to obtain first verification data;
decoding the audio data based on a hardware decoding mode, and verifying the obtained second decoded audio data to obtain second verification data;
when the first check data is matched with the second check data, decoding processing is carried out on the audio data to be decoded after the first time length based on the hardware decoding mode;
the method further comprises the following steps:
after the first time period, periodically acquiring a target amount of audio data based on the current decoding progress;
performing decoding processing on the target number of audio data based on the software decoding mode to obtain third decoded audio data; generating third check data of the third decoded audio data based on a target hash function, wherein the third check data comprises a hash value of the third decoded audio data;
performing decoding processing on the target number of audio data based on the hardware decoding mode to obtain fourth decoded audio data; generating fourth check data of the fourth decoded audio data based on the target hash function, wherein the fourth check data comprises a hash value of the fourth decoded audio data;
when the third check data is not matched with the fourth check data, decoding processing is performed on the audio data to be decoded after the current decoding progress based on the software decoding mode;
and when the third check data is matched with the fourth check data, continuously executing decoding processing on the audio data to be decoded after the current decoding progress based on the hardware decoding mode.
2. The method according to claim 1, wherein the performing of decoding processing on the audio data based on the software decoding manner includes:
acquiring the data format of the audio data;
and executing decoding processing on the audio data based on a target software program matched with the data format.
3. The method according to claim 1, wherein the performing a decoding process on the audio data based on the hardware decoding manner includes:
acquiring the operating system type of the audio decoding equipment;
when the operating system type is a first type, accessing a hardware decoding unit based on a first interface matched with the first type, and executing decoding processing on the audio data;
and when the operating system type is a second type, accessing the hardware decoding unit based on a second interface matched with the second type, and executing decoding processing on the audio data.
4. The method according to any one of claims 1 to 3, further comprising:
and when decoding processing is executed in parallel based on the hardware decoding mode and the software decoding mode, outputting decoded audio data obtained based on the software decoding mode as a decoding result.
5. The method of any of claims 1 to 3, wherein the obtaining the audio data to be decoded within a first time period after the playing starts comprises:
performing audio and video data separation on the multimedia file to obtain audio data of the multimedia file;
and acquiring the audio data to be decoded in the first time duration from the audio data of the multimedia file.
6. The method according to any one of claims 1 to 3, further comprising:
and when the first check data is not matched with the second check data, decoding the audio data to be decoded after the first duration based on the software decoding mode.
7. An audio decoding apparatus, characterized in that the apparatus comprises:
the first acquisition module is used for acquiring audio data to be decoded within a first time length after the start of playing after the multimedia file is started to be played;
the first processing module is used for executing decoding processing on the audio data based on a software decoding mode;
the verification module is used for performing verification processing on the obtained first decoding audio data to obtain first verification data;
the second processing module is used for executing decoding processing on the audio data based on a hardware decoding mode;
the check module is further configured to perform check processing on the obtained second decoded audio data to obtain second check data;
the second processing module is further configured to, when the first parity data matches the second parity data, perform decoding processing on the audio data to be decoded after the first duration based on the hardware decoding manner;
a second obtaining module, configured to periodically obtain, after the first time period, a target amount of audio data based on a current decoding progress;
the first processing module is further configured to perform decoding processing on the target number of audio data based on the software decoding manner to obtain third decoded audio data;
the check module is further configured to generate third check data of the third decoded audio data based on a target hash function, where the third check data includes a hash value of the third decoded audio data;
the second processing module is further configured to perform decoding processing on the target number of audio data based on the hardware decoding manner to obtain fourth decoded audio data;
the check module is further configured to generate fourth check data of the fourth decoded audio data based on the target hash function, where the fourth check data includes a hash value of the fourth decoded audio data;
the first processing module is further configured to, when the third check data does not match the fourth check data, perform decoding processing on the audio data to be decoded after the current decoding progress based on the software decoding manner;
the second processing module is further configured to, when the third check data matches the fourth check data, continue to perform decoding processing on the audio data to be decoded after the current decoding progress based on the hardware decoding manner.
8. The apparatus of claim 7, wherein the first processing module is further configured to obtain a data format of the audio data; and executing decoding processing on the audio data based on a target software program matched with the data format.
9. The apparatus of claim 7, wherein the second processing module is further configured to obtain an operating system type of the audio decoding device; when the operating system type is a first type, accessing a hardware decoding unit based on a first interface matched with the first type, and executing decoding processing on the audio data; and when the operating system type is a second type, accessing the hardware decoding unit based on a second interface matched with the second type, and executing decoding processing on the audio data.
10. The apparatus of any one of claims 7 to 9, further comprising:
and the output module is used for outputting the decoded audio data obtained based on the software decoding mode as a decoding result when the decoding processing is executed in parallel based on the hardware decoding mode and the software decoding mode.
11. A storage medium having stored therein at least one instruction which is loaded and executed by a processor to implement the audio decoding method of any one of claims 1 to 6.
12. Audio decoding device, characterized in that it comprises a processor and a memory, in which at least one instruction is stored, which is loaded and executed by said processor to implement an audio decoding method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910247138.4A CN111757168B (en) | 2019-03-29 | 2019-03-29 | Audio decoding method, device, storage medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910247138.4A CN111757168B (en) | 2019-03-29 | 2019-03-29 | Audio decoding method, device, storage medium and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111757168A CN111757168A (en) | 2020-10-09 |
| CN111757168B true CN111757168B (en) | 2022-08-19 |
Family
ID=72671751
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910247138.4A Active CN111757168B (en) | 2019-03-29 | 2019-03-29 | Audio decoding method, device, storage medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111757168B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107948735A (en) * | 2017-12-06 | 2018-04-20 | 北京金山安全软件有限公司 | Video playing method and device and electronic equipment |
| CN108389582A (en) * | 2016-12-12 | 2018-08-10 | 中国航空工业集团公司西安航空计算技术研究所 | MPEG-2/4AAC audio decoders error detection and processing method |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7321623B2 (en) * | 2002-10-01 | 2008-01-22 | Avocent Corporation | Video compression system |
| CN101350625B (en) * | 2007-07-18 | 2011-08-31 | 北京泰美世纪科技有限公司 | High-efficiency all-purpose decoder for QC-LDPC code and decoding method thereof |
| KR20090126829A (en) * | 2008-06-05 | 2009-12-09 | 삼성전자주식회사 | Iterative decoding method and iterative decoding apparatus |
| GB2466984B (en) * | 2009-01-16 | 2011-07-27 | Imagination Tech Ltd | Multi-threaded data processing system |
| CN102667921B (en) * | 2009-10-20 | 2014-09-10 | 弗兰霍菲尔运输应用研究公司 | Audio encoder, audio decoder, method for encoding an audio information, method for decoding an audio information |
| CN101847409B (en) * | 2010-03-25 | 2012-01-25 | 北京邮电大学 | Voice integrity protection method based on digital fingerprint |
| CN102361469B (en) * | 2011-06-21 | 2013-10-30 | 北京交大思诺科技有限公司 | Device and method for parallel decoding of software and hardware |
| TWI500038B (en) * | 2012-09-28 | 2015-09-11 | Univ Nat Chiao Tung | Fully parallel encoding method and fully parallel decoding method of memory system |
| JP2016143970A (en) * | 2015-01-30 | 2016-08-08 | 株式会社リコー | Image processing apparatus, image processing system and image processing program |
| CN104980788B (en) * | 2015-02-11 | 2018-08-07 | 腾讯科技（深圳）有限公司 | Video encoding/decoding method and device |
| CN106331765B (en) * | 2015-06-30 | 2020-06-05 | 腾讯科技（深圳）有限公司 | Hard solution testing method, terminal, server and computer readable storage medium |
| CN108282164B (en) * | 2017-01-05 | 2022-07-15 | 腾讯科技（深圳）有限公司 | Data encoding and decoding method and device |
| CN107172432A (en) * | 2017-03-23 | 2017-09-15 | 杰发科技（合肥）有限公司 | A kind of method for processing video frequency, device and terminal |
| CN109391843B (en) * | 2017-08-03 | 2022-02-25 | 腾讯科技（深圳）有限公司 | Online video speed doubling playing method, device, medium and intelligent terminal |
| CN107547984A (en) * | 2017-10-11 | 2018-01-05 | 上海展扬通信技术有限公司 | A kind of audio-frequency inputting method and audio output system based on intelligent terminal |
-
2019
- 2019-03-29 CN CN201910247138.4A patent/CN111757168B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108389582A (en) * | 2016-12-12 | 2018-08-10 | 中国航空工业集团公司西安航空计算技术研究所 | MPEG-2/4AAC audio decoders error detection and processing method |
| CN107948735A (en) * | 2017-12-06 | 2018-04-20 | 北京金山安全软件有限公司 | Video playing method and device and electronic equipment |
Non-Patent Citations (2)
| Title |
|---|
| hardware and software co-design for the AAC audio decoder;RC Sampaio;《SBCCI》;20121112;全文 * |
| LDPC码的研究及其解码器的设计和优化;戴剑臻;《中国优秀硕士学位论文全文数据库》;20070615;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111757168A (en) | 2020-10-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11336953B2 (en) | Video processing method, electronic device, and computer-readable medium | |
| US8583818B2 (en) | System and method for custom segmentation for streaming video | |
| JP2022519855A (en) | Video stream decryption methods, devices, terminals and programs | |
| EP2981955B1 (en) | Layered audio coding and transmission | |
| US11490173B2 (en) | Switch of audio and video | |
| CN104837052A (en) | Playing method of multimedia data and device | |
| TW201334518A (en) | Audio/video playing device, audio/video processing device, systems, and method thereof | |
| CN104159150A (en) | Cloud terminal, cloud server, media data stream playing system and method | |
| JP2016522601A (en) | Operations that are aware of the syntax of media files in container format | |
| WO2009064067A1 (en) | System and method for producing importance rate-based rich media, and server applied to the same | |
| CN106688015B (en) | Processing parameters for operations on blocks when decoding images | |
| CN109495776B (en) | Audio sending and playing method and intelligent terminal | |
| US20140082678A1 (en) | Video server and method for restarting rebuilding | |
| EP3693961A1 (en) | Encoding device and method, decoding device and method, and program | |
| CN111757168B (en) | Audio decoding method, device, storage medium and equipment | |
| US8873641B2 (en) | Moving picture coding apparatus | |
| US9009561B2 (en) | System and method for detecting errors in audio data | |
| CN110855645B (en) | Streaming media data playing method and device | |
| US20100076944A1 (en) | Multiprocessor systems for processing multimedia data and methods thereof | |
| CN113613058A (en) | Local storage method, equipment and medium for network video stream | |
| CN111246215A (en) | Video format conversion method and terminal | |
| KR20080073077A (en) | Device and method for accessing information in fusion memory | |
| CN106899757A (en) | The control method for playing back of audio-video document, device and terminal device | |
| CN111356003B (en) | Data writing method, system and terminal equipment | |
| KR20190101579A (en) | Reconfigurable Video System for Multi-Channel Ultra High Definition Video Processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40031399 Country of ref document: HK |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |