CN113082710A - Audio playing method and device, electronic equipment and storage medium - Google Patents
Audio playing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113082710A CN113082710A CN202110441066.4A CN202110441066A CN113082710A CN 113082710 A CN113082710 A CN 113082710A CN 202110441066 A CN202110441066 A CN 202110441066A CN 113082710 A CN113082710 A CN 113082710A
- Authority
- CN
- China
- Prior art keywords
- audio
- frame
- frames
- audio frames
- sending
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/50—Controlling the output signals based on the game progress
- A63F13/54—Controlling the output signals based on the game progress involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/165—Management of the audio stream, e.g. setting of volume, audio stream path
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
Abstract
The invention discloses an audio playing method, an audio playing device, electronic equipment and a storage medium, wherein the method comprises the following steps: when audio processing is started, determining the number of first audio frames cached by a sound card and the number of second audio frames sent to an audio software cache by audio source equipment; calculating the total number of the audio frames by using the first number of the audio frames and the second number of the audio frames, and performing frame dropping operation on the audio frames in the audio software cache when the total number of the audio frames is greater than a first threshold value; when the frame loss operation is finished and the number of the first audio frames is smaller than a second threshold value, sending the audio frames in the audio software cache to the sound card cache; when the sending operation is finished, ending the audio processing, and playing by using the audio frame in the sound card cache; the method can effectively control the total audio quantity, relieve the audio playing delay, simultaneously ensure that the sound card caches enough audio frames for playing, and effectively improve the audio playing experience of the cloud game.
Description
Technical Field
The present invention relates to the field of audio processing, and in particular, to an audio playing method and apparatus, an electronic device, and a storage medium.
Background
With the continuous development of the streaming technology and the cloud computer technology, the cloud game can run a game on the remote terminal, and the game operation is performed on the local terminal through the streaming technology, so that the effect of playing a highly configured game on the local terminal with lower configuration is achieved. In the related art, audio data of a cloud game needs to be generated by a remote terminal and sent to audio software of a local terminal, and played by a sound card of the local terminal, but due to the influence of network congestion, the local terminal has a situation that an audio data packet cannot be received or an excessive number of audio data packets are received, so that the sound card has no audio data for playing, or the sound card needs to process a large number of audio data packets, so that playing delay is caused, and user experience of the cloud game is influenced.
Disclosure of Invention
The invention aims to provide an audio playing method, an audio playing device, electronic equipment and a storage medium, which can effectively control the total audio quantity, relieve the audio playing delay, ensure that enough audio frames which can be played are stored in a sound card cache, and finally effectively improve the audio playing experience of a cloud game.
In order to solve the above technical problem, the present invention provides an audio playing method, including:
when audio processing is started, determining the number of first audio frames cached by a sound card and the number of second audio frames sent to an audio software cache by audio source equipment;
calculating the total number of audio frames by using the first number of audio frames and the second number of audio frames, and performing frame dropping operation on the audio frames in the audio software cache when the total number of audio frames is greater than a first threshold value;
when the frame loss operation is completed and the number of the first audio frames is smaller than a second threshold value, sending the audio frames in the audio software cache to the sound card cache;
and when the sending operation is finished, finishing the audio processing and playing by using the audio frame in the sound card cache.
Optionally, the performing a frame loss operation on the audio frame in the audio software buffer includes:
according to the receiving sequence of the audio frames in the audio software cache, performing frame loss on the audio frames received latest in the audio software cache, and updating the second audio frame number and the total audio frame number;
judging whether the second audio frame number reaches an audio frame reserved value or not;
if yes, judging that the frame dropping operation is finished;
if not, continuing to execute the step of losing the latest received audio frame in the audio software cache according to the receiving sequence of the audio frames in the audio software cache until the total number of the audio frames is less than or equal to the first threshold value, and judging to finish the frame losing operation.
Optionally, the performing a sending operation of sending the audio frame in the audio software cache to the sound card cache includes:
according to the receiving sequence, sending the audio frames received earliest in the audio software cache, and updating the second audio frame quantity and the audio frame sending quantity;
judging whether the second audio frame quantity reaches the audio frame reserved value or not;
if so, judging that the sending operation is finished;
if not, continuing to execute the step of sending the audio frames received earliest in the audio software cache according to the receiving sequence until the sending number of the audio frames is equal to a third threshold value, and judging that the sending operation is finished.
Optionally, the ending the audio processing includes:
recording the end time, and setting the number of first audio frames in the sound card cache as the number of historical audio frames;
correspondingly, the determining the first audio frame number buffered by the sound card when the audio processing is started includes:
determining a starting time, and calculating the playing quantity of audio frames from the ending time to the starting time by utilizing pre-stored audio frame format data;
and calculating the first audio frame number by using the historical audio frame number and the audio frame playing number.
Optionally, before starting the audio processing, the method further includes:
acquiring the audio frame format data to the audio source device when a connection is established with the audio source device.
The present invention also provides an audio playing device, comprising:
the number determining module is used for determining the number of first audio frames cached by the sound card and the number of second audio frames sent to the audio software cache by the audio source equipment when audio processing is started;
the frame loss module is used for calculating the total number of the audio frames by utilizing the first number of the audio frames and the second number of the audio frames, and performing frame loss operation on the audio frames in the audio software cache when the total number of the audio frames is greater than a first threshold value;
the sending module is used for sending the audio frames in the audio software cache to the sound card cache when the frame loss operation is finished and the number of the first audio frames is smaller than a second threshold value;
and the playing module is used for finishing the audio processing after the sending operation is finished and playing by using the audio frame in the sound card cache.
Optionally, the frame dropping module includes:
the frame loss submodule is used for performing frame loss on the audio frame received latest in the audio software cache according to the receiving sequence of the audio frame in the audio software cache and updating the second audio frame quantity and the total audio frame quantity;
the first judging module is used for judging whether the second audio frame number reaches an audio frame reserved value or not;
the first processing module is used for judging that the frame dropping operation is finished if the frame dropping operation is finished;
and if not, continuing to execute the step of losing the latest received audio frames in the audio software cache according to the receiving sequence of the audio frames in the audio software cache, and judging that the frame losing operation is finished when the total number of the audio frames is less than or equal to the first threshold.
Optionally, the sending module includes:
the sending submodule is used for sending the audio frame received earliest in the audio software cache according to the receiving sequence and updating the second audio frame quantity and the audio frame sending quantity;
a second judgment sub-module, configured to judge whether the second number of audio frames reaches the audio frame retention value;
a third processing module, configured to determine that the sending operation is completed if the sending operation is completed;
and if not, continuing to execute the step of sending the earliest received audio frame in the audio software cache according to the receiving sequence, and judging that the sending operation is finished when the sending number of the audio frames is equal to a third threshold value.
The present invention also provides an electronic device comprising:
a memory for storing a computer program;
a processor for implementing the audio playing method as described above when executing the computer program.
The invention also provides a storage medium, wherein the storage medium stores computer-executable instructions, and when the computer-executable instructions are loaded and executed by a processor, the audio playing method is realized.
The invention provides an audio playing method, which comprises the following steps: when audio processing is started, determining the number of first audio frames cached by a sound card and the number of second audio frames sent to an audio software cache by audio source equipment; calculating the total number of audio frames by using the first number of audio frames and the second number of audio frames, and performing frame dropping operation on the audio frames in the audio software cache when the total number of audio frames is greater than a first threshold value; when the frame loss operation is completed and the number of the first audio frames is smaller than a second threshold value, sending the audio frames in the audio software cache to the sound card cache; and when the sending operation is finished, finishing the audio processing and playing by using the audio frame in the sound card cache.
It can be seen that the method firstly determines the total audio number in the sound card cache and the audio software cache, and performs frame dropping operation on the audio frames in the audio cache when the total audio number is greater than a first threshold value, so as to effectively control the total audio number in the sound card cache and the audio software cache, thereby ensuring that the sound card can efficiently process the playing of the audio frames and avoiding the interference of receiving excessive audio data packets on the audio playing; meanwhile, after the frame dropping operation is completed, the method also judges the number of the first audio frames cached by the sound card, and when the number of the first audio frames is smaller than the second threshold value, the audio frames in the audio software cache are sent to the sound card cache, so that sufficient audio frames can be effectively ensured to be always stored in the sound card cache, the delay influence of network blocking on audio playing can be effectively relieved finally, the influence of no audio data in the sound card cache on the audio playing can be effectively relieved, and the game audio playing experience can be effectively improved. The invention also provides an audio playing device, electronic equipment and a storage medium, which have the beneficial effects.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of an audio playing method according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a time corresponding to audio processing according to an embodiment of the present invention;
fig. 3 is a block diagram of an audio playing apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the related art, audio data of a cloud game needs to be generated by a remote terminal and sent to audio software of a local terminal, and played by a sound card of the local terminal, but due to the influence of network congestion, the local terminal has a situation that an audio data packet cannot be received or an excessive number of audio data packets are received, so that the sound card has no audio data for playing, or the sound card needs to process a large number of audio data packets, so that playing delay is caused, and user experience of the cloud game is influenced. In view of this, the present invention provides an audio playing method, which can effectively control the total audio amount, alleviate the audio playing delay, and ensure that the sound card cache stores enough audio frames for playing, so as to effectively improve the audio playing experience of the cloud game. Referring to fig. 1, fig. 1 is a flowchart of an audio playing method according to an embodiment of the present invention, where the method includes:
s101, when audio processing is started, determining the number of first audio frames cached by a sound card and the number of second audio frames cached by audio source equipment to audio software.
It should be noted that the embodiment of the present invention does not limit the specific manner of determining the first number of audio frames in the sound card buffer or the second number of audio frames in the audio software buffer, for example, the audio frames in the sound card buffer and the audio software buffer may be counted to obtain the corresponding number of audio frames, or an individual counter may be maintained, and when the number of audio frames changes (for example, frame loss, transmission, playing, and the like), the counter is updated in real time. Considering that the first way of determining the number of audio frames requires statistics every time audio processing is performed, the efficiency of audio statistics will be reduced, and the way of maintaining the counter will enable efficient determination of the number of audio frames, so the number of audio frames can be determined by maintaining the counter in the embodiment of the present invention. Further, it is considered that the audio software buffer is used for storing audio frame data sent by the audio source device, and when the audio frame data is transmitted, the audio frame data cannot be stably transmitted due to the influence of network factors, that is, the transmission rate of the audio frame data cannot be determined. In this case, the number of audio frames in the audio software buffer is not determined accurately by maintaining the counter, so that the second number of audio frames can be determined for the audio software buffer by sending a query to the audio software buffer or directly performing statistics in the audio software buffer. For the sound card cache, since the sending action of sending the audio frames in the audio software cache to the sound card cache is a controllable action in the embodiment of the present invention, that is, the number of the sent audio frames can be specifically controlled, the first number of audio frames in the sound card cache can be quickly and accurately determined by using the manner of maintaining the counter, and further, for the sound card cache, the first number of audio frames can be determined by using the manner of maintaining the counter.
Further, it should be noted that, at the time of starting audio processing, the audio frames in the audio software buffer are the audio frames that have completed the decoding operation. The embodiment of the present invention does not limit the specific decoding method, and may refer to the related decoding technology. The embodiment of the invention also does not limit the specific mode and the receiving mode of the audio source equipment for sending the audio frame data, and can refer to the related technologies such as a cloud computer, audio streaming and the like.
S102, calculating the total number of the audio frames by using the first number of the audio frames and the second number of the audio frames, and performing frame dropping operation on the audio frames in the audio software cache when the total number of the audio frames is greater than a first threshold value.
In the embodiment of the invention, in order to ensure that the sound card can efficiently process the playing of the audio frames, the total number of the audio frames is obtained by summing the first number of the audio frames and the second number of the audio frames, and when the number of the audio frames is greater than a first threshold value, the frame dropping operation is performed on the audio frames in the audio software cache, so that the total number of the audio frames in the sound card cache and the audio software cache can be effectively controlled, and further, the excessive audio frames which can cause delay influence on the playing can not appear in the two caches, and the delay influence on the playing caused by the excessive audio frame data received by the audio software due to network blocking can be effectively relieved. It should be noted that, the embodiment of the present invention does not limit the specific value of the first threshold, and the specific value may be set according to the actual application requirement.
Further, the embodiment of the present invention does not limit the specific frame dropping operation, for example, the number of frame dropping may be determined first, and the audio frames of the number of frame dropping may be directly dropped, or of course, a single-frame audio frame may be dropped in a loop, and the second number of audio frames and the total number of audio frames may be continuously updated in each loop until the total number of audio frames is less than or equal to the first threshold, and it is determined that the frame dropping operation is completed. In order to perform the frame dropping operation more accurately, in the embodiment of the present invention, the frame dropping operation may be performed by cyclically dropping a single frame of audio frames. It will be appreciated that the audio frame received latest in the audio software buffer may be dropped according to the order in which the audio frames are received. Further, considering that the specifically discarded audio frames of the cyclically discarded single-frame audio frames come from the audio software buffer, and the exit condition is that the total number of audio frames is less than or equal to the first threshold, however, there is a case that there are fewer audio frames in the audio software buffer and more audio frames in the sound card buffer, that is, even if all the audio frames in the audio software buffer are discarded, it cannot be determined that the total number of audio frames is less than or equal to the first threshold, at this time, in each cycle, the second number of audio frames in the audio software buffer may be determined, and when the second number of audio frames is 0, it is determined that the frame dropping operation is completed. Certainly, in order to ensure that the audio can be spliced effectively, an audio frame retention value may also be set to ensure that at least the number of audio frames that is the audio frame retention value can be retained in the audio software buffer. It should be noted that, the embodiment of the present invention is not limited to a specific audio reservation value, and the audio reservation value may be set according to a practical application requirement, in one case, the audio reservation value may be set to 1, that is, at least one frame of audio frame is stored in the audio software buffer.
In one possible case, performing a frame dropping operation on an audio frame in an audio software buffer includes:
step 11: according to the receiving sequence of the audio frames in the audio software cache, performing frame loss on the audio frames received latest in the audio software cache, and updating the number of the second audio frames and the total number of the audio frames;
step 12: judging whether the number of the second audio frames reaches an audio frame reserved value or not; if yes, go to step 13; if not, entering step 14;
step 13: judging that the frame loss operation is finished;
step 14: and continuing to execute the step of losing the latest received audio frame in the audio software cache according to the receiving sequence of the audio frames in the audio software cache until the total number of the audio frames is less than or equal to the first threshold value, and judging that the frame losing operation is finished.
It can be understood that after the audio frame received latest in the audio software buffer is dropped, the number of the second audio frames and the total number of the audio frames are reduced by one to complete the number update.
S103, when the frame loss operation is completed and the number of the first audio frames is smaller than a second threshold value, sending the audio frames in the audio software cache to the sound card cache.
In the embodiment of the invention, in order to ensure that enough audio frames available for playing are stored in the sound card cache, after the frame loss operation is completed, the number of the first audio frames is also judged, and when the number of the first audio frames is smaller than a second threshold value representing the lowest playing limit, the audio frames in the audio software cache are sent to the sound card cache. It should be noted that, the embodiment of the present invention does not limit the specific value of the second threshold, and the user may set the second threshold according to the actual application requirement.
Further, the embodiment of the present invention does not limit the specific sending operation, for example, the sending number may be determined first, and the audio frames of the sending number may be sent directly, or of course, a single frame of audio frames may be sent in a loop. In order to perform the transmitting operation more accurately, in the embodiment of the present invention, the transmitting operation may be performed by cyclically transmitting a single frame of audio frames. It will be appreciated that the audio frame received earliest in the audio software buffer may be transmitted according to the order in which the audio frames are received. Further, the embodiment of the present invention does not limit the exit condition for exiting the single frame audio frame transmission cycle, for example, the second number of audio frames may be updated in real time, and the cycle may be exited when the second number of audio frames reaches the audio reservation value or is empty; of course, it is also possible to count the number of audio frame transmissions and exit the loop when the number of audio frame transmissions reaches a third threshold value representing an upper limit of audio playback. In order to ensure that the proper amount of audio frames in the sound card buffer is ensured, and at the same time, to ensure that the necessary audio frames are retained in the audio software buffer, in the embodiment of the present invention, the exit condition for exiting the single-frame audio frame transmission loop may be that the second number of audio frames reaches the audio retention value or that the number of audio frames reaches the third threshold value representing the upper limit of audio playing. It should be noted that, the embodiment of the present invention does not limit the specific value of the third threshold, and the third threshold may be set according to the actual application requirement. Further, it will be appreciated that the audio frame received earliest in the audio software buffer may be transmitted according to the order in which the audio frames are received.
In one possible case, the sending operation of sending the audio frames in the audio software buffer to the sound card buffer is performed, and the sending operation includes:
step 21: according to the receiving sequence, sending the audio frames received earliest in the audio software cache, and updating the number of the second audio frames and the sending number of the audio frames;
step 22: judging whether the number of the second audio frames reaches an audio frame reserved value or not; if yes, go to step 23; if not, go to step 24;
step 23: judging that the sending operation is finished;
step 24: and continuing to execute the step of sending the audio frame received earliest in the audio software cache according to the receiving sequence until the sending number of the audio frames is equal to the third threshold value, and judging that the sending operation is finished.
And S104, after the sending operation is finished, ending the audio processing, and playing by using the audio frame in the sound card cache.
It should be noted that there is a time interval between each end of audio processing and the start of the next audio processing, and the sound card plays with the audio frames in the sound card buffer in the time interval, that is, the number of the first audio frames in the sound card buffer is reduced. When the first audio frame number is determined by maintaining the counter, the number of audio plays between the time intervals needs to be calculated in order to ensure that the first audio frame number is accurate.
In one possible scenario, ending the audio processing comprises:
step 31: and recording the end time, and setting the first audio frame number in the sound card cache as the historical audio frame number.
Accordingly, determining a first number of audio frames buffered by a sound card when audio processing is started comprises:
step 41: determining the starting time, and calculating the playing number of the audio frames from the ending time to the starting time by utilizing pre-stored audio frame format data;
step 42: and calculating the number of the first audio frames by utilizing the historical audio frame number and the audio frame playing number.
It should be noted that, the embodiment of the present invention does not limit the specific audio frame format data, as long as the audio frame format data can calculate the playing number of audio frames in a unit time, and further can determine the playing number of audio frames at the starting time of the ending time value. In one possible case, the audio frame format data may include audio channel number, audio bit depth, and audio sampling rate calculation, and the number of audio frames played per unit time L is audio channel number by audio bit depth by audio sampling rate.
Furthermore, it should be noted that the embodiment of the present invention does not limit the setting manner of the audio frame format data, and for example, the audio frame format data may be manually set and stored locally, or may be acquired from the audio source device. The audio frame format data may be set in a manner that is retrieved to the audio source device, taking into account that different audio source devices may have different audio frame format data.
In one possible case, before starting the audio processing, the method further includes:
step 51: audio frame format data is acquired to an audio source device when a connection is established with the audio source device.
Based on the embodiment, the method firstly determines the total audio number in the sound card cache and the audio software cache, and performs frame dropping operation on the audio frames in the audio cache when the total audio number is greater than a first threshold value, so that the total audio number in the sound card cache and the audio software cache can be effectively controlled, the sound card can be ensured to efficiently process the playing of the audio frames, and the interference of receiving excessive audio data packets on the audio playing is avoided; meanwhile, after the frame dropping operation is completed, the method also judges the number of the first audio frames cached by the sound card, and when the number of the first audio frames is smaller than the second threshold value, the audio frames in the audio software cache are sent to the sound card cache, so that sufficient audio frames can be effectively ensured to be always stored in the sound card cache, the delay influence of network blocking on audio playing can be effectively relieved finally, the influence of no audio data in the sound card cache on the audio playing can be effectively relieved, and the game audio playing experience can be effectively improved.
The following describes the audio playing method based on a specific time axis diagram. Referring to fig. 2, fig. 2 is a schematic diagram illustrating a time corresponding to audio processing according to an embodiment of the present invention. For the ith audio processing, the specific flow is as follows:
1. in the ith processing cycle, the time when the host starts the frame dropping operation is recorded as
Time, the time when the transmission operation is started is recorded as
Time, the time when the playback started is recorded as
The time of day. Moment of starting playing in i-1 th processing cycle
By the moment of beginning of frame loss in the ith processing cycle
The amount of audio play in between is
L represents the audio playback volume in unit time, and is obtained by obtaining the volume from the host side after establishing connection with the host side (i.e., audio source device) of the cloud computer system, where L is the number of audio channels, the depth of audio bits, and the sampling rate of audio, and unit time.
2. The frame loss threshold of the audio buffer is set to be T1 (first threshold), the lower threshold of the audio playing is set to be T2 (second threshold), and the upper threshold of the audio playing is set to be T3 (third threshold).
3. The audio software cache in the system is set as cache A, and the sound card cache is set as cache B. And recording the data in the cache A as DA, and recording the data in the cache B as DB. In the ith processing cycle, the data amount in the buffer B when the frame loss starts is recorded as
Is calculated as
In the ith processing cycle, the data amount in the buffer A when the frame loss starts is recorded as
If it is
Then the data in the buffer A is subjected to frame loss, and a frame loss cycle is started. In the frame loss cycle, after each frame loss, the data amount in the buffer A is updated to
And judge
Whether or not to becomeAnd if so, continuing to drop frames. In the process of frame loss, at least one frame is reserved in the buffer A.
4. Obtaining the number of frames in the buffer A after the frame loss is finished, and recording the number as
If it is
The data in the audio buffer is not fetched into the sound card.
5. If it is
The data in cache a is fetched to cache B. In the frame fetching cycle, after each frame fetching, the buffer A is updated to
The frame number is calculated as
The number of data fetching frames is at most T3 frames, and at least one frame is reserved in the buffer A.
6. And after the ith processing cycle is finished, playing according to the received audio data in the sound card.
In the following, the audio playing device, the electronic device, and the storage medium according to the embodiments of the present invention are introduced, and the audio playing device, the electronic device, and the storage medium described below and the audio playing method described above may be referred to correspondingly.
Referring to fig. 3, fig. 3 is a block diagram of an audio playing apparatus according to an embodiment of the present invention, the apparatus includes:
the number determining module 301 is configured to determine, when audio processing is started, a first audio frame number buffered by a sound card and a second audio frame number sent to an audio software buffer by an audio source device;
a frame dropping module 302, configured to calculate a total number of audio frames by using the first number of audio frames and the second number of audio frames, and perform a frame dropping operation on the audio frames in the audio software cache when the total number of audio frames is greater than a first threshold;
a sending module 303, configured to send the audio frame in the audio software cache to the sound card cache when the frame dropping operation is completed and the number of the first audio frames is smaller than the second threshold;
and the playing module 304 is configured to end audio processing after the sending operation is completed, and play the audio frame in the sound card buffer.
Optionally, the frame dropping module 302 may include:
the frame loss submodule is used for performing frame loss on the audio frame received latest in the audio software cache according to the receiving sequence of the audio frame in the audio software cache and updating the number of the second audio frames and the total number of the audio frames;
the first judgment module is used for judging whether the number of the second audio frames reaches the audio frame reserved value or not;
the first processing module is used for judging that the frame loss operation is finished if the frame loss operation is finished;
and the second processing module is used for continuing to execute the step of losing the latest received audio frame in the audio software cache according to the receiving sequence of the audio frames in the audio software cache if the number of the audio frames in the audio software cache is not greater than the first threshold, and judging that the frame losing operation is finished when the total number of the audio frames is less than or equal to the first threshold.
Optionally, the sending module 303 may include:
the sending submodule is used for sending the audio frame received earliest in the audio software cache according to the receiving sequence and updating the number of the second audio frames and the sending number of the audio frames;
the second judgment submodule is used for judging whether the number of the second audio frames reaches the audio frame reserved value or not;
the third processing module is used for judging that the sending operation is finished if the sending operation is finished;
and the fourth processing module is used for continuously executing the step of sending the audio frame received earliest in the audio software cache according to the receiving sequence if the number of the audio frames is not equal to the third threshold value, and judging that the sending operation is finished when the sending number of the audio frames is equal to the third threshold value.
Optionally, the playing module 304 may include:
the end time setting submodule is used for recording the end time and setting the number of the first audio frames in the sound card cache as the number of historical audio frames;
accordingly, the quantity determining module 301 may include:
the starting time setting submodule is used for determining the starting time and calculating the playing quantity of the audio frames from the ending time to the starting time by utilizing prestored audio frame format data;
and the quantity determining submodule is used for calculating the quantity of the first audio frames by utilizing the quantity of the historical audio frames and the playing quantity of the audio frames.
Optionally, the apparatus may further include:
and the acquisition module is used for acquiring audio frame format data from the audio source equipment when the connection with the audio source equipment is established.
An embodiment of the present invention further provides an electronic device, including:
a memory for storing a computer program;
a processor for implementing the steps of the audio playing method as described above when executing the computer program.
Since the embodiment of the electronic device portion and the embodiment of the audio playing method portion correspond to each other, please refer to the description of the embodiment of the audio playing method portion for the embodiment of the electronic device portion, which is not repeated here.
The embodiment of the present invention further provides a storage medium, where a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the audio playing method according to any of the above embodiments are implemented.
Since the embodiment of the storage medium portion and the embodiment of the audio playing method portion correspond to each other, please refer to the description of the embodiment of the audio playing method portion for the embodiment of the storage medium portion, which is not repeated here.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The above provides a detailed description of an audio playing method, an audio playing apparatus, an electronic device and a storage medium. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Claims (10)
1. An audio playing method, comprising:
when audio processing is started, determining the number of first audio frames cached by a sound card and the number of second audio frames sent to an audio software cache by audio source equipment;
calculating the total number of audio frames by using the first number of audio frames and the second number of audio frames, and performing frame dropping operation on the audio frames in the audio software cache when the total number of audio frames is greater than a first threshold value;
when the frame loss operation is completed and the number of the first audio frames is smaller than a second threshold value, sending the audio frames in the audio software cache to the sound card cache;
and when the sending operation is finished, finishing the audio processing and playing by using the audio frame in the sound card cache.
2. The audio playing method of claim 1, wherein the performing a frame loss operation on the audio frames in the audio software buffer comprises:
according to the receiving sequence of the audio frames in the audio software cache, performing frame loss on the audio frames received latest in the audio software cache, and updating the second audio frame number and the total audio frame number;
judging whether the second audio frame number reaches an audio frame reserved value or not;
if yes, judging that the frame dropping operation is finished;
if not, continuing to execute the step of losing the latest received audio frame in the audio software cache according to the receiving sequence of the audio frames in the audio software cache until the total number of the audio frames is less than or equal to the first threshold value, and judging to finish the frame losing operation.
3. The audio playing method according to claim 2, wherein the performing of the sending operation of sending the audio frames in the audio software buffer to the sound card buffer includes:
according to the receiving sequence, sending the audio frames received earliest in the audio software cache, and updating the second audio frame quantity and the audio frame sending quantity;
judging whether the second audio frame quantity reaches the audio frame reserved value or not;
if so, judging that the sending operation is finished;
if not, continuing to execute the step of sending the audio frames received earliest in the audio software cache according to the receiving sequence until the sending number of the audio frames is equal to a third threshold value, and judging that the sending operation is finished.
4. The audio playing method according to claim 1, wherein the ending the audio processing comprises:
recording the end time, and setting the number of first audio frames in the sound card cache as the number of historical audio frames;
correspondingly, the determining the first audio frame number buffered by the sound card when the audio processing is started includes:
determining a starting time, and calculating the playing quantity of audio frames from the ending time to the starting time by utilizing pre-stored audio frame format data;
and calculating the first audio frame number by using the historical audio frame number and the audio frame playing number.
5. The audio playback method of claim 4, further comprising, before starting the audio processing:
acquiring the audio frame format data to the audio source device when a connection is established with the audio source device.
6. An audio playback apparatus, comprising:
the number determining module is used for determining the number of first audio frames cached by the sound card and the number of second audio frames sent to the audio software cache by the audio source equipment when audio processing is started;
the frame loss module is used for calculating the total number of the audio frames by utilizing the first number of the audio frames and the second number of the audio frames, and performing frame loss operation on the audio frames in the audio software cache when the total number of the audio frames is greater than a first threshold value;
the sending module is used for sending the audio frames in the audio software cache to the sound card cache when the frame loss operation is finished and the number of the first audio frames is smaller than a second threshold value;
and the playing module is used for finishing the audio processing after the sending operation is finished and playing by using the audio frame in the sound card cache.
7. The audio playing apparatus of claim 6, wherein the frame loss module comprises:
the frame loss submodule is used for performing frame loss on the audio frame received latest in the audio software cache according to the receiving sequence of the audio frame in the audio software cache and updating the second audio frame quantity and the total audio frame quantity;
the first judging module is used for judging whether the second audio frame number reaches an audio frame reserved value or not;
the first processing module is used for judging that the frame dropping operation is finished if the frame dropping operation is finished;
and if not, continuing to execute the step of losing the latest received audio frames in the audio software cache according to the receiving sequence of the audio frames in the audio software cache, and judging that the frame losing operation is finished when the total number of the audio frames is less than or equal to the first threshold.
8. The audio playback device of claim 7, wherein the sending module comprises:
the sending submodule is used for sending the audio frame received earliest in the audio software cache according to the receiving sequence and updating the second audio frame quantity and the audio frame sending quantity;
a second judgment sub-module, configured to judge whether the second number of audio frames reaches the audio frame retention value;
a third processing module, configured to determine that the sending operation is completed if the sending operation is completed;
and if not, continuing to execute the step of sending the earliest received audio frame in the audio software cache according to the receiving sequence, and judging that the sending operation is finished when the sending number of the audio frames is equal to a third threshold value.
9. An electronic device, comprising:
a memory for storing a computer program;
a processor for implementing the audio playback method as claimed in any one of claims 1 to 5 when executing the computer program.
10. A storage medium having stored thereon computer-executable instructions which, when loaded and executed by a processor, carry out an audio playback method as claimed in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110441066.4A CN113082710A (en) | 2021-04-23 | 2021-04-23 | Audio playing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110441066.4A CN113082710A (en) | 2021-04-23 | 2021-04-23 | Audio playing method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113082710A true CN113082710A (en) | 2021-07-09 |
Family
ID=76679699
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110441066.4A Pending CN113082710A (en) | 2021-04-23 | 2021-04-23 | Audio playing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113082710A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102362269A (en) * | 2008-12-05 | 2012-02-22 | 社会传播公司 | Realtime kernel |
| CN104113777A (en) * | 2014-08-01 | 2014-10-22 | 广州金山网络科技有限公司 | Audio stream decoding method and device |
| CN106775551A (en) * | 2016-10-31 | 2017-05-31 | 乐视控股（北京）有限公司 | Audio frequency playing method and system |
| CN107393556A (en) * | 2017-07-17 | 2017-11-24 | 京东方科技集团股份有限公司 | A kind of method and device for realizing audio frequency process |
| CN110007892A (en) * | 2014-04-11 | 2019-07-12 | 微软技术许可有限责任公司 | Audio Signal Processing |
| CN112206520A (en) * | 2020-10-21 | 2021-01-12 | 深圳市欢太科技有限公司 | Real-time audio acquisition method, system, server, client and storage medium |
-
2021
- 2021-04-23 CN CN202110441066.4A patent/CN113082710A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102362269A (en) * | 2008-12-05 | 2012-02-22 | 社会传播公司 | Realtime kernel |
| CN110007892A (en) * | 2014-04-11 | 2019-07-12 | 微软技术许可有限责任公司 | Audio Signal Processing |
| CN104113777A (en) * | 2014-08-01 | 2014-10-22 | 广州金山网络科技有限公司 | Audio stream decoding method and device |
| WO2016015670A1 (en) * | 2014-08-01 | 2016-02-04 | 广州金山网络科技有限公司 | Audio stream decoding method and device |
| CN106775551A (en) * | 2016-10-31 | 2017-05-31 | 乐视控股（北京）有限公司 | Audio frequency playing method and system |
| CN107393556A (en) * | 2017-07-17 | 2017-11-24 | 京东方科技集团股份有限公司 | A kind of method and device for realizing audio frequency process |
| CN112206520A (en) * | 2020-10-21 | 2021-01-12 | 深圳市欢太科技有限公司 | Real-time audio acquisition method, system, server, client and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20100208605A1 (en) | Method and device for processing network time delay characteristics | |
| ES2624910T3 (en) | Client side sequence switching | |
| CN104038846B (en) | Buffer status method of estimation and equipment | |
| CN112929741B (en) | Video frame rendering method and device, electronic equipment and storage medium | |
| CN110198495A (en) | A kind of method, apparatus, equipment and the storage medium of video download and broadcasting | |
| MX2011005782A (en) | Audio/video data play control method and apparatus. | |
| CN105611328A (en) | Video data accelerated downloading method and device based on HLS (HTTP Live Streaming)stream media | |
| US10389785B2 (en) | Method for adaptively streaming an audio/visual material | |
| CN106911927B (en) | Method and device for evaluating experience quality of network video user and DPI equipment | |
| KR20170012461A (en) | Method for reducing pre-fetching of multimedia streaming data with minimal impact on playback user experience | |
| CN110022498A (en) | A kind of method and device for realizing code rate switching | |
| CN101207822A (en) | Method for synchronization of audio frequency and video frequency of stream media terminal | |
| CN109379168B (en) | Voice smooth playing method for front-end real-time voice chat | |
| CN101232611A (en) | Image process apparatus and method thereof | |
| CN101702676A (en) | Data buffering process and device | |
| CN105828109B (en) | Server, client and the play system based on RTSP/RTP | |
| CN106791994A (en) | A kind of low delay quickly starts broadcasting method and device | |
| CN108494702B (en) | Transmission control method and apparatus, storage medium, and electronic apparatus | |
| CN107205159A (en) | The method for bitrate adjustment and device of Streaming Media in a kind of network direct broadcasting | |
| CN107770124A (en) | A kind of dynamic control method and device of ip voice buffering area | |
| CN113082710A (en) | Audio playing method and device, electronic equipment and storage medium | |
| US8976758B2 (en) | Wireless communication system and method for adjusting used depth of buffer unit in wireless communication system | |
| CN113409801B (en) | Noise processing method, system, medium and device for real-time audio stream playing | |
| CN115086732B (en) | Method and device for synchronizing audio and video data | |
| CN103354588A (en) | Determination method, apparatus and system for recording and playing sampling rate |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |