CN106658135A - Audio and video playing method and device - Google Patents
Audio and video playing method and device Download PDFInfo
- Publication number
- CN106658135A CN106658135A CN201611234859.4A CN201611234859A CN106658135A CN 106658135 A CN106658135 A CN 106658135A CN 201611234859 A CN201611234859 A CN 201611234859A CN 106658135 A CN106658135 A CN 106658135A
- Authority
- CN
- China
- Prior art keywords
- audio
- time
- video
- playing
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 230000001360 synchronised effect Effects 0.000 claims abstract description 49
- 230000003139 buffering effect Effects 0.000 claims description 45
- 238000005070 sampling Methods 0.000 claims description 14
- 238000004806 packaging method and process Methods 0.000 claims description 9
- 238000005538 encapsulation Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 abstract description 6
- 230000008569 process Effects 0.000 description 5
- 230000009471 action Effects 0.000 description 3
- 230000005236 sound signal Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4392—Processing of audio elementary streams involving audio buffer management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
The embodiment of the invention discloses an audio and video playing method and device which are applied to a multimedia player in an audio direct mode. The method comprises the steps of: at an interval of a preset synchronization duration or every when a preset synchronization time point is reached, calculating current buffer time of an audio buffer region; according to the buffer time and a first target timestamp, determining audio play time; according to the audio play time and a second target timestamp, determining whether audio frames currently stored in the audio buffer region and video frames currently stored in a video buffer region meet synchronous playing conditions; and if yes, pushing the video frames currently stored in the video buffer region to video equipment for playing, and pushing the audio frames currently stored in the audio buffer region to audio equipment for playing. By applying the audio and video playing method and device disclosed by the embodiment of the invention, the audio frames and the video frames, which meet the synchronous playing conditions, are simultaneously played, synchronous playing of an audio and a video when the multimedia player is in the audio direct mode is implemented, and a playing effect of the multimedia player is improved.
Description
Technical Field
The invention relates to the technical field of multimedia, in particular to an audio and video playing method and device.
Background
With the development of internet technology, channels for users to acquire multimedia information are more and more convenient, various multimedia players have a trend of rapid development, and audio and video synchronous playing is an important content for reflecting the service quality of the multimedia players.
The audio and video synchronization refers to maintaining the time synchronization relationship between the audio frame and the video frame, and can be realized by reducing the play time difference between the audio frame and the video frame. Because each audio frame is provided with an audio time stamp, and each video frame is provided with a video time stamp, the audio and video synchronization method mainly uses a time stamp synchronization method, namely, a reference clock is selected firstly (the time on the reference clock is required to be linearly increased); generating the stream of audiovisual data by time stamping each block of audio data and each block of video data with respect to a time on a reference clock (typically including a start time and an end time); at play, the time stamp on the data block is read while the play is scheduled with reference to the time on the current reference clock: if the starting time of the data block is larger than the time on the current reference clock, the data block is not urgently played until the reference clock reaches the starting time of the data block; if the starting time of the data block is less than the time on the current reference clock, the data block is played as soon as possible so that the playing progress catches up with the reference clock.
The multimedia player can play the received audio frame and video frame after decoding, and plays the audio frame through the audio equipment and plays the video frame through the video equipment. When the multimedia player processes audio frames, there are two main ways: one is that the multimedia player needs to decode and restore the audio frame into original audio data, i.e. PCM (Pulse Code Modulation) audio data, and then push the PCM audio data to the audio device for playing, and the decoding is completed by a decoder built in the player; the other is that the multimedia player does not need to decode and restore the audio frames to the original audio data, and directly outputs the audio frames to the audio device, and the audio device performs decoding and playing by itself, which is also called as an audio through output mode.
For a multimedia player in an audio through output mode, it is a technical difficulty to realize audio and video synchronous playing. At present, some multimedia players do not perform audio and video synchronization processing, for example: few multimedia players support the synchronous processing of dolby EAC3 audio. Therefore, the existing multimedia player lacks a playing method for realizing audio and video synchronization in an audio direct-through output mode, so that the playing effect of the multimedia player is poor.
Disclosure of Invention
The embodiment of the invention aims to provide an audio and video playing method and device, so that an audio frame and a video frame of a multimedia player can be synchronously played in an audio direct connection mode, and the playing effect of the multimedia player is improved.
In order to achieve the above object, an embodiment of the present invention discloses an audio and video playing method and device, which are applied to a multimedia player in an audio direct mode, and the method includes:
calculating the buffering time of a current audio buffer area every preset synchronization time length or every time a preset synchronization time point is reached, wherein audio frames stored in the current audio buffer area are audio frames packaged according to a preset packaging standard;
determining audio playing time according to the buffering time and a first target timestamp, wherein the first target timestamp is the timestamp of the audio frame with the smallest timestamp in the audio frames currently stored in the audio buffer area;
determining whether the audio frame stored in the current audio buffer area and the video frame stored in the current video buffer area meet a synchronous playing condition or not according to the audio playing time and a second target timestamp, wherein the second target timestamp is the timestamp of the video frame with the smallest timestamp in the video frames currently stored in the video buffer area;
and if so, pushing the video frame currently stored in the video buffer area to the video equipment for playing, and pushing the audio frame currently stored in the audio buffer area to the audio equipment for playing.
Optionally, the preset packaging standard is as follows: IEC61937 standard; the audio frame is: audio frames based on the EAC3 format.
Optionally, the calculating the buffering time of the current audio buffer includes:
according to the formula:calculating the buffering time of the current audio buffer area;
wherein, T is the buffering time of the current audio buffer, M is the total number of bytes of all audio frames stored in the audio buffer, S is the sampling rate for an audio frame, and N is the number of bytes of a single audio frame.
Optionally, the determining the audio playing time according to the buffering time and the first target timestamp includes:
and determining the sum of the buffering time and the time corresponding to the first target timestamp as an audio playing time.
Optionally, the determining, according to the audio playing time and the second target timestamp, whether the audio frame currently stored in the audio buffer and the video frame currently stored in the video buffer satisfy a synchronous playing condition includes:
and judging whether the audio playing time is greater than the time corresponding to the second target timestamp, and if so, determining that the audio frame stored in the current audio buffer area and the video frame stored in the current video buffer area meet the synchronous playing condition.
In order to achieve the above object, an embodiment of the present invention further discloses an audio/video playing device, which is applied to a multimedia player in an audio direct mode, and the device includes:
the computing module is used for computing the buffering time of the current audio buffer area every preset synchronous time or every time the preset synchronous time point is reached, wherein the audio frames stored in the current audio buffer area are the audio frames encapsulated according to the preset encapsulation standard;
a first determining module, configured to determine an audio playing time according to the buffering time and a first target timestamp, where the first target timestamp is a timestamp of an audio frame with a smallest timestamp among audio frames currently stored in the audio buffer;
a second determining module, configured to determine, according to the audio playing time and a second target timestamp, whether an audio frame currently stored in the audio buffer and a video frame currently stored in the video buffer satisfy a synchronous playing condition, where the second target timestamp is a timestamp of a video frame with a smallest timestamp in the video frames currently stored in the video buffer;
and the playing module is used for pushing the video frame currently stored in the video buffer area to the video equipment for playing and pushing the audio frame currently stored in the audio buffer area to the audio equipment for playing under the condition that the audio frame currently stored in the audio buffer area and the video frame currently stored in the video buffer area meet the synchronous playing condition.
Optionally, the preset packaging standard is as follows: IEC61937 standard; the audio frame is: audio frames based on the EAC3 format.
Optionally, the calculation module is specifically configured to:
every preset synchronous time or every time a preset synchronous time point is reached, according to the formula:calculating the buffering time of the current audio buffer area;
wherein, T is the buffering time of the current audio buffer, M is the total number of bytes of all audio frames stored in the audio buffer, S is the sampling rate for an audio frame, and N is the number of bytes of a single audio frame.
Optionally, the first determining module is specifically configured to:
and determining the sum of the buffering time and the time corresponding to the first target timestamp as the audio playing time according to the buffering time and the first target timestamp.
Optionally, the second determining module is specifically configured to:
and judging whether the audio playing time is greater than the time corresponding to the second target timestamp or not according to the audio playing time and the second target timestamp, and if so, determining that the audio frame stored in the current audio buffer area and the video frame stored in the current video buffer area meet the synchronous playing condition.
According to the technical scheme, the audio and video playing method and device provided by the embodiment of the invention calculate the buffer time of the current audio buffer area every preset synchronous time or every time the preset synchronous time point is reached, wherein the audio frames stored in the current audio buffer area are the audio frames which are encapsulated according to the preset encapsulation standard; determining audio playing time according to the buffering time and a first target timestamp, wherein the first target timestamp is the timestamp of the audio frame with the smallest timestamp in the audio frames currently stored in the audio buffer area; determining whether an audio frame stored in a current audio buffer area and a video frame stored in a current video buffer area meet a synchronous playing condition or not according to the audio playing time and a second target timestamp, wherein the second target timestamp is the timestamp of the video frame with the smallest timestamp in the video frames currently stored in the video buffer area; and if so, pushing the video frame currently stored in the video buffer area to the video equipment for playing, and pushing the audio frame currently stored in the audio buffer area to the audio equipment for playing.
By applying the embodiment of the invention, the audio frame and the video frame meeting the synchronous playing condition are simultaneously played, the synchronous playing of audio and video of the multimedia player in an audio direct mode is realized, and the playing effect of the multimedia player is improved.
Of course, it is not necessary for any product or method of practicing the invention to achieve all of the above-described advantages at the same time.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of an audio/video playing method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of an audio/video playing device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention discloses an audio and video playing method and device, which are applied to a multimedia player in an audio direct mode and are respectively explained in detail below.
Referring to fig. 1, fig. 1 is a schematic flow chart of an audio and video playing method provided by an embodiment of the present invention, including the following steps:
s101, calculating the buffer time of the current audio buffer area every preset synchronous time length or every time when the preset synchronous time point is reached.
It should be noted that the audio frames stored in the current audio buffer are audio frames encapsulated according to a preset encapsulation standard.
Exemplarily, assuming that the preset synchronization duration is 600ms, the buffering time of the current audio buffer is calculated every 600 ms; or, assuming that the audio time duration is 2 hours, and the preset synchronization time points are respectively the 10 th minute, the 30 th minute, the 60 th minute, the 90 th minute and the 105 th minute of the audio, the buffer time of the current audio buffer is calculated every time the 10 th minute, the 30 th minute, the 60 th minute, the 90 th minute and the 105 th minute are reached.
Specifically, the preset packaging standard is as follows: according to the IEC61937 standard, audio frames are: audio frames based on the EAC3 format. The IEC61937 standard is designed by the International Electrotechnical Commission (IEC) and can be used as an International standard for encapsulating audio frames, so that audio frames encapsulated according to the IEC61937 standard have commonality, and audio devices for playing audio frames all support decapsulation and decoding playback of audio frames encapsulated according to the IEC61937 standard. The audio format refers to an encoding technology of an audio file, and there are various implementations, for example: AC3(Audio Coding-3, Audio Coding 3), AAC (Advanced Audio Coding), EAC3(Enhanced AC-3), etc., and the sampling rates and the codec modes corresponding to Audio frames of different formats are different. The audio frame is encapsulated according to the preset encapsulation standard, a packet header field is added in front of the audio frame, and the audio equipment can obtain the audio format of the audio frame and the start-stop byte position of the audio frame without decoding the audio frame by acquiring the packet header information, so that the decoding speed and the accuracy of the audio frame are improved.
For example: an audio frame in the format of EAC3 is encapsulated according to the IEC61937 standard, and a header is added before the audio frame in the format of EAC3, where the length of the header is 8 bytes, and the header includes Pa, Pb, Pc, and Pd that respectively occupy two bytes, where Pa and Pb are synchronization bytes, Pc is Burst information used for recording the data type, and Pd is length information of payload. When the Pa value is detected to be 0xF872 and the Pb value is detected to be 0x4E1F, the start position of the audio frame can be obtained as the position of the Pa byte, when the Pc value is detected to be 0x0015 (i.e. 21), the audio format of the audio frame can be determined to be the EAC3 format, the length of the audio frame can be determined to be 6144 x4, and the end position of the audio frame can be obtained by calculating the sum of the position of the 8 th byte of the packet header and the 6144 x 4.
In practical applications, the buffering time of the current audio buffer is calculated according to the formula:calculating the buffering time of the current audio buffer area;
wherein, T is the buffering time of the current audio buffer, M is the total number of bytes of all audio frames stored in the audio buffer, S is the sampling rate for the audio frame, and N is the number of bytes of a single audio frame.
It should be noted that, calculating the buffering time of the current audio buffer may calculate the buffering time T of the current audio buffer, which is a × B, according to the number a of audio frames of the current audio buffer and the buffering time B of each audio frame, wherein,m is audio bufferingThe total byte number of all the audio frames stored in the area, wherein L is the number of sampling samples corresponding to one audio frame after being packaged according to a preset packaging standard, and N is the byte number of a single audio frame; wherein,s is the sample rate for the audio frame and, thus,
compared with the existing multimedia player in the audio pass-through mode, the calculation buffer time formula for the non-encapsulated audio frames is as follows:m is the total byte number of all audio frames stored in the audio buffer area, and P is the byte number of an unpackaged audio frame.
It should be noted that the acquired original sound signal is an analog signal that changes continuously, and therefore, the analog signal needs to be sampled, quantized, and encoded to obtain a digital signal, which is convenient for the multimedia player to transmit and process audio data. Since the frequency range of the audio signal audible to the human ear is 20Hz-20KHZ, the maximum bandwidth of the audio is 20KHZ, and the waveform can be completely and truly restored by sampling twice as often as a sine wave according to NYQUIST's (NYQUIST) sampling theorem. Therefore, in order to restore a digital signal to an analog signal, it is necessary to set the sampling rate to 40KHz or more for an audio signal. Typically, the sampling rate for audio frames in EAC3 format is 48KHz, with a 48KHz sampling rate representing 48000 data describing a 1 second sound waveform.
Assuming that the total byte number of all audio frames stored in the audio buffer is 100KB, the sampling rate for audio frames is 48KHz, the byte number of a single audio frame is 4B, and the buffering time of the current audio buffer is 521 ms.
S102, determining audio playing time according to the buffering time and the first target timestamp.
It should be noted that the first target timestamp is a timestamp of an audio frame with a smallest timestamp among audio frames currently stored in the audio buffer, the audio frames are sequentially stored in the audio buffer from small to large according to the timestamps, and are also sequentially played from small to large according to the timestamps when played, so that the audio frame corresponding to the first target timestamp is the audio frame that is played first in the audio buffer, and the remaining audio frames are sequentially played from small to large according to the timestamps.
Specifically, according to the buffering time and the first target timestamp, the sum of the buffering time and the time corresponding to the first target timestamp is determined as the audio playing time.
Assuming that the buffering time of the current audio buffer is 521ms and the time corresponding to the first target timestamp is 23ms, 521+23 is 544ms, which is determined as the audio playing time.
S103, determining whether the audio frame stored in the current audio buffer area and the video frame stored in the current video buffer area meet the synchronous playing condition or not according to the audio playing time and the second target timestamp, and executing S104 if the audio frame stored in the current audio buffer area and the video frame stored in the current video buffer area meet the synchronous playing condition.
It should be noted that the second target timestamp is a timestamp of a video frame with a smallest timestamp in video frames currently stored in the video buffer, that is, a video frame corresponding to the second target timestamp is a video frame played first in the video buffer.
Specifically, according to the audio playing time and the second target timestamp, it is determined whether the audio frame currently stored in the audio buffer area and the video frame currently stored in the video buffer area satisfy a synchronous playing condition, and the determination may be:
and judging whether the audio playing time is greater than the time corresponding to the second target timestamp, and if so, determining that the audio frame stored in the current audio buffer area and the video frame stored in the current video buffer area meet the synchronous playing condition.
It should be noted that, under the condition that the audio playing time is greater than the time corresponding to the second target timestamp, further, it may be considered whether a difference between the audio playing time and the time corresponding to the second target timestamp is less than a preset threshold, and if so, it is determined that the audio frame stored in the current audio buffer and the video frame stored in the current video buffer satisfy the synchronous playing condition.
It can be understood that the time corresponding to the second target timestamp is the playing time of the video frame corresponding to the second target timestamp, the audio playing time is selected as the reference time, and the relative time relationship between the audio frame and the video frame is determined by establishing the corresponding relationship between the audio frame playing time and the reference time and the corresponding relationship between the video frame timestamp and the reference time, so that the playing time of the audio frame and the video frame is determined, and the audio and video are synchronously played. When the audio playing time is reached, the audio frame with the smallest timestamp stored in the current audio buffer area is to be played, and when the audio playing time is greater than the time corresponding to the second target timestamp, it is indicated that the video frame lags behind the audio frame at the moment, and the video frame corresponding to the second target timestamp needs to be played immediately, so that the audio and video synchronous playing can be realized. The corresponding relationship between the audio frame timestamp and the reference time and the corresponding relationship between the video frame timestamp and the reference time are established, which belong to the prior art and are not described herein again.
If the audio playing time is 544ms, the time corresponding to the second target timestamp is 520ms, and the preset threshold is 100ms, it may be determined that the audio frame stored in the current audio buffer area and the video frame stored in the current video buffer area satisfy the synchronous playing condition according to the fact that the audio playing time 544ms is greater than the time 520ms corresponding to the second target timestamp; or, further considering the difference between the audio playing time and the time corresponding to the second target timestamp as: 544ms-520ms is 24ms, which is less than 100ms, it may be determined that the audio frame stored in the current audio buffer and the video frame stored in the current video buffer satisfy the synchronized playing condition.
And S104, pushing the video frame currently stored in the video buffer area to the video equipment for playing, and pushing the audio frame currently stored in the audio buffer area to the audio equipment for playing.
It should be noted that the audio device and the video device may be the same device, for example: a television set is provided for playing both audio and video frames. Alternatively, the audio device and the video device may be two separate devices, for example: the audio equipment is power amplifier equipment, and the video equipment is a display. Because the embodiment of the invention is applied to the multimedia player in the audio direct mode, the audio frame currently stored in the audio buffer area is an undecoded audio frame, and the audio frame currently stored in the audio buffer area needs to be pushed to the audio equipment, so that the audio equipment can play the audio frame after decapsulation and decoding.
For example, assuming that the audio device is a power amplifier device and the video device is a display, a digital audio/video interface may be used, for example: the HDMI (High Definition Multimedia Interface) or SPDIF (Sony/Philips Digital Interface Format) transmits the audio frame to the power amplifier device, so that the power amplifier device unpacks, decodes and plays the audio frame, and may also transmit the video frame to the display device by means of another Digital audio/video Interface, so that the display device plays the video frame.
For example, assuming that the audio device and the video device are televisions, the audio frame and the video frame may be transmitted to the televisions via the above digital audio/video interfaces, so that the televisions play the video frame, and the audio frame is de-encapsulated and decoded for playing.
Therefore, by applying the embodiment of the invention, the audio frame and the video frame meeting the synchronous playing condition are simultaneously played, the synchronous playing of the audio and the video of the multimedia player in the audio direct mode is realized, and the playing effect of the multimedia player is improved.
Corresponding to the above method embodiment, the embodiment of the present invention further provides an audio/video playing device.
Referring to fig. 2, fig. 2 is a schematic structural diagram of an audio/video playing device provided in an embodiment of the present invention, including:
a calculating module 201, configured to calculate a buffering time of a current audio buffer at intervals of a preset synchronization duration or each time a preset synchronization time point is reached, where an audio frame stored in the current audio buffer is an audio frame encapsulated according to a preset encapsulation standard;
a first determining module 202, configured to determine an audio playing time according to the buffering time and a first target timestamp, where the first target timestamp is a timestamp of an audio frame with a smallest timestamp in audio frames currently stored in the audio buffer;
a second determining module 203, configured to determine, according to the audio playing time and a second target timestamp, whether an audio frame currently stored in the audio buffer and a video frame currently stored in the video buffer satisfy a synchronous playing condition, where the second target timestamp is a timestamp of a video frame with a smallest timestamp in the video frames currently stored in the video buffer;
the playing module 204 is configured to push the video frame currently stored in the video buffer area to the video device for playing and push the audio frame currently stored in the audio buffer area to the audio device for playing under the condition that the audio frame currently stored in the audio buffer area and the video frame currently stored in the video buffer area meet a synchronous playing condition.
The preset packaging standard of the embodiment of the invention is as follows: according to the IEC61937 standard, the audio frame of the embodiment of the invention is as follows: audio frames based on the EAC3 format.
The calculating module 201 is specifically configured to:
every preset synchronization duration or everyWhen the preset synchronous time point is reached, according to a formula:calculating the buffering time of the current audio buffer area;
wherein, T is the buffering time of the current audio buffer, M is the total number of bytes of all audio frames stored in the audio buffer, S is the sampling rate for the audio frame, and N is the number of bytes of a single audio frame.
The first determining module 202 is specifically configured to:
and determining the sum of the buffering time and the time corresponding to the first target timestamp as the audio playing time according to the buffering time and the first target timestamp.
The second determining module 203 is specifically configured to:
and judging whether the audio playing time is greater than the time corresponding to the second target timestamp or not according to the audio playing time and the second target timestamp, and if so, determining that the audio frame stored in the current audio buffer area and the video frame stored in the current video buffer area meet the synchronous playing condition.
Therefore, by applying the embodiment of the invention, the audio frame and the video frame meeting the synchronous playing condition are simultaneously played, the synchronous playing of the audio and the video of the multimedia player in the audio direct mode is realized, and the playing effect of the multimedia player is improved.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
Those skilled in the art will appreciate that all or part of the steps in the above method embodiments may be implemented by a program to instruct relevant hardware to perform the steps, and the program may be stored in a computer-readable storage medium, which is referred to herein as a storage medium, such as: ROM/RAM, magnetic disk, optical disk, etc.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (10)
1. An audio and video playing method is applied to a multimedia player in an audio through mode, and the method comprises the following steps:
calculating the buffering time of a current audio buffer area every preset synchronization time length or every time a preset synchronization time point is reached, wherein audio frames stored in the current audio buffer area are audio frames packaged according to a preset packaging standard;
determining audio playing time according to the buffering time and a first target timestamp, wherein the first target timestamp is the timestamp of the audio frame with the smallest timestamp in the audio frames currently stored in the audio buffer area;
determining whether the audio frame stored in the current audio buffer area and the video frame stored in the current video buffer area meet a synchronous playing condition or not according to the audio playing time and a second target timestamp, wherein the second target timestamp is the timestamp of the video frame with the smallest timestamp in the video frames currently stored in the video buffer area;
and if so, pushing the video frame currently stored in the video buffer area to the video equipment for playing, and pushing the audio frame currently stored in the audio buffer area to the audio equipment for playing.
2. The method of claim 1, wherein the predetermined packaging criteria is: IEC61937 standard; the audio frame is: audio frames based on the EAC3 format.
3. The method of claim 1, wherein calculating the buffering time of the current audio buffer comprises:
according to the formula:calculating the buffering time of the current audio buffer area;
wherein, T is the buffering time of the current audio buffer, M is the total number of bytes of all audio frames stored in the audio buffer, S is the sampling rate for an audio frame, and N is the number of bytes of a single audio frame.
4. The method of claim 1, wherein determining an audio playback time based on the buffering time and a first target timestamp comprises:
and determining the sum of the buffering time and the time corresponding to the first target timestamp as an audio playing time.
5. The method according to any one of claims 1 to 4, wherein said determining whether the audio frame currently stored in the audio buffer and the video frame currently stored in the video buffer satisfy a synchronized playback condition according to the audio playback time and a second target timestamp comprises:
and judging whether the audio playing time is greater than the time corresponding to the second target timestamp, and if so, determining that the audio frame stored in the current audio buffer area and the video frame stored in the current video buffer area meet the synchronous playing condition.
6. An audio/video playing device, applied to a multimedia player in an audio pass-through mode, the device comprising:
the computing module is used for computing the buffering time of the current audio buffer area every preset synchronous time or every time the preset synchronous time point is reached, wherein the audio frames stored in the current audio buffer area are the audio frames encapsulated according to the preset encapsulation standard;
a first determining module, configured to determine an audio playing time according to the buffering time and a first target timestamp, where the first target timestamp is a timestamp of an audio frame with a smallest timestamp among audio frames currently stored in the audio buffer;
a second determining module, configured to determine, according to the audio playing time and a second target timestamp, whether an audio frame currently stored in the audio buffer and a video frame currently stored in the video buffer satisfy a synchronous playing condition, where the second target timestamp is a timestamp of a video frame with a smallest timestamp in the video frames currently stored in the video buffer;
and the playing module is used for pushing the video frame currently stored in the video buffer area to the video equipment for playing and pushing the audio frame currently stored in the audio buffer area to the audio equipment for playing under the condition that the audio frame currently stored in the audio buffer area and the video frame currently stored in the video buffer area meet the synchronous playing condition.
7. The apparatus of claim 6, wherein the predetermined packaging criteria is: IEC61937 standard; the audio frame is: audio frames based on the EAC3 format.
8. The apparatus of claim 6, wherein the computing module is specifically configured to:
every preset synchronous time or every time a preset synchronous time point is reached, according to the formula:calculating the buffering time of the current audio buffer area;
wherein, T is the buffering time of the current audio buffer, M is the total number of bytes of all audio frames stored in the audio buffer, S is the sampling rate for an audio frame, and N is the number of bytes of a single audio frame.
9. The apparatus of claim 6, wherein the first determining module is specifically configured to:
and determining the sum of the buffering time and the time corresponding to the first target timestamp as the audio playing time according to the buffering time and the first target timestamp.
10. The apparatus according to any one of claims 6 to 9, wherein the second determining module is specifically configured to:
and judging whether the audio playing time is greater than the time corresponding to the second target timestamp or not according to the audio playing time and the second target timestamp, and if so, determining that the audio frame stored in the current audio buffer area and the video frame stored in the current video buffer area meet the synchronous playing condition.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611234859.4A CN106658135B (en) | 2016-12-28 | 2016-12-28 | A kind of audio and video playing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611234859.4A CN106658135B (en) | 2016-12-28 | 2016-12-28 | A kind of audio and video playing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106658135A true CN106658135A (en) | 2017-05-10 |
| CN106658135B CN106658135B (en) | 2019-08-09 |
Family
ID=58832175
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611234859.4A Active CN106658135B (en) | 2016-12-28 | 2016-12-28 | A kind of audio and video playing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106658135B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107509100A (en) * | 2017-09-15 | 2017-12-22 | 深圳国微技术有限公司 | Audio and video synchronization method, system, computer installation and computer-readable recording medium |
| CN109168059A (en) * | 2018-10-17 | 2019-01-08 | 上海赛连信息科技有限公司 | A kind of labial synchronization method playing audio & video respectively on different devices |
| CN109729391A (en) * | 2018-12-18 | 2019-05-07 | 北京华夏电通科技有限公司 | A kind of sending method and system of multipath media stream |
| CN112235597A (en) * | 2020-09-17 | 2021-01-15 | 深圳市捷视飞通科技股份有限公司 | Method and device for synchronous protection of streaming media live broadcast audio and video and computer equipment |
| CN113300797A (en) * | 2021-03-29 | 2021-08-24 | 广州市保伦电子有限公司 | Multi-device self-adaptive audio synchronous playing method and processing terminal |
| CN115834943A (en) * | 2023-01-10 | 2023-03-21 | 荣耀终端有限公司 | Audio and video synchronization method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070081562A1 (en) * | 2005-10-11 | 2007-04-12 | Hui Ma | Method and device for stream synchronization of real-time multimedia transport over packet network |
| CN101237586A (en) * | 2008-02-22 | 2008-08-06 | 上海华平信息技术股份有限公司 | Synchronous playing method for audio and video buffer |
| CN101282482A (en) * | 2008-05-04 | 2008-10-08 | 中兴通讯股份有限公司 | Apparatus, system and method for synchronously playing video data and audio data |
| CN101466044A (en) * | 2007-12-19 | 2009-06-24 | 康佳集团股份有限公司 | Method and system for synchronously playing stream medium audio and video |
| CN102868939A (en) * | 2012-09-10 | 2013-01-09 | 杭州电子科技大学 | Method for synchronizing audio/video data in real-time video monitoring system |
| CN103167320A (en) * | 2011-12-15 | 2013-06-19 | 中国电信股份有限公司 | Audio and video synchronization method and audio and video synchronization system and mobile phone live broadcast client-side |
-
2016
- 2016-12-28 CN CN201611234859.4A patent/CN106658135B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070081562A1 (en) * | 2005-10-11 | 2007-04-12 | Hui Ma | Method and device for stream synchronization of real-time multimedia transport over packet network |
| CN101466044A (en) * | 2007-12-19 | 2009-06-24 | 康佳集团股份有限公司 | Method and system for synchronously playing stream medium audio and video |
| CN101237586A (en) * | 2008-02-22 | 2008-08-06 | 上海华平信息技术股份有限公司 | Synchronous playing method for audio and video buffer |
| CN101282482A (en) * | 2008-05-04 | 2008-10-08 | 中兴通讯股份有限公司 | Apparatus, system and method for synchronously playing video data and audio data |
| CN103167320A (en) * | 2011-12-15 | 2013-06-19 | 中国电信股份有限公司 | Audio and video synchronization method and audio and video synchronization system and mobile phone live broadcast client-side |
| CN102868939A (en) * | 2012-09-10 | 2013-01-09 | 杭州电子科技大学 | Method for synchronizing audio/video data in real-time video monitoring system |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107509100A (en) * | 2017-09-15 | 2017-12-22 | 深圳国微技术有限公司 | Audio and video synchronization method, system, computer installation and computer-readable recording medium |
| CN109168059A (en) * | 2018-10-17 | 2019-01-08 | 上海赛连信息科技有限公司 | A kind of labial synchronization method playing audio & video respectively on different devices |
| CN109168059B (en) * | 2018-10-17 | 2021-06-18 | 上海赛连信息科技有限公司 | Lip sound synchronization method for respectively playing audio and video on different devices |
| CN109729391A (en) * | 2018-12-18 | 2019-05-07 | 北京华夏电通科技有限公司 | A kind of sending method and system of multipath media stream |
| CN109729391B (en) * | 2018-12-18 | 2021-07-02 | 北京华夏电通科技股份有限公司 | Method and system for sending multi-path media streams |
| CN112235597A (en) * | 2020-09-17 | 2021-01-15 | 深圳市捷视飞通科技股份有限公司 | Method and device for synchronous protection of streaming media live broadcast audio and video and computer equipment |
| CN113300797A (en) * | 2021-03-29 | 2021-08-24 | 广州市保伦电子有限公司 | Multi-device self-adaptive audio synchronous playing method and processing terminal |
| CN113300797B (en) * | 2021-03-29 | 2022-09-16 | 广州市保伦电子有限公司 | Multi-device self-adaptive audio synchronous playing method and processing terminal |
| CN115834943A (en) * | 2023-01-10 | 2023-03-21 | 荣耀终端有限公司 | Audio and video synchronization method and device |
| CN115834943B (en) * | 2023-01-10 | 2023-07-04 | 荣耀终端有限公司 | Audio and video synchronization method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106658135B (en) | 2019-08-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106658135B (en) | A kind of audio and video playing method and device | |
| JP3629253B2 (en) | Audio reproduction device and audio reproduction control method used in the same | |
| US7302396B1 (en) | System and method for cross-fading between audio streams | |
| EP2752023B1 (en) | Method to match input and output timestamps in a video encoder and advertisement inserter | |
| JP5660895B2 (en) | Apparatus and method for synchronizing user observable signals | |
| US20140376873A1 (en) | Video-audio processing device and video-audio processing method | |
| JP4983923B2 (en) | Decoder device and decoding method | |
| CN108111997A (en) | Bluetooth equipment audio synchronization method and system | |
| KR20060125678A (en) | Buffer management system, digital audio receiver, headphones, loudspeaker, method of buffer management | |
| JP2006527864A (en) | Receiver device, transmitter device, and transmission system | |
| CN111182315A (en) | Multimedia file splicing method, device, equipment and medium | |
| JP5012859B2 (en) | Volume control device | |
| CN106331820B (en) | Audio and video synchronization processing method and device | |
| JPWO2006064689A1 (en) | Wireless communication system | |
| JP2007094234A (en) | Data recording and reproducing apparatus and method, and program thereof | |
| CN109040818A (en) | Audio and video synchronization method, storage medium, electronic equipment and system when live streaming | |
| KR20070008069A (en) | Appratus and method for synchronizing audio/video signal | |
| JP2004320424A (en) | Method and apparatus for audio visual transmission audio visual transmitting apparatus and audio visual receiving apparatus | |
| JP2007201983A (en) | Broadcast station synchronization method, and control apparatus | |
| JP4359024B2 (en) | Synchronous control method and apparatus, synchronous playback apparatus and television receiver using the same | |
| CN113055711B (en) | Audio and video synchronous detection method and detection system thereof | |
| TW200829009A (en) | Playback system capable of synchronizing playback of video signal and playback of audio signal and method thereof | |
| JP5325059B2 (en) | Video / audio synchronized playback device, video / audio synchronized processing device, video / audio synchronized playback program | |
| JP3491533B2 (en) | Method and apparatus for reproducing digital audio data | |
| JP2011205439A (en) | Reproducing apparatus and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |