CN114040237A - Audio and video synchronous playing method, terminal, multimedia playing system and medium - Google Patents
Audio and video synchronous playing method, terminal, multimedia playing system and medium Download PDFInfo
- Publication number
- CN114040237A CN114040237A CN202111173364.6A CN202111173364A CN114040237A CN 114040237 A CN114040237 A CN 114040237A CN 202111173364 A CN202111173364 A CN 202111173364A CN 114040237 A CN114040237 A CN 114040237A
- Authority
- CN
- China
- Prior art keywords
- audio
- video
- stream data
- playing
- terminal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 120
- 230000001360 synchronised effect Effects 0.000 title claims abstract description 59
- 230000008569 process Effects 0.000 claims abstract description 47
- 230000005540 biological transmission Effects 0.000 claims description 91
- 238000012360 testing method Methods 0.000 claims description 41
- 230000003111 delayed effect Effects 0.000 claims description 24
- 230000015654 memory Effects 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 10
- 238000004891 communication Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005266 casting Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000002427 irreversible effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004984 smart glass Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
- H04N21/43076—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of the same content streams on multiple devices, e.g. when family members are watching the same movie on different devices
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/43615—Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
- H04N21/43637—Adapting the video stream to a specific local network, e.g. a Bluetooth® network involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses an audio and video synchronous playing method, a terminal, a multimedia playing system and a medium, and belongs to the technical field of video playing. The method is used for a video playing terminal, the video playing terminal is connected with an audio playing terminal, and the method comprises the following steps: acquiring multimedia data to be played and delay information, wherein the delay information is loss time in the process that audio data are coded and transmitted to an audio playing terminal at the video playing terminal, the audio playing terminal decodes the audio data to obtain the audio data, and the multimedia data to be played comprises video stream data and audio stream data which are mutually associated; and coding the audio stream data and then sending the coded audio stream data to an audio playing terminal so as to enable the audio playing terminal to play the decoded audio stream data in real time, and delaying to play the video stream data based on the delay information so as to enable the video stream data and the audio stream data to be played synchronously. The invention can realize the synchronous playing of the audio playing terminal and the video playing terminal.
Description
Technical Field
The present application relates to the field of video playing technologies, and in particular, to a method, a terminal, a multimedia playing system, and a medium for synchronously playing audio and video.
Background
In the related art, a video playing terminal, such as a television, can be connected with an additional audio playing terminal, such as a bluetooth sound box, through bluetooth, so that video playing at the video playing terminal and audio playing at the audio playing terminal are realized.
However, the playing mode has the problem that the playing time of the audio playing terminal is delayed relative to the playing time of the video playing terminal.
The above is only for the purpose of assisting understanding of the technical solutions of the present application, and does not represent an admission that the above is prior art.
Disclosure of Invention
The present application mainly aims to provide an audio and video synchronous playing method, a terminal, a multimedia playing system and a medium, and aims to solve the technical problem that the playing of the existing audio playing terminal is delayed relative to the playing of the video playing terminal.
In order to achieve the above object, in a first aspect, the present application provides an audio and video synchronous playing method, which is used for a video playing terminal, where the video playing terminal is connected to the audio playing terminal through a bluetooth, and the method includes:
acquiring multimedia data to be played and delay information, wherein the delay information is loss time in the process that audio data are coded and transmitted to an audio playing terminal at the video playing terminal, the audio playing terminal decodes the audio data to obtain the audio data, and the multimedia data to be played comprises video stream data and audio stream data which are mutually associated;
and coding the audio stream data and then sending the coded audio stream data to an audio playing terminal so as to enable the audio playing terminal to play the decoded audio stream data in real time, and delaying to play the video stream data based on the delay information so as to enable the video stream data and the audio stream data to be played synchronously.
In an embodiment, before the multimedia data to be played and the delay information are acquired, the method further includes:
acquiring prestored audio stream data;
coding pre-stored audio stream data to obtain a coded file, sending the coded file to an audio playing terminal so that the audio playing terminal decodes the coded file to obtain pre-stored audio stream data, determining test transmission loss time of the pre-stored audio stream data in the coding, transmission and decoding processes, and feeding back the test transmission loss time to the video playing terminal;
based on the trial transmission loss time, delay information is determined.
In an embodiment, the experimental transmission loss time is a first display timestamp of an audio frame played by the audio playing terminal at any moment in the pre-stored audio stream data;
based on the experimental transmission loss time, determining delay information, including:
determining delay information based on a time difference between the first display timestamp and a second display timestamp of the current video frame; and the current video frame is a video frame which is correspondingly played by the video playing terminal at any moment in the pre-stored video data.
In a second aspect, the present application further provides an audio and video synchronous playing method, which is used for an audio playing terminal, where the audio playing terminal is connected to a video playing terminal through a bluetooth, and the method includes:
receiving encoded audio stream data sent by a video playing terminal when video stream data is played in a delayed manner based on delay information; the audio stream data and the video stream data are associated, the audio stream data and the video stream data are obtained by a video playing terminal after acquiring multimedia data to be played, the delay information is loss time in the process that the audio playing terminal encodes the audio data and then transmits the encoded audio data to the audio playing terminal, and the audio playing terminal decodes the encoded audio data to obtain the audio data;
and decoding the coded audio stream data to obtain audio stream data and playing the audio stream data in real time so as to synchronously play the video stream data and the audio stream data.
In an embodiment, before receiving encoded audio stream data sent by a video playing terminal when playing video stream data in a delayed manner based on delay information, the method further includes:
receiving a coding file sent by a video playing terminal, wherein the coding file is obtained by coding prestored audio stream data by the video playing terminal;
decoding the encoded file to obtain pre-stored audio stream data;
and determining the test transmission loss time of the pre-stored audio stream data in the encoding, transmission and decoding processes, and feeding back the test transmission loss time to the video playing terminal, so that the video playing terminal determines the delay information based on the test transmission loss time.
In an embodiment, determining a test transmission loss time of pre-stored audio stream data, and feeding back the test transmission loss time to a video playing terminal, so that the video playing terminal determines the delay information based on the test transmission loss time includes:
taking a first display timestamp of an audio frame played by the audio playing terminal at any moment in the pre-stored audio stream data as a test transmission loss time, and feeding back the test transmission loss time to the video playing terminal, so that the video playing terminal determines delay information based on a time difference between the first display timestamp and a second display timestamp of the current video frame; and the current video frame is a video frame which is correspondingly played by the video playing terminal at any moment in the pre-stored video data.
In a third aspect, the present application further provides a video playing terminal, where the video playing terminal includes a first processor, a first memory, and a first computer program stored in the first memory, and when the first computer program is executed by the first processor, the method for synchronously playing audio and video is implemented.
In a fourth aspect, the present application further provides an audio playback terminal, where the audio playback terminal includes a second processor, a second memory, and a second computer program stored in the second memory, and when the second computer program is executed by the second processor, the audio and video synchronous playback method is implemented.
In a fifth aspect, the present application further provides a multimedia playing system, including:
a video playing terminal; and
and the audio playing terminal is connected with the video playing terminal through Bluetooth.
In a sixth aspect, the present application further provides a computer-readable storage medium, where an audio and video synchronous playing program is stored on the computer-readable storage medium, and when executed by a processor, the audio and video synchronous playing program implements the audio and video synchronous playing method as described above.
The application provides an audio and video synchronous playing method, which comprises the following steps: the video playing terminal obtains the multimedia data to be played and the delay information, then plays the video stream data of the multimedia data to be played in a delay mode according to the delay information, encodes the associated audio stream data and then sends the encoded audio stream data to the audio playing terminal through Bluetooth, and when the audio playing terminal decodes and plays the audio stream data, the video stream data and the audio stream data are played synchronously. That is, the method controls the video playing terminal to encode the audio data and transmit the encoded audio data to the audio playing terminal, the audio playing terminal decodes the audio data to obtain the loss time in the process of the audio data, namely, the video stream data is played in a delayed manner by the delay information, and the actual transmission loss time consumed by the audio stream data is consistent with the delay information, so that the synchronous playing of the video and the audio is realized. Compared with the prior art that both the video playing terminal and the audio playing terminal play in real time, the method solves the technical problem that the playing of the existing audio playing terminal is delayed relative to the video playing terminal by using the steps, and provides better multimedia watching experience for users.
Drawings
FIG. 1 is a schematic diagram of a multimedia playing system according to the present invention;
FIG. 2 is a schematic structural diagram of a video playback terminal or an audio playback terminal according to the present invention;
FIG. 3 is a flowchart illustrating a method for synchronously playing AV data according to a first embodiment of the present invention;
FIG. 4 is a flowchart illustrating a method for synchronously playing AV data according to a second embodiment of the present invention;
FIG. 5 is a flowchart illustrating a third embodiment of a method for synchronously playing AV data according to the present invention;
FIG. 6 is a flowchart illustrating a fourth embodiment of a method for synchronously playing AV data according to the present invention;
FIG. 7 is a flowchart illustrating a fifth embodiment of a method for synchronously playing AV data according to the present invention;
FIG. 8 is a flowchart illustrating a sixth embodiment of a method for synchronously playing AV data according to the present invention;
FIG. 9 is a functional block diagram of an AV synchronization apparatus according to a first embodiment of the present invention;
FIG. 10 is a block diagram of an audio/video synchronization apparatus according to a second embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In the related art, more and more smart devices are linked together by radio technologies such as bluetooth or WiFi. For example, as an important component of a home wireless audio/video solution, a video playing terminal such as a television can be connected to an audio playing terminal such as a bluetooth speaker through bluetooth, so that video playing at the video playing terminal and audio playing at the audio playing terminal are realized.
However, when the television plays multimedia data, the television is an input terminal for the multimedia data to be played, and performs splitting to obtain video stream data and audio stream data, where the video stream data is played in real time at the television, but the television also needs to encode the audio stream data according to a bluetooth protocol and transmit the encoded audio stream data to a bluetooth speaker. And the Bluetooth sound box receives the audio stream data, decodes the audio stream data and plays the audio stream data. The inventor of the application finds that time is consumed in the process of encoding, decoding and transmitting the audio stream data, and the audio stream data is irreversible, so that when the multimedia data is played, the Bluetooth sound box has playing delay relative to a television end.
Therefore, the main technical solution of the embodiment of the present application is as follows: when multimedia data are played, a video playing terminal is controlled to encode the audio data at the video playing terminal and transmit the encoded audio data to the audio playing terminal, and the audio playing terminal decodes the encoded audio data to obtain loss time delay playing video stream data in the process of playing the video stream data and the audio stream data, so that the video stream data and the audio stream data of the multimedia data are respectively played synchronously at two terminals.
In the following, a multimedia playing system applied in the implementation of the present application will be described:
referring to fig. 1, the multimedia playing system includes a video playing terminal and an audio playing terminal. The video playing terminal and the audio playing terminal are connected through radio technologies such as Bluetooth or WiFi.
The video playing terminal related in the embodiment of the application comprises: the audio and video synchronous playing program is configured to implement the steps of the audio and video synchronous playing program method of the following method embodiment. The video playing terminal can be a smart television, a smart phone, a tablet computer, a VR terminal, an AR terminal or a projector and other devices.
The audio playing terminal related in the embodiment of the application comprises: the audio and video synchronous playing program is configured to implement the steps of the audio and video synchronous playing program method of the following method embodiment. The audio playing terminal can be a Bluetooth sound box, a Bluetooth headset or Bluetooth intelligent glasses and other devices.
Referring to fig. 2, fig. 2 is a schematic structural diagram of a video playback terminal or an audio playback terminal in a hardware operating environment according to an embodiment of the present application.
As shown in fig. 2, the cast terminal may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a WIreless interface (e.g., a WIreless-FIdelity (WI-FI) interface). The Memory 1005 may be a Random Access Memory (RAM) Memory, or may be a Non-Volatile Memory (NVM), such as a disk Memory. The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the configuration shown in fig. 2 does not constitute a limitation of the cast terminal and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 2, the memory 1005, which is a storage medium, may include an operating system, a data storage module, a bluetooth communication module, a user interface module, and an audio-video synchronized playback program.
In the playback terminal shown in fig. 2, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the play terminal of the present invention may be disposed in the play terminal, and the play terminal invokes the audio and video synchronous play program stored in the memory 1005 through the processor 1001 and executes the audio and video synchronous play method provided in the embodiment of the present invention.
The method embodiment of the present application is provided below by taking a video playing terminal as an intelligent television and an audio playing terminal as a bluetooth speaker as an example. And the Bluetooth sound box receives audio stream data corresponding to the video stream data played by the intelligent television by utilizing a Bluetooth principle and a mode of receiving frequency in a wireless receiving mode and a matching link of the intelligent television of a user, and plays the audio stream data. It should be noted that, according to the disclosure of the embodiment of the present application, a person skilled in the art can easily think of the specific steps when the video playing terminal is a smart phone, a tablet computer or a projector, and the audio playing terminal is a bluetooth headset or bluetooth smart glasses, etc.
Referring to fig. 3, fig. 3 is a schematic flowchart illustrating a first embodiment of a method for synchronously playing audio and video according to an embodiment of the present application.
In this embodiment, the method includes the steps of:
s101, multimedia data to be played and delay information are obtained, wherein the delay information is loss time in the process that the audio data are coded and transmitted to an audio playing terminal at the video playing terminal, the audio playing terminal decodes the audio data to obtain the audio data, and the multimedia data to be played comprises video stream data and audio stream data which are related to each other.
The main execution body of the audio and video synchronous playing method of the embodiment is a television end. The television end can be provided with a human-computer interaction interface, and a video playing instruction input by a user can be received through the human-computer interaction interface. In response to the video playing instruction, the television terminal can send a multimedia data downloading request to be played to the network server. Thereby acquiring the multimedia data to be played sent by the network server. Or, the television terminal can also obtain the multimedia data to be played, which is sent by the smart phone or the tablet computer, by responding to a screen-casting request of the smart phone or the tablet computer of the user.
The multimedia data to be played comprises video stream data and audio stream data which are related to each other. It can be understood that common multimedia data, such as mp4, mov, flv, avi, rmvb, etc., is a container package, which contains two parts of audio stream data and video stream data, and is obtained by encoding and compressing through a specific encoding algorithm. After receiving the multimedia data to be played, the television end can decode the multimedia data, so that audio stream data and video stream data which are internally packaged are obtained. For example, H264, Xvid, etc. are video encoding formats, and MP3, AAC, etc. are audio encoding formats. After an Xvid video encoded file and an MP3 audio encoded file are encapsulated according to the AVI encapsulation standard, an AVI suffix video stream data is obtained. And it can be understood that in the same multimedia data to be played, the video stream data and the audio stream data are associated with each other to realize sound and picture synchronous playing.
The delay information is the loss time in the process that the audio data are coded and transmitted to the audio playing terminal at the video playing terminal, and the audio playing terminal decodes the audio data to obtain the audio data. It is easy to find that, after the television decodes the multimedia data to obtain the video stream data, the television immediately starts playing the video stream data in real time, and the audio stream data needs additional encoding, transmission and decoding processes before being played in real time, so that in the process, the audio stream data played in real time has a corresponding delay relative to the video stream data. And different audio playing devices have different delay information due to different decoding efficiencies of different playing devices. And for the same audio playing terminal, the delay information can be regarded as a fixed value.
It can be understood that, because the audio playing terminal and the video playing terminal are connected through the bluetooth, and the bluetooth connection has a distance limitation, for the two connected audio playing terminals and video playing terminal, after the connection is established between the two audio playing terminals and the video playing terminal, the time consumption change of the bluetooth transmission is small. And for the two determined audio playing terminals and the video playing terminal, the encoding and decoding efficiency of the same audio playing terminal is determined, the decoding time consumption is less changed, and the encoding and decoding efficiency can be regarded as a fixed value. Therefore, the transmission loss time of the audio data between the video player terminal and the audio player terminal after the bluetooth connection is established, i.e., the delay information, can be regarded as a fixed value.
In this embodiment, after the connection between the television and the bluetooth speaker is established, the delay information between the television and the bluetooth speaker may be determined or calculated, and when the television and the bluetooth speaker play the same multimedia data to be played, the delay information is used as a delay reference.
S102, the audio stream data are coded and then sent to the audio playing terminal, so that the audio playing terminal plays the audio stream data in real time after decoding, and the video stream data are played in a delayed mode based on the delay information, so that the video stream data and the audio stream data are played synchronously.
It can be understood that the television has a display screen, so that after the multimedia data to be played is decoded to obtain video stream data and audio stream data, the picture content of the multimedia data can be displayed on the display screen in a delayed manner according to the delay information. Meanwhile, after the audio stream data is obtained through decoding, the television end transmits the audio stream data to the Bluetooth sound box, so that the Bluetooth sound box plays the audio stream data. Specifically, the transmission process specifically includes: and the television end encodes the audio stream data according to a Bluetooth transmission protocol to obtain an encoded file. And then the coded file is transmitted to a Bluetooth sound box through a Bluetooth communication module of the television. After receiving the encoded file, the Bluetooth sound box decodes the encoded file through a Bluetooth transmission protocol to obtain audio stream data. And the Bluetooth sound box plays the audio stream data in real time.
It can be understood that, in the present embodiment:
the video stream playing actual timestamp T1 on the television end is the video timestamp + the delay information delay;
the audio stream playing real time stamp T2 on the bluetooth speaker is the audio time stamp + the real transmission loss time.
The delay information delay is the transmission loss time between the television terminal and the bluetooth speaker, and for the known television terminal and the known bluetooth speaker, the delay information delay of the television terminal and the bluetooth speaker is equal to the actual transmission loss time. Therefore, the video stream playing real time stamp T1 is the audio stream playing real time stamp T2, i.e., the two are played synchronously.
In this embodiment, after the audio playing terminal and the video playing terminal establish the bluetooth connection, when the video playing terminal obtains that the audio playing terminal and the video playing terminal play the same multimedia data, the audio data are coded and transmitted to the audio playing terminal at the video playing terminal, the audio playing terminal decodes the audio data to obtain the loss time in the process of the audio data, namely the time delay information, so that when the multimedia data to be played is played subsequently, the audio stream data of the multimedia data to be played is sent to the audio playing terminal to play the audio in real time, and the video is played in a time delay mode according to the time delay information, because the actual transmission loss time consumed by the audio stream data is consistent with the delay information, the timestamps of the video after delay playing and the audio with the transmission time consumption are consistent, and the synchronous playing of the video and the audio is realized. Compared with the prior art that the video playing terminal and the audio playing terminal play in real time and the audio is delayed, the method solves the technical problem that the existing audio playing terminal delays the playing relative to the video playing terminal by using the steps, and provides better multimedia playing experience for users.
As an embodiment, based on the first embodiment of the audio and video synchronous playing method of the present application, a second embodiment of the audio and video synchronous playing method of the present application is proposed. Referring to fig. 4, fig. 4 is a schematic flowchart illustrating a second embodiment of the audio-video synchronous playing method of the present application.
In this embodiment, the method includes the steps of:
step S201, obtaining pre-stored audio stream data.
In this embodiment, after the connection between the tv and the bluetooth speaker is established, an initialization determination step is performed between the tv and the bluetooth speaker. It can be understood that, since the transmission loss time of the audio data between the video playback terminal and the audio playback terminal after the bluetooth connection is established, i.e., the delay information, can be regarded as a fixed value. Therefore, after the connection between the two is established, the value of the delay information can be measured through an initialization decision step.
Specifically, the embodiment determines the value of the delay information by a pre-stored audio stream data. The pre-stored audio stream data may be a small file in an audio format, for example, only a few frames of data, which may be calculated to obtain the value of the delay information.
In one embodiment, to avoid affecting the normal use of the bluetooth speaker. The pre-stored audio stream data may have a play tag, so that the bluetooth speaker still decodes normally when receiving the pre-stored audio stream data, but does not play after decoding.
Step S202, coding the pre-stored audio stream data to obtain a coded file, and sending the coded file to the audio playing terminal, so that the audio playing terminal decodes the coded file to obtain pre-stored audio stream data, determines the test transmission loss time of the pre-stored audio stream data in the coding, transmission and decoding processes, and feeds back the test transmission loss time to the video playing terminal.
In this embodiment, the initialization determination step is consistent with the normal audio file playing process, and the television terminal still sends the encoded file to the bluetooth speaker after obtaining the encoded file according to the bluetooth transmission protocol encoding. After the Bluetooth sound box receives the coding file, the coding file is still decoded to obtain pre-stored audio stream data.
In this process, the Bluetooth speaker can determine the trial transmission loss time of the pre-stored audio stream data.
Step S203, determining the delay information based on the test transmission loss time.
In the step, the test transmission loss time of the audio stream data prestored in the initialization judging step is used as the delay information between the television and the Bluetooth sound box.
And step S204, acquiring multimedia data to be played and delay information.
Step S205, encode the audio stream data and send the encoded audio stream data to the audio playing terminal, so that the audio playing terminal plays the encoded audio stream data in real time, and play the delayed video stream data based on the delay information, so that the video stream data and the audio stream data are played synchronously.
Step S204 and step S205 can refer to steps S101 and S102 of the above embodiments, and are not described herein again.
In this embodiment, when the bluetooth establishes a connection between the video playback terminal and the audio playback terminal, the delay information is determined by the test transmission loss time consumed by a pre-stored audio stream data in the encoding-transmission and decoding processes, so that in the subsequent playback process of the video playback terminal and the audio playback terminal, the video playback terminal is controlled to play the video in a delayed manner by using the delay information as a reference, thereby achieving synchronous playback of the video and the audio.
As an embodiment, the bluetooth speaker can calculate the time consumed in the encoding-transmission-decoding process to obtain the experimental transmission loss time, and feed the experimental transmission loss time back to the television.
Specifically, in this case, step S202 includes:
and step A10, coding the pre-stored audio stream data to obtain a coded file, and recording the coding time information of the pre-stored audio stream data.
When the television end encodes the pre-stored audio stream data according to the Bluetooth transmission protocol to obtain an encoded file, the television end can also record encoding time information in the process. The encoding time information reflects the time consumed by the television side for encoding the audio stream data.
Step A20, sending the coding time information and the coding file to the audio playing terminal, so that the audio playing terminal decodes the coding file to obtain audio stream data, records the decoding time information of the pre-stored audio stream data, determines the test transmission loss time of the pre-stored audio stream data based on the coding time information and the decoding time information, and feeds back the test transmission loss time, so that the video playing terminal determines the delay information based on the test transmission loss time.
The television end sends the coding time information and the coding file to the Bluetooth sound box, the Bluetooth sound box decodes the coding file according to a Bluetooth transmission protocol and obtains prestored audio stream data, and the Bluetooth sound box can record decoding time consumed in the decoding process, namely decoding time information. Because the distance of Bluetooth transmission is short, the required time consumption is extremely small, and the influence on the time delay can be ignored. Accordingly, the trial transmission loss time of the pre-stored audio stream data can be determined based on the encoding time information and the decoding time information. Specifically, the bluetooth speaker adds the encoding time information and the decoding time information, and the sum of the two can be used as the experimental transmission loss time of the pre-stored audio stream data. The television terminal may use the trial transmission loss time obtained in the initialization decision step as the delay information.
As another embodiment, the bluetooth speaker may calculate the time consumed in the encoding-transmitting-decoding process according to the time stamp difference value when the audio segment and the video frame associated with each other are played, so as to obtain the delay information, and feed the delay information back to the television.
Specifically, the pre-stored video stream data and the pre-stored audio stream data associated with each other can theoretically realize sound-picture synchronization according to the display Time Stamp (PTS) of each frame of video frame and the display Time Stamp of each frame of audio frame. The PTS is used to measure when a decoded video frame or audio frame is played, and for example, the PTS of a certain frame may be represented as 03.23s on a second time basis.
In this embodiment, a multimedia data segment may be used as an initialization judgment material, the television terminal decodes the multimedia data segment to obtain pre-stored video stream data, and the pre-stored video stream data may be played in real time, so that the playing may be performed with reference to a local clock of the television terminal in combination with the display time stamps of the video frames. Or, in other embodiments, the television end may not play the pre-stored video stream data, but still uses the local clock of the television end as a reference, uses the local time after the audio/video stream separation as the starting time, and determines the video frame corresponding to the current time by combining the display time stamp of each video frame. That is, the display time stamp of the video frame corresponding to the current time is obtained.
Step S202 includes: coding pre-stored audio stream data to obtain a coded file, sending the coded file to an audio playing terminal so that the audio playing terminal decodes the coded file to obtain pre-stored audio stream data, taking a first display timestamp of an audio frame played by the audio playing terminal at any moment in the pre-stored audio stream data as test transmission loss time, and feeding back the test transmission loss time to the video playing terminal.
At any moment, when the bluetooth speaker decodes the encoded file to obtain the pre-stored audio stream data, the audio frame corresponding to any moment may be the first audio frame of the pre-stored audio stream data. The first audio frame has a corresponding presentation time stamp, i.e. a first presentation time stamp.
Specifically, when the bluetooth sound box decodes the encoded file to obtain the pre-stored audio stream data, the television end plays or corresponds to the current video frame, and the display timestamp of the current video frame is the second display timestamp. It can be understood that the television end encodes the pre-stored audio stream data and then sends the encoded pre-stored audio stream data to the bluetooth speaker, and the bluetooth speaker decodes the pre-stored audio stream data to obtain the pre-stored audio stream data. Since decoding, bluetooth transmission, and encoding are time consuming, it is inevitable that there is a time difference between the first presentation time stamp and the second presentation time stamp. I.e. the second presentation time stamp is subsequent to the first presentation time stamp.
It should be noted that, because the bluetooth speaker and the television are connected through bluetooth, the two can perform clock synchronization calibration after establishing bluetooth connection, so as to synchronize the local time of the television with the local time of the bluetooth speaker. Therefore, errors of the two are avoided at the current moment, and the accuracy of synchronous playing of the audio and the video is improved.
Step S203 includes: and determining delay information based on the time difference between the first display time stamp and the second display time stamp.
After the television end receives the first display timestamp of the pre-stored audio stream data, the delay information of the audio frequency in the Bluetooth sound box relative to the same frame of video frequency of the television end can be accurately calculated according to the time difference between the first display timestamp and the second display timestamp.
For example, in one embodiment, the bluetooth speaker decodes the encoded file at 15:30:00 to obtain the pre-stored audio stream data, and starts playing the first audio frame. The first audio frame has a first presentation time stamp of 01.12 s. The bluetooth speaker feeds back both 15:30:00 and the first display timestamp 01.12s to the television end, and the television end has played to the 5 th video frame at 15:30:00, and the 5 th video frame has the second display timestamp 01.54 s. The difference 0.42s between the second presentation time stamp 01.54s and the first presentation time stamp 01.12s is the delay information.
As an embodiment, on the basis of the first embodiment and the second embodiment of the audio and video synchronous playing method of the present application, a third embodiment of the audio and video synchronous playing method of the present application is provided. Referring to fig. 5, fig. 5 is a schematic flowchart illustrating a third embodiment of the audio-video synchronous playing method of the present application.
In this embodiment, the method includes the steps of:
and S301, judging whether the video playing terminal is connected with an audio playing terminal through Bluetooth.
In this embodiment, the television end can judge whether the television end is connected with the bluetooth speaker through bluetooth after the television end is turned on. Therefore, when the television is connected with the Bluetooth sound box, the mode that the television end simultaneously plays the video stream data and the audio stream data is adopted, or when the television is not connected with the Bluetooth sound box, the mode that the television end plays the video stream data and transmits the audio stream data to the Bluetooth sound box for playing is adopted.
Specifically, the television end can judge whether the video playing terminal is connected with the audio playing terminal through the Bluetooth in a mode of detecting the Bluetooth equipment node.
If the audio playing terminal is connected, step S302 is executed to obtain the multimedia data to be played and the delay information.
Step S303, the audio stream data is coded and then sent to the audio playing terminal, so that the audio playing terminal plays the audio stream data in real time after decoding, and the video stream data is played in a delayed manner based on the delay information, so that the video stream data and the audio stream data are played synchronously.
Step S302 to step S303 can refer to step S101 to step S102 of the above embodiments, and are not described herein again.
In this embodiment, a judgment step is added to correctly enter a normal play mode or a mode for synchronously playing audio and video data by using different devices. The phenomenon that the watching experience of a user is influenced due to the fact that the user mistakenly enters the delayed playing mode when the television end is not connected with the Bluetooth loudspeaker box through Bluetooth is avoided.
As an embodiment, a fourth embodiment of the audio and video synchronous playing method of the present application is provided. Referring to fig. 6, fig. 6 is a schematic flowchart illustrating a fourth embodiment of the audio-video synchronous playing method of the present application.
In this embodiment, the method includes the steps of:
step S401, receiving coded audio stream data sent by a video playing terminal when playing video stream data in a delayed manner based on delay information; the audio stream data and the video stream data are associated, the audio stream data and the video stream data are obtained by the video playing terminal after the multimedia data to be played are obtained, the delay information is the loss time in the process that the audio playing terminal encodes the audio data and then transmits the encoded audio data to the audio playing terminal, and the audio playing terminal decodes the encoded audio data to obtain the audio data.
In this embodiment, the main execution body of the audio and video synchronous playing method is a bluetooth speaker. The Bluetooth sound box is connected with the television end through Bluetooth. Therefore, when the television end plays the multimedia data to be played selected by the user under the control of the user, the Bluetooth sound box can play audio stream data in the multimedia data to be played.
It can be understood that the television end has a display screen, so that after the video stream data and the audio stream data are obtained through decoding, the picture content of the multimedia data can be displayed on the display screen in real time. Meanwhile, after audio stream data is obtained through decoding, the television end transmits the audio stream data to the Bluetooth sound box. The Bluetooth sound box receives the audio stream data.
Specifically, the process of audio stream data transmission specifically includes: and the television end encodes the audio stream data according to a Bluetooth transmission protocol to obtain an encoded file. And then the coded file is transmitted to a Bluetooth sound box through a Bluetooth communication module of the television. And at the moment, the television end enters a delay playing mode, and video stream data is played in a delayed mode according to delay information.
Step S402, decoding the coded audio stream data to obtain audio stream data and playing the audio stream data in real time so as to synchronously play the video stream data and the audio stream data.
After receiving the encoded file, the Bluetooth sound box decodes the encoded file through a Bluetooth transmission protocol to obtain audio stream data. And the Bluetooth sound box plays the audio stream data in real time.
It is easy to find that, after the television decodes the multimedia data to obtain the video stream data, the television starts to play the video stream data in a delayed manner, and the audio stream data can be played in real time only after additional encoding, transmission and decoding processes are required, and the actual transmission loss time consumed in the process is basically consistent with the delay information, so that the audio stream data played in real time is synchronous with the delayed video stream data in the process.
Referring to fig. 7, a fifth embodiment of the audio and video synchronous playing method of the present application is provided.
After the connection between the television and the Bluetooth sound box is established, the initialization judgment step is carried out between the television and the Bluetooth sound box. It can be understood that, since the transmission loss time of the audio data between the video playback terminal and the audio playback terminal after the bluetooth connection is established, i.e., the delay information, can be regarded as a fixed value. Therefore, after the connection between the two is established, the value of the delay information can be measured through an initialization decision step.
Specifically, the embodiment determines the value of the delay information by a pre-stored audio stream data. The pre-stored audio stream data may be a small file in an audio format, for example, only a few frames of data, which may be calculated to obtain the value of the delay information.
At this time, in this embodiment, the method includes:
step S501, receiving a coding file sent by the video playing terminal, wherein the coding file is obtained by coding pre-stored audio stream data by the video playing terminal.
And S502, decoding the encoded file to obtain pre-stored audio stream data.
Step S503, determining the test transmission loss time of the pre-stored audio stream data in the encoding, transmission and decoding processes, and feeding back the test transmission loss time to the video playing terminal.
Specifically, in one embodiment, the bluetooth speaker can calculate the time consumed in the encoding-transmission-decoding process to obtain the delay information, and feed the delay information back to the television. When the television end encodes the pre-stored audio stream data according to the Bluetooth transmission protocol to obtain an encoded file, the television end can also record encoding time information in the process. The coding time information reflects the time consumed by the television side for coding the pre-stored audio stream data.
The television terminal sends the coding time information and the coding file to the audio playing terminal, the Bluetooth loudspeaker box decodes the coding file according to the Bluetooth transmission protocol and obtains prestored audio stream data, and the Bluetooth loudspeaker box can record decoding time consumed in the decoding process, namely decoding time information.
Since the time required for bluetooth transmission is extremely small, the effect on the delay is negligible. Accordingly, the trial transmission loss time of the pre-stored audio stream data in this process can be determined based on the encoding time information and the decoding time information. Specifically, the bluetooth speaker adds the encoding time information and the decoding time information, and the sum of the encoding time information and the decoding time information can be used as the experimental transmission loss time of the prestored audio stream data in the process.
In another embodiment, the bluetooth speaker may calculate the time consumed in the encoding-transmitting-decoding process according to the time stamp difference value when the audio segment and the video frame picture associated with each other are played, so as to obtain the delay information, and feed the delay information back to the television end.
Specifically, in this case, step S503 includes:
taking a first display timestamp of an audio frame played by the audio playing terminal at any moment in the pre-stored audio stream data as a test transmission loss time, and feeding back the test transmission loss time to the video playing terminal, so that the video playing terminal determines delay information based on a time difference between the first display timestamp and a second display timestamp of the current video frame; and the current video frame is a video frame which is correspondingly played by the video playing terminal at any moment in the pre-stored video data.
In this embodiment, a multimedia data segment may be used as an initialization judgment material, and after the television decodes the multimedia data segment to obtain pre-stored video stream data, the pre-stored video stream data may be played in real time, so that the playing may be performed with the local clock of the television as a reference and in combination with the display time stamps of the video frames. Or, in other embodiments, the television end may not play the pre-stored video stream data, but still uses the local clock of the television end as a reference, uses the local time after the audio/video stream separation as the starting time, and determines the video frame corresponding to any time by combining the display time stamp of each video frame. I.e. the display time stamp of the corresponding video frame at any moment is available. And the television end encodes the pre-stored audio stream data to obtain an encoded file and sends the encoded file to the audio playing terminal. In this embodiment, any time may be when the bluetooth speaker decodes the encoded file to obtain the pre-stored audio stream data, and the audio frame corresponding to any time may be the first audio frame of the pre-stored audio stream data. The first audio frame has a corresponding presentation time stamp, i.e. a first presentation time stamp.
At this time, namely when the bluetooth sound box decodes the encoded file to obtain the pre-stored audio stream data, the television end correspondingly plays the current video frame, and the display timestamp of the current video frame is the second display timestamp. It can be understood that, since the tv end encodes the pre-stored audio stream data and then sends the encoded pre-stored audio stream data to the bluetooth speaker, the bluetooth speaker decodes the pre-stored audio stream data to obtain the first display timestamp of the first audio frame. Since both decoding and encoding are time consuming, it is inevitable that the first presentation time stamp has a time difference with respect to the second presentation time stamp. I.e. the second presentation time stamp is subsequent to the first presentation time stamp.
For example, in one embodiment, the bluetooth speaker decodes the encoded file at 15:30:00 to obtain the pre-stored audio stream data, and starts playing the first audio frame. The first audio frame has a first display timestamp. The bluetooth speaker feeds back both the 15:30:00 and the first display timestamp to the tv end, and the tv end knows that the 5 th video frame is played at 15:30:00, and the 5 th video frame has the second display timestamp. And the difference value between the second display time stamp and the first display time stamp is the delay information.
It should be noted that, because the bluetooth speaker and the television are connected through bluetooth, the two can perform clock synchronization calibration after establishing bluetooth connection, so as to synchronize the local time of the television with the local time of the bluetooth speaker. Therefore, the influence of the error existing at the current moment of the audio and video playing device due to the clock error is avoided, and the accuracy of the synchronous playing of the audio and video is improved.
As an embodiment, a sixth embodiment of the audio and video synchronous playing method of the present application is provided. Referring to fig. 8, fig. 8 is a flowchart illustrating a sixth embodiment of the audio-video synchronous playing method of the present application.
In this embodiment, the method includes the steps of:
step S601, the television end obtains multimedia data to be played and delay information; the delay information is the loss time in the process that the audio playing terminal encodes the audio data and then transmits the encoded audio data to the audio playing terminal, the audio playing terminal decodes the encoded audio data to obtain the audio data, and the multimedia data to be played comprises video stream data and audio stream data which are related to each other;
and step S602, the television terminal plays the video stream data in a delayed mode based on the delay information, encodes the audio stream data and then sends the encoded audio stream data to the Bluetooth sound box.
Step S603, the Bluetooth sound box decodes the audio stream data to obtain the audio stream data, and then plays the audio stream data in real time, so that the video stream data and the audio stream data are played synchronously.
In this embodiment, the video playing device is controlled to play the video stream data in a delayed manner according to the delay information, and since the actual transmission consumption time consumed in the encoding-transmission-decoding process of the delay information and the audio stream data on the audio playing terminal is consistent, the video and the audio can be played synchronously. Compared with the prior art that both the video playing terminal and the audio playing terminal play in real time, the method solves the technical problem that the playing of the existing audio playing terminal is delayed relative to the video playing terminal by utilizing the steps, and provides better multimedia playing experience for users.
Based on the same inventive concept, referring to fig. 9, the present application further provides a first embodiment of an audio/video synchronous playing device, configured to a video playing terminal, where the video playing terminal is connected to an audio playing terminal through bluetooth, and the device includes:
the data acquisition step module is used for acquiring multimedia data to be played and delay information, wherein the delay information is loss time in the process that the audio data are coded and transmitted to the audio playing terminal by the video playing terminal and decoded by the audio playing terminal to obtain the audio data, and the multimedia data to be played comprises video stream data and audio stream data which are mutually related;
and the video playing and audio sending module is used for coding the audio stream data and then sending the coded audio stream data to the audio playing terminal so as to enable the audio playing terminal to play the coded audio stream data in real time, and delaying to play the video stream data based on the delay information so as to enable the video stream data and the audio stream data to be played synchronously.
As an embodiment, the audio and video synchronous playing device further comprises:
the initialization judgment module is used for acquiring pre-stored audio stream data;
the audio playing terminal is used for coding pre-stored audio stream data to obtain a coded file and sending the coded file to the audio playing terminal so that the audio playing terminal can decode the coded file to obtain pre-stored audio stream data, determining the test transmission loss time of the pre-stored audio stream data in the coding, transmission and decoding processes and feeding back the test transmission loss time to the video playing terminal;
and the processor is further configured to determine delay information based on the trial transmission loss time.
As an embodiment, the initialization judgment module is further configured to determine the delay information based on a time difference between the first display timestamp and a second display timestamp of the current video frame; and the current video frame is a video frame which is correspondingly played by the video playing terminal at any moment in the pre-stored video data.
As an embodiment, the audio and video synchronous playing device further comprises:
the judgment module is used for judging whether the video playing terminal is connected with the audio playing terminal through Bluetooth; and if the audio playing terminal is connected, controlling the data acquisition step module to acquire the multimedia data to be played and the delay information.
Based on the same inventive concept, in addition, referring to fig. 10, the present application further provides a second embodiment of an audio and video synchronous playing device, which is used for an audio playing terminal, the audio playing terminal is connected with a video playing terminal through bluetooth, and the device includes:
the audio receiving module is used for receiving encoded audio stream data sent by the video playing terminal when the video stream data is played in a delayed manner based on the delay information; the audio stream data and the video stream data are associated, the audio stream data and the video stream data are obtained by a video playing terminal after acquiring multimedia data to be played, the delay information is loss time in the process that the audio playing terminal encodes the audio data and then transmits the encoded audio data to the audio playing terminal, and the audio playing terminal decodes the encoded audio data to obtain the audio data;
and the audio playing module is used for decoding the coded audio stream data to obtain the audio stream data and playing the audio stream data in real time so as to synchronously play the video stream data and the audio stream data.
As an embodiment, the audio and video synchronous playing device further comprises:
the second initialization judgment module is used for receiving a coding file sent by the video playing terminal, wherein the coding file is obtained by coding pre-stored audio stream data by the video playing terminal;
the audio decoding device is also used for decoding the coding file to obtain pre-stored audio stream data;
and the device is also used for determining the test transmission loss time of the pre-stored audio stream data in the encoding, transmission and decoding processes and feeding back the test transmission loss time to the video playing terminal.
As an embodiment, the second initialization decision module is further configured to: and taking a first display time stamp of an audio frame played by the audio playing terminal at any moment in the pre-stored audio stream data as a test transmission loss time, and feeding back the test transmission loss time to the video playing terminal, so that the video playing terminal determines delay information based on the time difference between the first display time stamp and the second display time stamp, wherein the current video frame is a video frame played by the video playing terminal correspondingly at any moment in the pre-stored video data.
Other embodiments and specific implementations of the audio and video synchronous playing device of the present application may refer to the foregoing method embodiments, and are not described herein again.
In addition, an embodiment of the present invention further provides a computer storage medium, where an audio and video synchronous playing program is stored on the storage medium, and when the audio and video synchronous playing program is executed by a processor, the steps of the audio and video synchronous playing method described above are implemented. Therefore, a detailed description thereof will be omitted. In addition, the beneficial effects of the same method are not described in detail. For technical details not disclosed in embodiments of the computer-readable storage medium referred to in the present application, reference is made to the description of embodiments of the method of the present application. It is determined that, by way of example, the program instructions may be deployed to be executed on one computing device or on multiple computing devices at one site or distributed across multiple sites and interconnected by a communication network.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
It should be noted that the above-described embodiments of the apparatus are merely schematic, where the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. In addition, in the drawings of the embodiment of the apparatus provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, and may be specifically implemented as one or more communication buses or signal lines. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that the present invention may be implemented by software plus necessary general hardware, and may also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components and the like. Generally, functions performed by computer programs can be easily implemented by corresponding hardware, and specific hardware structures for implementing the same functions may be various, such as analog circuits, digital circuits, or dedicated circuits. However, the implementation of a software program is a more preferable embodiment for the present invention. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, where the computer software product is stored in a readable storage medium, such as a floppy disk, a usb disk, a removable hard disk, a Read-only memory (ROM), a random-access memory (RAM), a magnetic disk or an optical disk of a computer, and includes instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. An audio and video synchronous playing method is characterized in that the method is used for a video playing terminal, the video playing terminal is connected with an audio playing terminal, and the method comprises the following steps:
acquiring multimedia data to be played and delay information, wherein the delay information is loss time in the process that the audio data are coded and transmitted to the audio playing terminal at the video playing terminal, the audio playing terminal decodes the audio data to obtain the audio data, and the multimedia data to be played comprises video stream data and audio stream data which are related to each other;
and coding the audio stream data and then sending the coded audio stream data to the audio playing terminal so as to enable the audio playing terminal to play the decoded audio stream data in real time, and playing the video stream data in a delayed manner based on the delay information so as to enable the video stream data and the audio stream data to be played synchronously.
2. The audio-video synchronous playing method according to claim 1, wherein before the obtaining the multimedia data to be played and the delay information, the method further comprises:
acquiring prestored audio stream data;
coding the pre-stored audio stream data to obtain a coded file, sending the coded file to the audio playing terminal, so that the audio playing terminal decodes the coded file to obtain the pre-stored audio stream data, determining the test transmission loss time of the pre-stored audio stream data in the coding, transmission and decoding processes, and feeding back the test transmission loss time to the video playing terminal;
and determining the delay information based on the test transmission loss time.
3. The audio and video synchronous playing method according to claim 2, wherein the experimental transmission loss time is a first display time stamp of an audio frame played by the audio playing terminal at any moment in playing the pre-stored audio stream data;
determining the delay information based on the experimental transmission loss time includes:
determining the delay information based on the time difference between the first display timestamp and a second display timestamp of the current video frame; and the current video frame is a video frame which is correspondingly played by the video playing terminal at any moment in the pre-stored video data.
4. An audio and video synchronous playing method is characterized in that the method is used for an audio playing terminal, the audio playing terminal is connected with a video playing terminal, and the method comprises the following steps:
receiving encoded audio stream data sent by the video playing terminal when the video stream data is played in a delayed manner based on the delay information; the audio stream data and the video stream data are associated, the audio stream data and the video stream data are obtained by the video playing terminal after acquiring multimedia data to be played, the delay information is loss time in the process that the audio playing terminal encodes the audio data and then transmits the encoded audio data to the audio playing terminal, and the audio playing terminal decodes the encoded audio data to obtain the audio data;
and decoding the coded audio stream data to obtain the audio stream data and playing the audio stream data in real time so as to synchronously play the video stream data and the audio stream data.
5. The av sync playback method of claim 4, wherein the receiving of the encoded audio stream data sent by the video playback terminal when playing back the video stream data in a delayed manner based on the delay information further comprises:
receiving a coding file sent by the video playing terminal, wherein the coding file is obtained by coding pre-stored audio stream data by the video playing terminal;
decoding the coding file to obtain the pre-stored audio stream data;
and determining the test transmission loss time of the pre-stored audio stream data in the encoding, transmission and decoding processes, and feeding back the test transmission loss time to the video playing terminal, so that the video playing terminal determines the delay information based on the test transmission loss time.
6. The audio and video synchronous playing method according to claim 5, wherein the determining a test transmission loss time of the pre-stored audio stream data and feeding back the test transmission loss time to the video playing terminal, so that the video playing terminal determines the delay information based on the test transmission loss time includes:
taking a first display timestamp of an audio frame played by the audio playing terminal at any moment in the pre-stored audio stream data as the test transmission loss time, and feeding back the test transmission loss time to the video playing terminal, so that the video playing terminal determines the delay information based on a time difference between the first display timestamp and a second display timestamp of the current video frame; and the current video frame is a video frame which is correspondingly played by the video playing terminal at any moment in the pre-stored video data.
7. A video playback terminal, comprising: a first processor, a first memory and a first computer program stored in the first memory, the first computer program, when executed by the first processor, implementing the steps of the audio video synchronous playback method according to any one of claims 1-3.
8. An audio playback terminal, comprising: second processor, a second memory and a second computer program stored in the second memory, the second computer program, when executed by the second processor, implementing the steps of the audio video synchronous playback method according to any of claims 4-6.
9. A multimedia playback system, comprising:
the video playback terminal of claim 7; and
the audio player terminal according to claim 8, wherein the audio player terminal is connected to the video player terminal via bluetooth.
10. A computer-readable storage medium, wherein an audio-video synchronous playing program is stored on the computer-readable storage medium, and when the audio-video synchronous playing program is executed by a processor, the audio-video synchronous playing method according to any one of claims 1 to 3 or the audio-video synchronous playing method according to any one of claims 4 to 6 is implemented.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111173364.6A CN114040237A (en) | 2021-09-30 | 2021-09-30 | Audio and video synchronous playing method, terminal, multimedia playing system and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111173364.6A CN114040237A (en) | 2021-09-30 | 2021-09-30 | Audio and video synchronous playing method, terminal, multimedia playing system and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114040237A true CN114040237A (en) | 2022-02-11 |
Family
ID=80134790
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111173364.6A Pending CN114040237A (en) | 2021-09-30 | 2021-09-30 | Audio and video synchronous playing method, terminal, multimedia playing system and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114040237A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114679665A (en) * | 2022-03-16 | 2022-06-28 | 深圳市冠旭电子股份有限公司 | Bluetooth headset audio and video synchronization method and device, electronic equipment and storage medium |
| CN114945105A (en) * | 2022-05-13 | 2022-08-26 | 宜百科技(深圳)有限公司 | Wireless earphone audio frequency hysteresis cancelling method under combination of sound compensation |
| CN115022731A (en) * | 2022-05-17 | 2022-09-06 | 蔚来汽车科技(安徽)有限公司 | Vehicle-mounted film viewing system, vehicle-mounted film viewing method and computer storage medium |
| CN115426514A (en) * | 2022-07-22 | 2022-12-02 | 北京罗克维尔斯科技有限公司 | Cross-device audio and video synchronization method, device, equipment and medium |
| CN115942021A (en) * | 2023-02-17 | 2023-04-07 | 央广新媒体文化传媒(北京)有限公司 | Audio and video stream synchronous playing method and device, electronic equipment and storage medium |
| CN116112722A (en) * | 2023-02-17 | 2023-05-12 | 央广新媒体文化传媒(北京)有限公司 | Audio playing method and device, electronic equipment and storage medium |
| WO2024093570A1 (en) * | 2022-10-31 | 2024-05-10 | 华为技术有限公司 | Screen projection method and apparatus |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101933333A (en) * | 2007-10-08 | 2010-12-29 | 摩托罗拉公司 | Remote audio and fixed video is synchronous |
| US20120206650A1 (en) * | 2011-02-16 | 2012-08-16 | Amtran Technology Co., Ltd | Method for synchronized playback of wireless audio and video and playback system using the same |
| CN105376628A (en) * | 2014-08-27 | 2016-03-02 | 深圳Tcl新技术有限公司 | Method and device for synchronizing audio-video signals |
| CN105898502A (en) * | 2016-04-11 | 2016-08-24 | 深圳Tcl新技术有限公司 | Audio and video playing synchronization method and synchronization device |
| CN105959786A (en) * | 2016-04-28 | 2016-09-21 | 乐视控股（北京）有限公司 | Audio and video playing method and device |
| CN106686438A (en) * | 2016-12-29 | 2017-05-17 | 北京奇艺世纪科技有限公司 | Cross-device audio/image synchronous playing method, equipment and system |
| CN107135413A (en) * | 2017-03-20 | 2017-09-05 | 福建天泉教育科技有限公司 | A kind of audio and video synchronization method and system |
| CN109257642A (en) * | 2018-10-12 | 2019-01-22 | Oppo广东移动通信有限公司 | Video resource playback method, device, electronic equipment and storage medium |
-
2021
- 2021-09-30 CN CN202111173364.6A patent/CN114040237A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101933333A (en) * | 2007-10-08 | 2010-12-29 | 摩托罗拉公司 | Remote audio and fixed video is synchronous |
| US20120206650A1 (en) * | 2011-02-16 | 2012-08-16 | Amtran Technology Co., Ltd | Method for synchronized playback of wireless audio and video and playback system using the same |
| CN105376628A (en) * | 2014-08-27 | 2016-03-02 | 深圳Tcl新技术有限公司 | Method and device for synchronizing audio-video signals |
| CN105898502A (en) * | 2016-04-11 | 2016-08-24 | 深圳Tcl新技术有限公司 | Audio and video playing synchronization method and synchronization device |
| CN105959786A (en) * | 2016-04-28 | 2016-09-21 | 乐视控股（北京）有限公司 | Audio and video playing method and device |
| CN106686438A (en) * | 2016-12-29 | 2017-05-17 | 北京奇艺世纪科技有限公司 | Cross-device audio/image synchronous playing method, equipment and system |
| CN107135413A (en) * | 2017-03-20 | 2017-09-05 | 福建天泉教育科技有限公司 | A kind of audio and video synchronization method and system |
| CN109257642A (en) * | 2018-10-12 | 2019-01-22 | Oppo广东移动通信有限公司 | Video resource playback method, device, electronic equipment and storage medium |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114679665A (en) * | 2022-03-16 | 2022-06-28 | 深圳市冠旭电子股份有限公司 | Bluetooth headset audio and video synchronization method and device, electronic equipment and storage medium |
| CN114945105A (en) * | 2022-05-13 | 2022-08-26 | 宜百科技(深圳)有限公司 | Wireless earphone audio frequency hysteresis cancelling method under combination of sound compensation |
| CN114945105B (en) * | 2022-05-13 | 2024-02-06 | 宜百科技(深圳)有限公司 | Wireless earphone audio hysteresis cancellation method combined with sound compensation |
| CN115022731A (en) * | 2022-05-17 | 2022-09-06 | 蔚来汽车科技(安徽)有限公司 | Vehicle-mounted film viewing system, vehicle-mounted film viewing method and computer storage medium |
| CN115426514A (en) * | 2022-07-22 | 2022-12-02 | 北京罗克维尔斯科技有限公司 | Cross-device audio and video synchronization method, device, equipment and medium |
| WO2024093570A1 (en) * | 2022-10-31 | 2024-05-10 | 华为技术有限公司 | Screen projection method and apparatus |
| CN115942021A (en) * | 2023-02-17 | 2023-04-07 | 央广新媒体文化传媒(北京)有限公司 | Audio and video stream synchronous playing method and device, electronic equipment and storage medium |
| CN116112722A (en) * | 2023-02-17 | 2023-05-12 | 央广新媒体文化传媒(北京)有限公司 | Audio playing method and device, electronic equipment and storage medium |
| CN116112722B (en) * | 2023-02-17 | 2023-06-27 | 央广新媒体文化传媒(北京)有限公司 | Audio playing method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114040237A (en) | Audio and video synchronous playing method, terminal, multimedia playing system and medium | |
| EP3850859B1 (en) | Dynamically adjusting video to improve synchronization with audio | |
| US8199780B2 (en) | Multimedia apparatus and synchronization method thereof | |
| JP3834257B2 (en) | Method and system for distributing multimedia data over heterogeneous networks | |
| US7664872B2 (en) | Media transfer protocol | |
| US20090234983A1 (en) | Methods and apparatus for sharing a computer display screen | |
| US20130003863A1 (en) | Network media adapter | |
| US20140139735A1 (en) | Online Media Data Conversion Method, Online Video Playing Method and Corresponding Device | |
| US9942620B2 (en) | Device and method for remotely controlling the rendering of multimedia content | |
| TWI735476B (en) | Method for fast channel change and corresponding device | |
| TW201448582A (en) | Mechanism for facilitating synchronization of audio and video between multiple media devices | |
| US20120133829A1 (en) | Video display apparatus and video display method, audio reproduction apparatus and audio reproduction method, and video/audio synchronous control system | |
| WO2016192431A1 (en) | Film source pushing method, set-top box and video server | |
| WO2021031590A1 (en) | Audio and video pushing method, mobile terminal and display terminal | |
| WO2016200520A1 (en) | Tunneling hdmi data over wireless connections | |
| CN109756744B (en) | Data processing method, electronic device and computer storage medium | |
| US20180205986A1 (en) | Method and Device for Implementing Synchronous Playing | |
| EP2695389B1 (en) | Processing media streams for synchronised output at multiple end points | |
| CN111385590A (en) | Live broadcast data processing method and device and terminal | |
| CN115474082A (en) | Method and apparatus for playing media data, system, vehicle, device and medium | |
| US20190028522A1 (en) | Transmission of subtitle data for wireless display | |
| CN114840282A (en) | Screen recording method and screen recording device of intelligent interactive tablet | |
| van Deventer et al. | Media synchronisation for television services through HbbTV | |
| CN116112722B (en) | Audio playing method and device, electronic equipment and storage medium | |
| CN115942021B (en) | Audio and video stream synchronous playing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |