CN108377406B - Method and device for adjusting sound and picture synchronization - Google Patents
Method and device for adjusting sound and picture synchronization Download PDFInfo
- Publication number
- CN108377406B CN108377406B CN201810374722.1A CN201810374722A CN108377406B CN 108377406 B CN108377406 B CN 108377406B CN 201810374722 A CN201810374722 A CN 201810374722A CN 108377406 B CN108377406 B CN 108377406B
- Authority
- CN
- China
- Prior art keywords
- audio
- played
- time
- frame
- sound
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The embodiment of the invention provides a method and a device for adjusting sound and picture synchronization, and relates to the technical field of video and audio playing. The embodiment of the invention can ensure the synchronization of sound and pictures when playing the video and audio of the film source file. The method comprises the steps that a sound and picture synchronization device sends an audio frame to be played to an audio playing device; the sound and picture synchronization device receives a return signal sent by the audio sending device, wherein the audio playing device sends the return signal after receiving an audio frame to be played; recording the time of receiving the return signal by the sound and picture synchronization device; the sound and picture synchronization device sends a video frame to be played to the video playing device and records the sending time, wherein the video frame to be played corresponds to the audio frame to be played; and the sound-picture synchronization device determines the time for sending the next audio frame to the audio playing device according to the time for receiving the return signal and the sending time. The invention is applied to audio and video playing.
Description
Technical Field
The invention relates to the technical field of video and audio processing, in particular to a method and a device for adjusting sound and picture synchronization.
Background
When playing a film source file using an electronic device such as a television, a computer, etc., the processing speed of the main chip in the electronic device for the audio data in the film source file is faster than the processing speed of the video data, which results in that when the main chip sends the processed audio data and video data to the audio playing device and the video playing device respectively for playing, the output time of the audio data is earlier than the output time of the corresponding video data, as shown in fig. 1. And then the time that the audio playing device plays the audio is earlier than the time that the video playing device plays the corresponding video, so that the condition that the output sound and the output picture are not synchronous is generated, and the use experience of a user is influenced.
In view of the above problems, a common solution in the prior art is to add a fixed delay to audio data at the host chip to achieve the purpose of audio-video synchronization. However, as the data amount of the audio and video changes, the delay between the audio playing time and the corresponding video playing time also changes, so the above solutions in the prior art are not ideal. In addition, as external devices such as wireless speakers are used more and more, under the condition that the transmission path of the audio data is more complicated, it is more difficult to satisfy the requirement of information transmission by using the method of adding the fixed delay to the audio data in the prior art.
Disclosure of Invention
The embodiment of the invention provides a method, a device and equipment for adjusting sound and picture synchronization, which can ensure the sound and picture synchronization when playing a film source file.
In order to achieve the above purpose, the embodiment of the invention adopts the following technical scheme:
in a first aspect, an embodiment of the present invention provides a method for adjusting synchronization of sound and pictures, including: the sound and picture synchronization device sends an audio frame to be played to the audio playing device; the sound and picture synchronization device receives a return signal sent by the audio sending device, wherein the audio playing device sends the return signal after receiving the audio frame to be played; the sound and picture synchronization device records the time of receiving the return signal; the sound and picture synchronization device sends a video frame to be played to a video playing device and records sending time, wherein the video frame to be played corresponds to the audio frame to be played; and the sound and picture synchronization device determines the time for sending the next audio frame to the audio playing device according to the time for receiving the return signal and the sending time.
In a second aspect, an embodiment of the present invention provides another method for adjusting synchronization of sound and pictures, including: the audio playing device receives the audio frame to be played sent by the sound and picture synchronizing device; and the audio playing device sends a return signal to the audio and video synchronization device after receiving the audio frame to be played, so that the audio and video synchronization device records the time of receiving the return signal after receiving the return signal, and the audio and video synchronization device determines the time of sending the next audio frame to the audio playing device according to the time of receiving the return signal and the sending time after recording the sending time of sending the video frame to be played to the video playing device, wherein the video frame to be played corresponds to the audio frame to be played.
In a third aspect, an embodiment of the present invention provides a sound and picture synchronization apparatus, including: the interface module is used for sending the audio frame to be played to the audio playing device; the interface module is further configured to receive the return signal, where the audio playing apparatus sends the return signal after receiving the audio frame to be played; the recording module is used for recording the time of receiving the return signal after the interface module receives the return signal; the interface module is further configured to send a video frame to be played to a video playing device, where the video frame to be played corresponds to the audio frame to be played; the recording module is also used for recording the sending time of sending the video frame to be played to the video playing device; and the computing module is used for determining the time for sending the next audio frame to the audio playing device according to the time for receiving the return signal and the sending time.
In a fourth aspect, an embodiment of the present invention provides an audio playing apparatus, including: the interface module is used for receiving an audio frame to be played, which is sent to the audio playing device by the sound and picture synchronization device; the interface module is further configured to send a transfer signal back to the sound and picture synchronization device after receiving the audio frame to be played, so that the sound and picture synchronization device records the time of receiving the return signal after receiving the return signal, so that the sound and picture synchronization device determines the time of sending a next audio frame to the audio playing device according to the time of receiving the return signal and the sending time after recording the sending time of sending the video frame to be played to the video playing device, where the video frame to be played corresponds to the audio frame to be played.
In a fifth aspect, an embodiment of the present invention provides an electronic device, including the sound and picture synchronization apparatus and the video playback apparatus provided in the third aspect.
In a sixth aspect, an embodiment of the present invention provides a wireless sound box device, including the audio playing apparatus provided in the fourth aspect.
In the embodiment of the invention, after the audio and video synchronization device sends the audio frame to be played to the audio playing device, the audio playing device feeds back the return signal to the audio and video synchronization device, and then the audio and video synchronization device can determine the time for the audio playing device to play the audio frame to be played by recording the time for receiving the return signal, namely the time for a user to hear the audio frame to be played. And the sound-picture synchronization device determines the time for the video playing device to play the video frame to be played by recording the time for sending the video frame to be played to the video playing device, namely the time for the user to see the video frame to be played is determined. And then the delay value between video playing and audio playing can be determined by the time of receiving the return signal and the sending time of sending the video frame to be played to the video playing device. Finally, according to the delay value between the video playing and the audio playing, the time for sending the next audio frame to the audio playing device is adjusted, so that the synchronous playing of the audio frame and the video frame is ensured.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
Fig. 1 is a schematic structural diagram of an electronic device for playing audio and video in the prior art;
fig. 2 is a schematic structural diagram of an audio/video playing device according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of another audio/video playing device according to an embodiment of the present invention;
fig. 4 is a flowchart illustrating a method for adjusting audio-video synchronization according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of data transmitted between a sound and picture synchronization apparatus and a video playback apparatus and an audio playback apparatus according to an embodiment of the present invention;
FIG. 6 is a flowchart illustrating another method for adjusting the synchronization of sound and picture according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a sound-picture synchronization apparatus according to an embodiment of the present invention;
FIG. 8 is a schematic structural diagram of another audio-visual synchronization apparatus according to an embodiment of the present invention;
FIG. 9 is a schematic structural diagram of another audio-visual synchronization apparatus according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of an audio playing apparatus according to an embodiment of the present invention.
Detailed Description
Embodiments of the present invention will be described below with reference to the accompanying drawings.
The invention is applied to audio and video playing scenes. Exemplarily, as shown in fig. 2, which is a schematic structural diagram of an audio and video playing device provided in an embodiment of the present invention, a sound and picture synchronization device 101 is respectively connected to an audio playing device 102, a video playing device 103, and a main chip 104. The main chip 104 is configured to read a film source file in the storage space, decode the film source file, generate video data and audio data for playing, and send the video data and the audio data to the sound and picture synchronization apparatus 101; the audio and video synchronization device 101 is configured to receive the audio data and the video data sent by the main chip 104, and send the audio data and the video data to the audio playing device 102 and the video playing device 103 respectively for playing according to the method for adjusting audio and video synchronization provided by the embodiment of the present invention.
In addition, it should be noted that the audio and video synchronization device 101 and the main chip 104 in the audio and video playing device provided in the embodiment of the present invention may be two separate integrated circuits, processing chips, and the like. The sound-picture synchronization device 101 can also read the film source file in the storage space and decode the film source file to realize the function of the main chip 104. The above design can be performed by those skilled in the art according to the actual needs, and the present invention is not limited thereto.
In addition, it should be noted that each unit device included in the above-mentioned audio/video playing device provided in the embodiment of the present invention may be a plurality of unit devices in the same product device. For example, taking a television as an example, the functions of the main chip 104 and the audio and video synchronization device 101 may be implemented by a corresponding control circuit of the television, the function of the audio playing device 102 may be implemented by a power amplifier speaker unit of the television, and the function of the video playing device 103 may be implemented by a display unit of the television. In addition, the audio/video playing device may further include a plurality of product devices, as shown in fig. 3, the functions of the main chip 104 and the audio/video synchronization device 101 are also implemented by a corresponding control circuit of the television, the function of the video playing device 103 is also implemented by a display unit of the television, but the function of the audio playing device 102 may be implemented by a wireless sound box device connected to the television, the television sends audio data to the wireless sound box device through bluetooth and WIFI during audio/video playing, and the wireless sound box device feeds back a return signal to the television after receiving an audio frame sent by the television each time.
The first embodiment is as follows:
based on the audio/video playing device provided in the above embodiment or other audio/video playing devices having similar structures and functions to the above audio/video playing device, an embodiment of the present invention provides a method for adjusting synchronization of sound and pictures, where the flow is shown in fig. 4, and the method includes:
s201, the sound and picture synchronization device reads the film source file in the storage space, decodes the film source file and generates audio data and video data.
It should be noted that, in the embodiment of the present invention, the audio and video synchronization device may complete the reading of the film source file in the storage space and the decoding of the film source file, or an independent main chip may decode the film source file and then send the audio data and the video data to the audio and video synchronization device.
The audio data and the video data are transmitted, read and played in the form of data frames.
S202, the sound and picture synchronization device sends the audio frame to be played to the audio playing device.
The audio frame to be played refers to a frame of data in the audio data generated after the film source file is decoded. After receiving the audio frame to be played, the audio playing device may output corresponding sound according to the audio frame data to be played.
In addition, the audio playing device also sends a return signal to the sound and picture synchronization device after receiving the audio frame to be played. In one implementation manner, considering that the audio and video synchronization device continuously sends audio frames to the audio playing device during the audio and video playing process, in order to enable the audio and video synchronization device to identify which audio frame the received return signal corresponds to, the embodiment of the present invention further includes:
the audio frame to be played comprises a synchronous identifier, and the synchronous identifier is used for determining the position of the audio frame to be played in the audio data, so that the audio playing device can generate a return signal according to the synchronous identifier.
The position of the audio frame to be played in the audio data refers to which time period of the audio frame in the audio data, for example, the nth minute and nth second of the audio frame in the audio data. For example, when the audio frame to be played is transmitted, the sound-picture synchronization apparatus may add a synchronization flag at the end of each frame of audio data. Wherein the synchronization mark comprises the position of the frame of audio data in the audio data. And then after receiving the audio frame to be played, the audio playing device decomposes the audio data and the synchronous identifier from the data packet of the audio frame to be played, puts the audio data into a power amplifier to push a loudspeaker to sound, generates a return signal according to the synchronous identifier and feeds the return signal back to the sound and picture synchronizing device. The decomposed synchronous identifier can be directly packaged and sent as a return signal. Therefore, after the sound and picture synchronization device receives the return signal generated according to the synchronization identifier, the sound and picture synchronization device can know which audio frame corresponds to the return signal. In addition, the synchronization identifier may further include information such as a time stamp when the audio frame to be played is sent by the audio-visual synchronization device, and a value of the time stamp is obtained by multiplying the number of minutes of the current system time by 60 plus a number of seconds and converting the number of seconds into a binary integer 32.
S203, the sound and picture synchronization device receives the return signal sent by the audio playing device and records the time of receiving the return signal.
Specifically, after receiving the return signal, the sound-picture synchronization device records the time when the return signal is received when determining that the return signal is sent after the audio playing device receives the audio frame to be played.
In the invention, considering that delay may exist in the process of decoding the audio data and the process of sending the audio frame to the audio playing device, especially when the audio data is transmitted through a Bluetooth and WIFI network, the delay fluctuation caused by the complex transmission path is larger, so that a return signal is sent to the sound and picture synchronization device after the audio playing device receives one audio frame, the time for receiving the return signal is recorded by the sound and picture synchronization device, and the time for a user to hear the audio frame is determined.
And S204, the sound and picture synchronization device sends the video frame to be played to the video playing device and records the sending time.
The video frame to be played corresponds to the audio frame to be played.
In the invention, after the video frame to be played corresponding to the audio frame to be played is generated by decoding the film source file, the audio-video synchronizer sends the video frame to be played to the video playing device, so that the video playing device plays the corresponding picture according to the video frame to be played, and simultaneously records the time for the audio-video synchronizer to send the video frame to be played to the video playing device, thereby determining the time for a user to see the video frame to be played.
S205, the sound and picture synchronization device determines the time for sending the next audio frame to the audio playing device according to the time for receiving the return signal and the sending time for sending the video frame to be played to the video playing device.
The time when the user hears the audio frame to be played is determined according to the time when the return signal is received, and the time when the user sees the video frame to be played is determined according to the sending time when the video frame to be played is sent to the video playing device. And then the delay value between video playing and audio playing can be determined by the time of receiving the return signal and the sending time of sending the video frame to be played to the video playing device. Finally, according to the delay value between the video playing and the audio playing, the time for sending the next audio frame to the audio playing device is adjusted, so that the synchronous playing of the audio frame and the video frame is ensured.
In one implementation, step S205 may specifically include:
s2051, using the formula DeltaT(N)＝K×(TVS(N)-TAR(N)) Calculating the delay time delta T for sending the next audio frame to the audio playing device(N)Wherein T isVS(N)Indicating the transmission time, T, of the video frame to be played to the video playback deviceAR(N)And represents the time of receiving the return signal, and K is more than 0 and less than 1.
The value of K can be finely adjusted according to the operation capability of the CPU.
The time difference between the time when the sound and picture synchronization device receives the return signal of the audio frame to be played and the sending time when the video frame to be played is sent is considered, namely the condition of the time difference between the video frame and the audio frame which correspond to each other in the playing process is reflected. And then the invention calculates the time that the next frame audio needs to delay by calculating the difference between the time of receiving the return signal and the sending time of sending the video frame to be played and multiplying the calculated difference by a coefficient K.
S2052, utilizing formula TAS(N+1)＝T’AS(N+1)+△T(N)Determining a time to transmit a next audio frame to an audio playback device, where T'AS(N+1)Indicating the time at which the next audio frame is normally sent to the audio playback device.
The time for normally sending the next audio frame to the audio playing device refers to the sending time when the audio frame sending is not delayed by adopting the method of the invention. Illustratively, for example, in the case of not delaying the transmission of the audio frame according to the method of the present invention, the master chip transmits the audio frame data to the audio playback device every time the master chip decodes one frame of audio frame data, which is T'AS(N+1). Further, the method of the present invention calculates Δ T(N)Then is at T'AS(N+1)Delay delta T on the basis of(N)And then sending the next audio frame data to the audio playing device. For another example, in the case of not delaying the transmission of the audio frame according to the method of the present invention, the original master chip will delay the transmission of the audio frame data T 'to the audio playback device after decoding one frame of audio frame data for a certain time'AS(N+1)The time at which the audio frame data is currently transmitted to the audio playback apparatus. Further, the method of the present invention calculates Δ T(N)Then is at T'AS(N+1)Delay delta T on the basis of(N)And then sending the next audio frame data to the audio playing device.
Specifically, assume that the nth frame of audio frame and video frame is currently being played, and the value of K is 0.5. When the system time of the sound-picture synchronization device sending the nth frame of video frame to the video playing device is 1 point, 12 minutes, 42 seconds and 100 milliseconds, and the time of receiving the return signal of the nth frame of audio frame is 1 minute, 12 minutes, 41 seconds and 200 milliseconds, the system time can be calculated to play the audio frame to the video playing deviceTime delay delta T of next audio frame sent by device(N)900ms × 0.5ms 450 ms. And is at T 'when the next audio frame is transmitted'AS(N+1)Then the transmission is carried out after 0.5ms of delay. Thereby reducing the delay between audio playback and video playback.
In the embodiment of the invention, after the audio and video synchronization device sends the audio frame to be played to the audio playing device, the audio playing device feeds back the return signal to the audio and video synchronization device, and then the audio and video synchronization device can determine the time for the audio playing device to play the audio frame to be played by recording the time for receiving the return signal, namely the time for a user to hear the audio frame to be played. And the sound-picture synchronization device determines the time for the video playing device to play the video frame to be played by recording the time for sending the video frame to be played to the video playing device, namely the time for the user to see the video frame to be played is determined. And then the delay value between video playing and audio playing can be determined by the time of receiving the return signal and the sending time of sending the video frame to be played to the video playing device. Finally, according to the delay value between the video playing and the audio playing, the time for sending the next audio frame to the audio playing device is adjusted, so that the synchronous playing of the audio frame and the video frame is ensured.
The method for adjusting the sound-picture synchronization provided by the invention is described by combining the following examples:
as shown in fig. 5, in the Video data sent from the sound and picture synchronization apparatus to the Video playback apparatus, each Video Frame data includes "Video Frame" and "MVS(*)"two parts," Video Frame "refers to Video data of the first Frame," MVS(*)"refers to the synchronous identification of the first frame video data, wherein" MVS(*)The "may include the frame number of the frame video frame in the video data, the system time when the frame video frame is transmitted, and other information. As shown in the figure, in the Audio data transmitted from the sound and picture synchronization apparatus to the Audio playback apparatus, each Audio Frame data includes "Audio Frame" and "MAS(*)"two parts," Audio Frame "refers to the first Frame of Audio data," MAS(*)"means the number of first frame audioAccording to the synchronous identification,' MAS(*)"may include information such as the number of frames of the frame audio frame in the audio data and the system time when the frame audio frame is transmitted. As shown in the figure, after the Audio playing device receives the Audio Frame, it feeds back the signal T to the Audio and video synchronizerAR(*)。
Taking the video frame and the audio frame of the (N-1) th frame as an example, the master chip decodes the film source file to generate an audio frame of the (N-1) th frame and a video frame of the (N-1) th frame respectively (usually, the audio frame is output earlier than the video frame). Then, the sound-picture synchronization device sends the audio frame of the (N-1) th frame to the audio playing device, sends the video frame of the (N-1) th frame to the video playing device, and records the time T for sending the video frame of the (N-1) th frame to the video playing deviceVS(N-1). The audio playing device feeds back a return signal after receiving the N-1 frame video frame, and the sound and picture synchronization device records the time T of receiving the return signalAR(N-1). Wherein the feedback signal may be a synchronization mark M obtained by decomposing the audio frameAS(N-1)Then, the synchronous signal M is transmittedAS(N-1)As the content of the backtransmission signal. Then calculating Δ T(N-1)＝K×(TVS(N-1)-TAR(N-1)) And determining the delay time for sending the next audio frame to the audio playing device. After the Nth frame audio frame is generated by the main chip, the sound and picture synchronization device delays time delta T(N-1)And then the Nth frame of audio frame is sent to the audio playing device. Therefore, if the problem of picture and sound asynchronism exists when the audio and video of the (N-1) th frame is played, the audio playing can be delayed for a corresponding time length in time by the method, and the picture and sound synchronism when the next frame, namely the N-th frame of audio and video data is played is further ensured. Then, the same method is used to transmit the N frame video frame to the video playing device according to the time TVS(N)And the time T of receiving the return signal corresponding to the Nth frame of video frameAR(N-1)Determining a delay time DeltaT for transmitting the (N + 1) th frame of audio frame to the audio playing device(N)＝K×(TVS(N)-TAR(N)) After the sound and picture synchronizer delays delta T(N)And then the (N + 1) th frame of audio frame is sent to the audio playing device.
Example two:
the embodiment of the invention also provides a method for adjusting the sound-picture synchronization, which is applied to the audio playing device. As shown in fig. 6, the method includes:
s301, the audio playing device receives the audio frame to be played sent by the sound and picture synchronization device.
And S302, after receiving the audio frame to be played, the audio playing device sends a return signal to the sound and picture synchronization device. The sound and picture synchronization device records the time of receiving the return signal after receiving the return signal, so that the sound and picture synchronization device determines the time of sending the next audio frame to the audio playing device according to the time of receiving the return signal and the sending time after recording the sending time of sending the video frame to be played to the video playing device, wherein the video frame to be played corresponds to the audio frame to be played.
The audio frame to be played comprises a synchronous identifier, and the synchronous identifier is used for determining the frame number of the audio frame to be played in the audio data. Further, the method may further include: and the audio playing device generates a return signal according to the synchronous identifier.
The method for adjusting the audio-visual synchronization provided by the second embodiment of the present invention has the same or similar technical features and effects as the method provided by the first embodiment in terms of principle and implementation, the first embodiment describes the method from the perspective of an audio-visual synchronization device, and the second embodiment describes the method from the perspective of an audio playing device. Therefore, various implementation manners and effects in the second embodiment can refer to the descriptions in the first embodiment, and are not described herein again.
Example three:
the embodiment of the invention also provides a sound and picture synchronization device which is used for executing the method for reporting the information by the household appliance. The usage scenario of the audio and video synchronization apparatus provided by the present invention can refer to the corresponding description of the first embodiment. The embodiment of the present invention may perform functional module division on the information reporting apparatus according to the above method example, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated in one processing module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. It should be noted that, the division of the modules in the embodiment of the present invention is schematic, and is only a logic function division, and there may be another division manner in actual implementation. Fig. 7 is a schematic diagram of a possible structure of the sound-picture synchronization apparatus according to the embodiment of the present invention, in a case where each functional module is divided according to each function. Specifically, the sound-picture synchronization device 70 includes: an interface module 701, a recording module 702, and a calculating module 703. Wherein:
the interface module 701 is configured to send an audio frame to be played to an audio playing apparatus.
The interface module 701 is further configured to receive a backhaul signal. The audio playing device sends a return signal to the sound and picture synchronization device after receiving the audio frame to be played.
The recording module 702 is configured to record a time when the interface module receives the return signal.
The interface module 701 is further configured to send a video frame to be played to the video playing apparatus, where the video frame to be played corresponds to the audio frame to be played.
The recording module 702 is further configured to record a sending time for sending the video frame to be played to the video playing apparatus.
The calculating module 703 is configured to determine, according to the time when the return signal is received and the sending time, a time when a next audio frame is sent to the audio playing apparatus.
In one implementation, the audio frame to be played includes a synchronization identifier, where the synchronization identifier is used to determine the number of frames of the audio frame to be played in the audio data. Wherein the return signal is generated by the audio playing device according to the synchronization identifier.
In an implementation manner, the calculating module 703 is specifically configured to:
using the formula DeltaT(N)＝K×(TVS(N)-TAR(N)) Calculating the delay time delta T for sending the next audio frame to the audio playing device(N)Wherein T isVS(N)Indicating the transmission time, T, of the video frame to be played to the video playback deviceAR(N)Indicating the time when the return signal is received, 0 < K < 1.
The calculating module 703 is specifically further configured to utilize the formula TAS(N+1)＝T’AS(N+1)+△T(N)Determining a time to transmit a next audio frame to an audio playback device, where T'AS(N+1)And the time for the sound and picture synchronization device to normally send the next audio frame to the audio playing device is represented.
It should be noted that, for other corresponding descriptions corresponding to each unit in the sound-picture synchronization apparatus provided in the embodiment of the present invention, reference may be made to the corresponding descriptions in the foregoing embodiment, and details are not described here again.
In the case of an integrated unit, fig. 8 shows a schematic view of a possible construction of the above-described picture and sound synchronization device. The sound-picture synchronization device 80 includes: a memory unit 801, a processing unit 802, and an interface unit 803. The processing unit 802 is configured to control and manage actions of the information reporting apparatus 80.
For example, the processing unit is a processor, the storage unit is a memory, and the interface unit is a transceiver. Further, the sound-picture synchronization apparatus is shown in fig. 9, and includes a transceiver 903, a processor 902, a memory 901, and a bus 904, and the transceiver 903, the processor 902 are connected to the memory 901 via the bus 904.
The processor 902 may be a general-purpose Central Processing Unit (CPU), a microprocessor, an Application-Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to control the execution of programs in accordance with the present invention.
The Memory 901 may be a Read-Only Memory (ROM) or other types of static storage devices that can store static information and instructions, a Random Access Memory (RAM) or other types of dynamic storage devices that can store information and instructions, an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Compact Disc Read-Only Memory (CD-ROM) or other optical Disc storage, optical Disc storage (including Compact Disc, laser Disc, optical Disc, digital versatile Disc, blu-ray Disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to these. The memory may be self-contained and coupled to the processor via a bus. The memory may also be integral to the processor.
The memory 901 is used for storing application program codes for executing the scheme of the present invention, and the execution of the application program codes is controlled by the processor 902. The transceiver 903 is used for receiving the content input by the external device, and the processor 902 is used for executing the application program code stored in the memory 901, thereby implementing the method for adjusting the sound and picture synchronization described in the embodiment of the present invention.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented using a software program, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions described in accordance with the embodiments of the invention are all or partially effected when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or can comprise one or more data storage devices, such as a server, a data center, etc., that can be integrated with the medium. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
Example four:
as shown in fig. 10, the present invention further provides an audio playing apparatus 100, including: the interface module 1001, where the interface module 1001 is configured to receive an audio frame to be played, which is sent by the audio and video synchronization apparatus to the audio playing apparatus.
The interface module 1001 is further configured to send a transfer signal to the sound and picture synchronizer after receiving the audio frame to be played. The sound and picture synchronization device records the time of receiving the return signal after receiving the return signal, so that the sound and picture synchronization device determines the time of sending the next audio frame to the audio playing device according to the time of receiving the return signal and the sending time after recording the sending time of sending the video frame to be played to the video playing device, wherein the video frame to be played corresponds to the audio frame to be played.
In one implementation, the audio frame to be played includes a synchronization identifier, where the synchronization identifier is used to determine the number of frames of the audio frame to be played in the audio data.
The interface module 1001 is specifically configured to acquire a synchronization identifier from an audio frame to be played, and generate and send a return signal according to the synchronization identifier.
In one implementation, as shown in fig. 10, the audio playing apparatus 10 further includes a playing module 1002. The playing module 1002 is configured to play audio data in a received audio frame to be played.
For other corresponding descriptions corresponding to each unit in the audio playing device provided in the embodiment of the present invention, reference may be made to the corresponding descriptions in the foregoing embodiments, which are not described herein again.
In another embodiment, the present invention further provides an electronic device, which includes the audio and video synchronization apparatus and the video playing apparatus provided in the third embodiment.
In an implementation manner, the electronic device further includes an audio playing apparatus provided in the fourth embodiment.
Specifically, the electronic device may be any one of a television, a computer, and a mobile phone.
In another embodiment, the present invention further provides a wireless sound box device, including the audio playing apparatus provided in the fourth embodiment.
Specifically, the wireless sound box device can communicate with the sound and picture synchronization device provided by the invention through wireless communication protocols such as bluetooth, WIFI and other proprietary protocols.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (6)
1. A method for adjusting sound and picture synchronization is characterized by comprising the following steps:
the sound and picture synchronization device sends an audio frame to be played to the audio playing device; the audio frame to be played comprises a synchronous identifier, and the synchronous identifier is used for determining the position of the audio frame to be played in audio data;
the sound and picture synchronization device receives a return signal sent by the audio playing device, wherein the audio playing device sends the return signal after receiving the audio frame to be played, and the return signal is generated by the audio playing device according to the synchronization identifier;
the sound and picture synchronization device records the time of receiving the return signal;
the sound and picture synchronization device sends a video frame to be played to a video playing device and records sending time, wherein the video frame to be played corresponds to the audio frame to be played;
and the sound and picture synchronization device determines the time for sending the next audio frame to the audio playing device according to the time for receiving the return signal and the sending time.
2. The method according to claim 1, wherein the audio-visual synchronization device determines a time for sending a next audio frame to the audio playback device according to the time of receiving the feedback signal and the sending time, and specifically comprises:
using the formula DeltaT(N)＝K×(TVS(N)-TAR(N)) Calculating the delay time Delta T for sending the next audio frame to the audio playing device(N)Wherein T isVS(N)Represents said transmission time, TAR(N)Representing the time of receiving the return signal, K is more than 0 and less than 1;
using the formula TAS(N+1)＝T’AS(N+1)+△T(N)Determining a time to send a next audio frame to the audio playback device, where T'AS(N+1)Indicating the time when the next audio frame is normally sent to the audio playback device.
3. A method for adjusting sound and picture synchronization is characterized by comprising the following steps:
the audio playing device receives the audio frame to be played sent by the sound and picture synchronizing device; the audio frame to be played comprises a synchronous identifier, and the synchronous identifier is used for determining the position of the audio frame to be played in audio data;
the audio playing device sends a return signal to the sound and picture synchronization device after receiving the audio frame to be played, so that the sound and picture synchronization device records the time of receiving the return signal after receiving the return signal, the sound and picture synchronization device determines the time of sending the next audio frame to the audio playing device according to the time of receiving the return signal and the sending time after recording the sending time of sending the video frame to be played to the video playing device, wherein the video frame to be played corresponds to the audio frame to be played, and the audio playing device generates the return signal according to the synchronization identifier.
4. A sound and picture synchronization device, comprising:
the interface module is used for sending the audio frame to be played to the audio playing device; the audio frame to be played comprises a synchronous identifier, and the synchronous identifier is used for determining the frame number of the audio frame to be played in the audio data;
the interface module is further configured to receive a return signal, where the audio playing device sends the return signal after receiving the audio frame to be played, and the return signal is generated by the audio playing device according to the synchronization identifier;
the recording module is used for recording the time of receiving the return signal after the interface module receives the return signal;
the interface module is further configured to send a video frame to be played to a video playing device, where the video frame to be played corresponds to the audio frame to be played;
the recording module is further configured to record sending time for sending the video frame to be played to the video playing device;
and the computing module is used for determining the time for sending the next audio frame to the audio playing device according to the time for receiving the return signal and the sending time.
5. The sound-picture synchronization device according to claim 4, wherein the calculation module is specifically configured to:
using the formula DeltaT(N)＝K×(TVS(N)-TAR(N)) Calculating the delay time delta T for sending the next audio frame to the audio playing device(N)Wherein T isVS(N)Represents said transmission time, TAR(N)Representing the time of receiving the return signal, K is more than 0 and less than 1;
using the formula TAS(N+1)＝T’AS(N+1)+△T(N)Determining a time to send a next audio frame to the audio playback device, where T'AS(N+1)And the time for the sound and picture synchronization device to normally send the next audio frame to the audio playing device is represented.
6. An audio playback apparatus, comprising:
the interface module is used for receiving audio frames to be played, which are sent to the audio playing device by the sound and picture synchronization device, wherein the audio frames to be played comprise synchronization marks, and the synchronization marks are used for determining the number of the audio frames to be played in audio data;
the interface module is further configured to send a return signal to the sound and picture synchronization device after receiving the audio frame to be played, so that the sound and picture synchronization device records the time of receiving the return signal after receiving the return signal, so that the sound and picture synchronization device determines the time of sending a next audio frame to the audio playing device according to the time of receiving the return signal and the sending time after recording the sending time of sending the video frame to be played to the video playing device, where the video frame to be played corresponds to the audio frame to be played;
the interface module is specifically configured to acquire the synchronization identifier from the audio frame to be played, and generate and send a return signal according to the synchronization identifier.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810374722.1A CN108377406B (en) | 2018-04-24 | 2018-04-24 | Method and device for adjusting sound and picture synchronization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810374722.1A CN108377406B (en) | 2018-04-24 | 2018-04-24 | Method and device for adjusting sound and picture synchronization |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108377406A CN108377406A (en) | 2018-08-07 |
| CN108377406B true CN108377406B (en) | 2020-12-22 |
Family
ID=63032744
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810374722.1A Active CN108377406B (en) | 2018-04-24 | 2018-04-24 | Method and device for adjusting sound and picture synchronization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108377406B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109168059B (en) * | 2018-10-17 | 2021-06-18 | 上海赛连信息科技有限公司 | Lip sound synchronization method for respectively playing audio and video on different devices |
| CN109379619B (en) * | 2018-11-20 | 2021-05-18 | 海信视像科技股份有限公司 | Sound and picture synchronization method and device |
| CN109379613B (en) * | 2018-12-21 | 2021-11-09 | 深圳Tcl新技术有限公司 | Audio and video synchronization adjustment method, television, computer readable storage medium and system |
| CN109819303B (en) * | 2019-03-06 | 2021-04-23 | Oppo广东移动通信有限公司 | Data output method and related equipment |
| CN110430457B (en) * | 2019-07-25 | 2021-09-10 | 北京奇艺世纪科技有限公司 | Method and device for playing different-end audio and video and audio playing system |
| CN110545454A (en) * | 2019-08-27 | 2019-12-06 | 北京奇艺世纪科技有限公司 | Data synchronous playing method and device |
| CN111988603A (en) * | 2020-08-12 | 2020-11-24 | 深圳创维－Rgb电子有限公司 | External power amplifier debugging method and device, electronic equipment and storage medium |
| CN114827696B (en) * | 2021-01-29 | 2023-06-27 | 华为技术有限公司 | Method for synchronously playing audio and video data of cross-equipment and electronic equipment |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6055247A (en) * | 1995-07-13 | 2000-04-25 | Sony Corporation | Data transmission method, data transmission apparatus and data transmission system |
| JP2004193868A (en) * | 2002-12-10 | 2004-07-08 | Alps Electric Co Ltd | Wireless transmission and reception system and wireless transmission and reception method |
| EP1736000A1 (en) * | 2004-04-07 | 2006-12-27 | Koninklijke Philips Electronics N.V. | Video-audio synchronization |
| JP2012191583A (en) * | 2011-03-14 | 2012-10-04 | Yamaha Corp | Signal output device |
| KR102201617B1 (en) * | 2014-01-07 | 2021-01-12 | 삼성전자 주식회사 | Av device and control method thereof |
| CN104980820B (en) * | 2015-06-17 | 2018-09-18 | 小米科技有限责任公司 | Method for broadcasting multimedia file and device |
| US9886233B2 (en) * | 2015-10-23 | 2018-02-06 | Echostar Technologies L.L.C. | Apparatus, systems and methods for audio content diagnostics |
-
2018
- 2018-04-24 CN CN201810374722.1A patent/CN108377406B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108377406A (en) | 2018-08-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108377406B (en) | Method and device for adjusting sound and picture synchronization | |
| JP6522014B2 (en) | Video remote commentary synchronization method and system and terminal device | |
| US10951857B2 (en) | Method and system for video recording | |
| JP6626440B2 (en) | Method and apparatus for playing multimedia files | |
| US10394518B2 (en) | Audio synchronization method and associated electronic device | |
| CN109167890B (en) | Sound and picture synchronization method and device and display equipment | |
| EP3697088A1 (en) | Video sending and receiving method, device, and terminal | |
| US20070111801A1 (en) | Method, apparatus and system for transmitting and receiving media data | |
| US20170168660A1 (en) | Voice bullet screen generation method and electronic device | |
| US9820076B2 (en) | Information processing method and electronic device | |
| CN108810575B (en) | Method and device for sending target video | |
| US20240154837A1 (en) | System and method for controlling playback or recording of media assets based on a state of a secondary device | |
| CN108616767B (en) | Audio data transmission method and device | |
| US20060197880A1 (en) | Signal processing device and stream processing method | |
| JP2018182390A (en) | Control method, transmission device, and reception device | |
| CN103401580A (en) | Voice recording and playing interphone | |
| KR20170002245A (en) | Signal Processing Unit, Sound Source playing Device and Method using the same | |
| JP2006186689A (en) | Signal processor and stream processing method | |
| EP1796306A1 (en) | Data communication system, data reproducing apparatus and data reproducing method | |
| CN113905321A (en) | Object-based audio channel metadata and generation method, device and storage medium | |
| JP2005110024A (en) | Data transmitter, data transmission/reception system, and data transmission/reception method | |
| KR102292416B1 (en) | System and method for providing music streaming service | |
| US20230080857A1 (en) | Audio-visual offset process | |
| CN112069356B (en) | Video file storage method and device, electronic equipment and readable storage medium | |
| US20240153522A1 (en) | Audio playback method, electronic device, and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 266555 Qingdao economic and Technological Development Zone, Shandong, Hong Kong Road, No. 218 Applicant after: Hisense Video Technology Co., Ltd Address before: 266100 Zhuzhou Road, Laoshan District, Shandong, No. 151, No. Applicant before: HISENSE ELECTRIC Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |