CN109168059A - A kind of labial synchronization method playing audio & video respectively on different devices - Google Patents
A kind of labial synchronization method playing audio & video respectively on different devices Download PDFInfo
- Publication number
- CN109168059A CN109168059A CN201811210525.2A CN201811210525A CN109168059A CN 109168059 A CN109168059 A CN 109168059A CN 201811210525 A CN201811210525 A CN 201811210525A CN 109168059 A CN109168059 A CN 109168059A
- Authority
- CN
- China
- Prior art keywords
- video
- audio
- playback apparatus
- video playback
- synchronization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 230000007246 mechanism Effects 0.000 claims abstract description 19
- 230000003139 buffering effect Effects 0.000 claims description 20
- 230000005540 biological transmission Effects 0.000 claims description 12
- 238000001914 filtration Methods 0.000 claims description 11
- 230000001360 synchronised effect Effects 0.000 claims description 9
- 238000012790 confirmation Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 7
- 238000005516 engineering process Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 208000032370 Secondary transmission Diseases 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/4104—Peripherals receiving signals from specially adapted client devices
- H04N21/4122—Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
- H04N21/43632—Adapting the video stream to a specific local network, e.g. a Bluetooth® network involving a wired protocol, e.g. IEEE 1394
- H04N21/43635—HDMI
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
- H04N21/43637—Adapting the video stream to a specific local network, e.g. a Bluetooth® network involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a kind of labial synchronization methods for playing audio & video respectively on different devices, the distinct device includes audio-frequence player device and video playback apparatus, including using transmitting terminal to send audio-video code to the audio-frequence player device and video playback apparatus, the data packet that the transmitting terminal sends it increases timestamp information, and the audio-frequence player device and video playback apparatus are respectively using synchronization mechanism to be played simultaneously the audio and video information for having identical time stamp.The labial synchronization that the present invention designs plays mechanism, can cope with 99% network jitter situation, guarantee that, when network quality is unstable, audio-video on different devices still can be played simultaneously.
Description
Technical field
The present invention relates to multimedia technology field, in particular to it is a kind of play respectively on different devices audio with
The labial synchronization method of video.
Background technique
In video conference scene, the terminal each attended a meeting receives video and audio code from other participants
Stream, and by local image display (such as display screen) and audio playing device (such as loudspeaker) the view received
Frequency and audio are shown and are played.Since audio-video code stream is received and played by same equipment, so corresponding sound
Frequency and video image are labial synchronizations when playing, i.e., shape of the mouth as one speaks when participant speaks in video image is with corresponding sound
It is consistent.
But in the case where no human intervention, the play time of sound and video has due to the unstability of IP network
It may be inconsistent.Cause sound that user experiences and video asynchronous, i.e. labial is asynchronous.
Such as under some special scenes (such as Fig. 1-2), to sound view and video reception and play sometimes in difference
Equipment on carry out, for example the reception of video image is completed by the video terminal in meeting room, and the reception of audio is then
By what is completed with the terminal by wireless microphone+speaker that certain mode matches connection；Or such as intelligent terminal all-in-one machine
NE60 is the audio/video communication equipment that a suitable desktop or small conference room use, touch screen manipulation, easy-to-use, it can be with
It is matched with ME terminal, realization plays sound using the audio frequency apparatus on the NE60 on desktop, and by being connected with ME terminal
Video screen output video image.Under these scenes, sound and video will be received and be broadcast from different equipment
It puts and shows.Since two independent equipment receive audio, video data code stream, and the unstability of IP network, sound by IP network
The time that sound and video reach distinct device may be different, if each equipment directly plays received audio-video code
Stream will have the nonsynchronous problem of audio-video, i.e., the shape of the mouth as one speaks and corresponding sound when shown participant speaks are not
It is consistent.
What the present invention solved is exactly when transmitting sound and video data respectively under the conditions of using IP network to distinct device,
The problem of carrying out audio-visual synchronization broadcasting.
Summary of the invention
The invention proposes a kind of labial synchronization method for playing audio & video respectively on different devices, the differences
Equipment includes audio-frequence player device and video playback apparatus, which is characterized in that the method includes,
Audio-video code is sent to the audio-frequence player device and video playback apparatus using transmitting terminal, and the transmitting terminal is to it
The data packet of transmission increases timestamp information, and the audio-frequence player device and video playback apparatus use synchronization mechanism with same respectively
Step plays the audio and video information for having identical time stamp.
Further, audio-frequence player device and video playback apparatus are that sound and video respectively maintain a buffering team
Column, a certain number of data packets to be buffered and then are played out by the synchronization mechanism.
It further, further include that the audio playing device is set as masters in synchronization mechanism, from local
Voice data is equably played in buffering queue, while regularly being synchronized and being disappeared to video display apparatus transmission by a period of time
The acquisition time stamp for the voice data being currently played is synchronized to video display apparatus by breath, and video display apparatus is according to institute
The timestamp received controls the buffering queue length of video data, carries out appropriate broadcasting, to guarantee the same of sound and video
Step
Further, as soon as the every data for playing time cycle of the audio playing device, to video playback apparatus
A synchronization message is sent, after the video playback apparatus receives synchronization message, one is returned and confirms message to the transmitting terminal,
It include the currently transmitted timestamp from audio-frequence player device received in the confirmation message；When confirmation message is sent
When termination receives, system time when transmitting terminal subtracts transmission with currently received system time is to confirm under current network
Round trip delay time value Δ；
The transmitting terminal is sent to the sound after a weighting period is weighted processing to the round trip delay time value Δ
Sound playback equipment, the audio playing device is by acquisition time stamp plus sending out after current weighting treated round trip delay time value Δ
It send to the video playback apparatus, the video playback apparatus carries out being played simultaneously for video according to the timestamp.
Further, described processing is weighted to the round trip delay time value Δ includes, by round trip delay time value Δ divided by 2
It is handled to one-way delay value and to the one-way delay value using filtering algorithm.
Further, the filtering algorithm includes that the one-way delay value in a period of time is weighted and averaged to obtain
One-way delay value after filtering processing.
It is preferred that or further including the video playback apparatus being set as masters in synchronization mechanism, and pass through optionally
Video playback apparatus collects the nonsynchronous difference of audio-video, and carries out the difference and a preset threshold after being filtered to it
Compare, this deviation is sent to audio playing device when the difference is greater than the preset threshold and according to the deviation meeting
The buffering queue of audio playing device is increased into corresponding length.
In the case where no human intervention, for the play time of sound and video due to the unstability of IP network, having can
It can be inconsistent.Cause sound that user experiences and video asynchronous, i.e. labial is asynchronous.And it is designed based on above method
Labial synchronization play mechanism, 99% network jitter situation can be coped with, guarantee to set when network quality is unstable in difference
Standby upper audio-video still can be played simultaneously.
Detailed description of the invention
From following description with reference to the accompanying drawings it will be further appreciated that the present invention.Component in figure is not drawn necessarily to scale,
But it focuses on and shows in the principle of embodiment.In the figure in different views, identical appended drawing reference is specified to be corresponded to
Part.
Fig. 1 is one schematic diagram of application scenarios of the invention；
Fig. 2 is two schematic diagram of application scenarios of the invention；
Fig. 3, which is synchronization mechanism-audio-frequence player device of the invention, sends synchronization message schematic diagram to video playback apparatus；
Fig. 4, which is synchronization mechanism-video playback apparatus of the invention, sends synchronization message schematic diagram to audio-frequence player device.
Specific embodiment
In order to enable the objectives, technical solutions, and advantages of the present invention are more clearly understood, below in conjunction with embodiment, to this
Invention is further elaborated；It should be appreciated that described herein, the specific embodiments are only for explaining the present invention, and does not have to
It is of the invention in limiting.To those skilled in the art, after access is described in detail below, other systems of the present embodiment
System, method and/or feature will become obvious.All such additional systems, method, feature and advantage are intended to be included in
It in this specification, is included within the scope of the invention, and by the protection of the appended claims.In description described in detail below
The other feature of the disclosed embodiments, and these characteristic roots will be apparent according to described in detail below.
Embodiment one:
The invention proposes a kind of labial synchronization method for playing audio & video respectively on different devices, the differences
Equipment includes audio-frequence player device and video playback apparatus, which is characterized in that the method includes,
Audio-video code is sent to the audio-frequence player device and video playback apparatus using transmitting terminal, and the transmitting terminal is to it
The data packet of transmission increases timestamp information, and the audio-frequence player device and video playback apparatus use synchronization mechanism with same respectively
Step plays the audio and video information for having identical time stamp.
Further, audio-frequence player device and video playback apparatus are that sound and video respectively maintain a buffering team
Column, a certain number of data packets to be buffered and then are played out by the synchronization mechanism.
It further, further include that the audio playing device is set as masters in synchronization mechanism, from local
Voice data is equably played in buffering queue, while regularly being synchronized and being disappeared to video display apparatus transmission by a period of time
The acquisition time stamp for the voice data being currently played is synchronized to video display apparatus by breath, and video display apparatus is according to institute
The timestamp received controls the buffering queue length of video data, carries out appropriate broadcasting, to guarantee the same of sound and video
Step
Further, as soon as the every data for playing time cycle of the audio playing device, to video playback apparatus
A synchronization message is sent, after the video playback apparatus receives synchronization message, one is returned and confirms message to the transmitting terminal,
It include the currently transmitted timestamp from audio-frequence player device received in the confirmation message；When confirmation message is sent
When termination receives, system time when transmitting terminal subtracts transmission with currently received system time is to confirm under current network
Round trip delay time value Δ；
The transmitting terminal is sent to the sound after a weighting period is weighted processing to the round trip delay time value Δ
Sound playback equipment, the audio playing device is by acquisition time stamp plus sending out after current weighting treated round trip delay time value Δ
It send to the video playback apparatus, the video playback apparatus carries out being played simultaneously for video according to the timestamp.
Further, described processing is weighted to the round trip delay time value Δ includes, by round trip delay time value Δ divided by 2
It is handled to one-way delay value and to the one-way delay value using filtering algorithm.The filtering algorithm includes, will be in a period of time
One-way delay value be weighted and averaged with the one-way delay value after being filtered.
Certainly, the filtering method is a kind of example, other available filtering methods can also be used as replacement, but this
Method tested in the present embodiment effect preferably or it is best and as recommendation.
Embodiment two:
The invention proposes a kind of labial synchronization method for playing audio & video respectively on different devices, the differences
Equipment includes audio-frequence player device and video playback apparatus, which is characterized in that the method includes,
Audio-video code is sent to the audio-frequence player device and video playback apparatus using transmitting terminal, and the transmitting terminal is to it
The data packet of transmission increases timestamp information, and the audio-frequence player device and video playback apparatus use synchronization mechanism with same respectively
Step plays the audio and video information for having identical time stamp.
Further, audio-frequence player device and video playback apparatus are that sound and video respectively maintain a buffering team
Column, a certain number of data packets to be buffered and then are played out by the synchronization mechanism.
In the present embodiment, the method also includes the video playback apparatus is set as masters in synchronization mechanism,
And the nonsynchronous difference of audio-video is collected by video playback apparatus, and by the difference and a default threshold after being filtered to it
Value is compared, when the difference be greater than the preset threshold when by this deviation be sent to audio playing device and according to this partially
The buffering queue of audio playing device can be increased corresponding length by difference.The similar previous implementation of filter processing method therein
Example, or other filtering methods that can be common using this field, repeat no more again.
Embodiment three:
In the present embodiment, to solve the problems, such as that labial is asynchronous, the transmitting terminal in audio-video code stream is needed, it is sent
Data packet increases timestamp information, i.e., timestamp letter when carrying its acquisition in each audio, video data packet sent
Breath, in the packets of audio data and video data packet of synchronization Tx acquisition, it is all TSx that the timestamp carried, which is identical,.
In receiving end, since audio and video are received and played by different equipment respectively, in receiving end
Two equipment can extract respectively the acquisition time in audio and video data packet stamp.It is led to cope with network jitter
The nonsynchronous phenomenon of the audio and video playing of cause respectively maintains one on two distinct devices of receiving end for sound and video
Buffering queue, it is intended to mitigate the influence of network jitter, when exchanging broadcasting for certain delay it is smooth with it is synchronous.Receiving device exists
After receiving audio-video code stream, it is not directly unconditional broadcasting, but is first put into buffering queue, a certain number of numbers to be buffered
It is played out according to packet and then by synchronization mechanism.
Audio playing device A1 is set as masters by the present invention in synchronization mechanism, it defaults can be from local buffering queue
In equably play voice data, meanwhile, it also regularly (every 20ms) to video display apparatus V1 send synchronization message,
The acquisition time stamp for the voice data being currently played is synchronized to video display apparatus V1, and video display apparatus is according to being received
Timestamp control the buffering queue length of video data, carry out appropriate broadcasting, to guarantee that sound is synchronous with video.
(figure three)
Since the synchronization message between A1 and V1 sends transmission also with IP network, there is also certain systems to prolong
When, and as network jitter and caused by random delay.Therefore it needs that following method is taken to eliminate synchronization message delay institute's band
The error come:
The every data for playing 20ms of audio playing device A1, just send a synchronization message to video playback apparatus V1.Together
This packet is carried in data packet in step message when sending, the local system time of audio playing device A1 stabs LT1, and local
The acquired original time stamp T S1 of playing audio-fequency data.After video playback apparatus V1 receives synchronization message, one can be immediately returned to really
Recognize message.The sending time from audio-frequence player device A1 received is carried in data packet in confirmation message stabs LT1.When
When confirmation message is received by transmitting terminal, transmitting terminal subtracts system time when transmission with currently received system time LT2
LT1 can confirm the round trip delay time value Δ under current network between A1 and V1.Round trip delay time value Δ is divided by 2, as unidirectionally
Delay value.After having obtained the one-way delay value of each packet, data are handled using filtering algorithm, such as will a period of time
Interior one-way delay value is weighted and averaged, and uses this value as compensation of delay Δ 1.After obtaining delay value Δ 1, one at A1
When the secondary transmission synchronization message to V1, the sound collection timestamp in data packet will be modified, it is changed to TS2+ Δ by original value TS2
1.Video playback apparatus V1 will receive the voice data time stamp T S2+ Δ 1 modified, and will carry out video according to this timestamp
Be played simultaneously.The shake of the transmission delay of IP network is sometimes bigger, and the one-way delay value Δ of single calculation may change
Frequently.But due to the calculating by the way that in a period of time each synchronization packets are involved in delay value, by filtered delay
Data can all achieve the effect that relatively good in speed and accuracy.
Under normal circumstances, the bandwidth as occupied by video image data can (i.e. video data packet more much higher than voice data
It is more much larger than audio, video data packet), so usually voice data transmits faster than image data in transmission process,
Receiving end equipment receives and starts to play earlier.In some extreme environments, for example video data packet is transmitted across slowly, delay
It is excessive, it will lead to the video data cached in the buffering queue of vision facilities V1 and be all played, not new data arrive
It reaches, queue is entirely sky.The one-way synchronization message sent at this time from audio playing device A1 not can guarantee audio-visual synchronization yet.Cause
This, needs to collect the nonsynchronous difference of audio-video on video playback apparatus V1, and be filtered it elimination shaking interference.Such as
Fruit finds that the deviation of long period is excessive, then this deviation can be sent to audio playing device A1.Audio playing device A1 meeting
The buffering queue of oneself is increased into corresponding length, is equivalent to and more voice data packets is allowed to enter buffering queue, to wait view
The successful reception of frequency evidence, to guarantee final audio-visual synchronization.(figure four)
Although describing the present invention by reference to various embodiments above, but it is to be understood that of the invention not departing from
In the case where range, many changes and modifications can be carried out.That is methods discussed above, system or equipment etc. show
Example.Various configurations can be omitted suitably, replace or add various processes or component.For example, in alternative configuration, can with
Described order in a different order executes method, and/or can add, and omits and/or combine the various stages.Moreover, about
The feature of certain configuration descriptions can be combined with various other configurations.Can combine in a similar way configuration different aspect and
Element.In addition, many elements are only range of the example without limiting the disclosure or claims with the development of technology.
Give detail in the description to provide to the thorough understanding for including the exemplary configuration realized.However,
Configuration can be practiced without these specific details for example, having been illustrated with well-known circuit, process, calculation
Method, structure and technology are without unnecessary details, to avoid fuzzy configuration.The description only provides example arrangement, and unlimited
The scope of the claims processed, applicability or configuration.It is used on the contrary, front will provide the description of configuration for those skilled in the art
Realize the enabled description of described technology.It, can be to the function of element without departing from the spirit or the scope of the present disclosure
It can and arrange and carry out various changes.
In addition, many operations can be in parallel or concurrently although each operation can describe the operations as sequential process
It executes.Furthermore it is possible to rearrange the sequence of operation.One process may have other steps.Furthermore, it is possible to pass through hardware, soft
Part, firmware, middleware, code, hardware description language or any combination thereof carry out the example of implementation method.When software, firmware, in
Between when realizing in part or code, program code or code segment for executing necessary task can store in such as storage medium
In non-transitory computer-readable medium, and described task is executed by processor.
To sum up, be intended to foregoing detailed description be considered as it is illustrative and not restrictive, and it is to be understood that described
Claim (including all equivalents) is intended to limit the spirit and scope of the present invention.The above embodiment is interpreted as only using
In illustrating the present invention rather than limit the scope of the invention.After the content for having read record of the invention, technology
Personnel can make various changes or modifications the present invention, these equivalence changes and modification equally fall into the claims in the present invention and limited
Fixed range.
Claims (7)
1. playing the labial synchronization method of audio & video respectively on different devices, the distinct device includes that audio broadcasting is set
Standby and video playback apparatus, which is characterized in that the method includes,
Audio-video code is sent to the audio-frequence player device and video playback apparatus using transmitting terminal, and the transmitting terminal sends it
Data packet increase timestamp information, the audio-frequence player device and video playback apparatus respectively using synchronization mechanism with synchronize broadcast
Put the audio and video information for having identical time stamp.
2. the method as described in claim 1, which is characterized in that audio-frequence player device and video playback apparatus are sound and video
It respectively maintains a buffering queue, a certain number of data packets to be buffered and then is played out by the synchronization mechanism.
3. method according to claim 2, which is characterized in that its further include by the audio playing device in synchronization mechanism
Masters are set as, voice data are equably played from local buffering queue, while by a period of time regularly to view
Frequency display equipment sends synchronization message, the acquisition time of the voice data being currently played stamp is synchronized to video shows and set
Standby, video display apparatus controls the buffering queue length of video data according to the timestamp received, carries out appropriate broadcasting, from
And guarantee that sound is synchronous with video.
4. method as claimed in claim 3, which is characterized in that audio playing device one time cycle of every broadcasting
Data just send a synchronization message to video playback apparatus, after the video playback apparatus receives synchronization message, return to one
Confirm message to the transmitting terminal, include in the confirmation message receive from audio-frequence player device it is currently transmitted when
Between stab；When confirmation message is received by transmitting terminal, when system when transmitting terminal subtracts transmission with currently received system time
Between to confirm round trip delay time value Δ under current network；
The transmitting terminal is sent to the sound after a weighting period is weighted processing to the round trip delay time value Δ and broadcasts
Put equipment, the audio playing device is by acquisition time stamp plus being sent to after current weighting treated round trip delay time value Δ
The video playback apparatus, the video playback apparatus carry out being played simultaneously for video according to the timestamp.
5. method as claimed in claim 4, which is characterized in that described to be weighted processing packet to the round trip delay time value Δ
It includes, round trip delay time value Δ is obtained into one-way delay value divided by 2 and the one-way delay value is handled using filtering algorithm.
6. method as claimed in claim 5, which is characterized in that the filtering algorithm includes, by unidirectionally prolonging in a period of time
Duration is weighted and averaged with the one-way delay value after being filtered.
7. method according to claim 2, which is characterized in that its further include by the video playback apparatus in synchronization mechanism
It is set as masters, and the nonsynchronous difference of audio-video is collected by video playback apparatus, and by the difference after being filtered to it
Value is compared with a preset threshold, this deviation is sent to sound broadcasting when the difference is greater than the preset threshold and is set
The buffering queue of audio playing device can be increased corresponding length for and according to the deviation.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811210525.2A CN109168059B (en) | 2018-10-17 | 2018-10-17 | Lip sound synchronization method for respectively playing audio and video on different devices |
| CN202110568007.3A CN113286184B (en) | 2018-10-17 | 2018-10-17 | Lip synchronization method for respectively playing audio and video on different devices |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811210525.2A CN109168059B (en) | 2018-10-17 | 2018-10-17 | Lip sound synchronization method for respectively playing audio and video on different devices |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110568007.3A Division CN113286184B (en) | 2018-10-17 | 2018-10-17 | Lip synchronization method for respectively playing audio and video on different devices |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109168059A true CN109168059A (en) | 2019-01-08 |
| CN109168059B CN109168059B (en) | 2021-06-18 |
Family
ID=64878546
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110568007.3A Active CN113286184B (en) | 2018-10-17 | 2018-10-17 | Lip synchronization method for respectively playing audio and video on different devices |
| CN201811210525.2A Active CN109168059B (en) | 2018-10-17 | 2018-10-17 | Lip sound synchronization method for respectively playing audio and video on different devices |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110568007.3A Active CN113286184B (en) | 2018-10-17 | 2018-10-17 | Lip synchronization method for respectively playing audio and video on different devices |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN113286184B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109819303A (en) * | 2019-03-06 | 2019-05-28 | Oppo广东移动通信有限公司 | Data output method and relevant device |
| CN114124631A (en) * | 2021-11-15 | 2022-03-01 | 四川九洲空管科技有限责任公司 | Processing method suitable for audio frequency synchronous control between embedded equipment of airborne passenger cabin |
| CN114173208A (en) * | 2021-11-30 | 2022-03-11 | 广州番禺巨大汽车音响设备有限公司 | Audio and video playing control method and device of sound box system based on HDMI (high-definition multimedia interface) |
| CN114554270A (en) * | 2022-02-28 | 2022-05-27 | 维沃移动通信有限公司 | Audio and video playing method and device |
| CN114827681A (en) * | 2022-04-24 | 2022-07-29 | 咪咕视讯科技有限公司 | Video synchronization method and device, electronic equipment, terminal equipment and storage medium |
| CN114827696A (en) * | 2021-01-29 | 2022-07-29 | 华为技术有限公司 | Method for synchronously playing cross-device audio and video data and electronic device |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11856275B1 (en) * | 2022-10-19 | 2023-12-26 | For Eyes Ug (Haftungsbeschraenkt) | Video reproduction system and media reproduction system and method of synchronized reproducing of a video data stream of an audio-visual data stream and computer-readable storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103024517A (en) * | 2012-12-17 | 2013-04-03 | 四川九洲电器集团有限责任公司 | Method for synchronously playing streaming media audios and videos based on parallel processing |
| CN103905878A (en) * | 2014-03-13 | 2014-07-02 | 北京奇艺世纪科技有限公司 | Video data and audio data synchronized playing method and device and equipment |
| CN104618786A (en) * | 2014-12-22 | 2015-05-13 | 深圳市腾讯计算机系统有限公司 | Audio/video synchronization method and device |
| CN104853239A (en) * | 2015-04-27 | 2015-08-19 | 浙江生辉照明有限公司 | Audio and video synchronous playback control method and system |
| CN106658135A (en) * | 2016-12-28 | 2017-05-10 | 北京奇艺世纪科技有限公司 | Audio and video playing method and device |
| CN106792073A (en) * | 2016-12-29 | 2017-05-31 | 北京奇艺世纪科技有限公司 | Method, playback equipment and system that the audio, video data of striding equipment is synchronously played |
| CN106791271A (en) * | 2016-12-02 | 2017-05-31 | 福建星网智慧科技股份有限公司 | A kind of audio and video synchronization method |
| US20170331756A1 (en) * | 2016-05-12 | 2017-11-16 | Citrix Systems, Inc. | Interactivity driven error correction for audio communication in lossy packet-switched networks |
| CN107801080A (en) * | 2017-11-10 | 2018-03-13 | 普联技术有限公司 | A kind of audio and video synchronization method, device and equipment |
| CN108377406A (en) * | 2018-04-24 | 2018-08-07 | 青岛海信电器股份有限公司 | A kind of adjustment sound draws the method and device of synchronization |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7843974B2 (en) * | 2005-06-30 | 2010-11-30 | Nokia Corporation | Audio and video synchronization |
| CN101212690B (en) * | 2006-12-26 | 2011-04-20 | 中兴通讯股份有限公司 | Method for testing lip synchronization for multimedia audio/video stream |
| CN100579238C (en) * | 2008-02-22 | 2010-01-06 | 上海华平信息技术股份有限公司 | Synchronous playing method for audio and video buffer |
| CN103269448A (en) * | 2013-05-24 | 2013-08-28 | 浙江工商大学 | Method for achieving synchronization of audio and video on the basis of RTP/RTCP feedback early-warning algorithm |
| CN103905880A (en) * | 2014-03-13 | 2014-07-02 | 北京奇艺世纪科技有限公司 | Playing method of audio data and video data, smart television set and mobile equipment |
| TWI548278B (en) * | 2014-03-25 | 2016-09-01 | 鴻海精密工業股份有限公司 | Audio/video synchronization device and audio/video synchronization method |
| CN104735470B (en) * | 2015-02-11 | 2018-06-19 | 海信集团有限公司 | A kind of streaming media data transmission method and device |
| US9532099B2 (en) * | 2015-03-24 | 2016-12-27 | Intel Corporation | Distributed media stream synchronization control |
-
2018
- 2018-10-17 CN CN202110568007.3A patent/CN113286184B/en active Active
- 2018-10-17 CN CN201811210525.2A patent/CN109168059B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103024517A (en) * | 2012-12-17 | 2013-04-03 | 四川九洲电器集团有限责任公司 | Method for synchronously playing streaming media audios and videos based on parallel processing |
| CN103905878A (en) * | 2014-03-13 | 2014-07-02 | 北京奇艺世纪科技有限公司 | Video data and audio data synchronized playing method and device and equipment |
| CN104618786A (en) * | 2014-12-22 | 2015-05-13 | 深圳市腾讯计算机系统有限公司 | Audio/video synchronization method and device |
| CN104853239A (en) * | 2015-04-27 | 2015-08-19 | 浙江生辉照明有限公司 | Audio and video synchronous playback control method and system |
| US20170331756A1 (en) * | 2016-05-12 | 2017-11-16 | Citrix Systems, Inc. | Interactivity driven error correction for audio communication in lossy packet-switched networks |
| CN106791271A (en) * | 2016-12-02 | 2017-05-31 | 福建星网智慧科技股份有限公司 | A kind of audio and video synchronization method |
| CN106658135A (en) * | 2016-12-28 | 2017-05-10 | 北京奇艺世纪科技有限公司 | Audio and video playing method and device |
| CN106792073A (en) * | 2016-12-29 | 2017-05-31 | 北京奇艺世纪科技有限公司 | Method, playback equipment and system that the audio, video data of striding equipment is synchronously played |
| CN107801080A (en) * | 2017-11-10 | 2018-03-13 | 普联技术有限公司 | A kind of audio and video synchronization method, device and equipment |
| CN108377406A (en) * | 2018-04-24 | 2018-08-07 | 青岛海信电器股份有限公司 | A kind of adjustment sound draws the method and device of synchronization |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109819303A (en) * | 2019-03-06 | 2019-05-28 | Oppo广东移动通信有限公司 | Data output method and relevant device |
| CN109819303B (en) * | 2019-03-06 | 2021-04-23 | Oppo广东移动通信有限公司 | Data output method and related equipment |
| CN114827696A (en) * | 2021-01-29 | 2022-07-29 | 华为技术有限公司 | Method for synchronously playing cross-device audio and video data and electronic device |
| CN114124631A (en) * | 2021-11-15 | 2022-03-01 | 四川九洲空管科技有限责任公司 | Processing method suitable for audio frequency synchronous control between embedded equipment of airborne passenger cabin |
| CN114124631B (en) * | 2021-11-15 | 2023-10-27 | 四川九洲空管科技有限责任公司 | Processing method suitable for audio synchronous control between embedded equipment of aircraft cabin |
| CN114173208A (en) * | 2021-11-30 | 2022-03-11 | 广州番禺巨大汽车音响设备有限公司 | Audio and video playing control method and device of sound box system based on HDMI (high-definition multimedia interface) |
| CN114554270A (en) * | 2022-02-28 | 2022-05-27 | 维沃移动通信有限公司 | Audio and video playing method and device |
| CN114827681A (en) * | 2022-04-24 | 2022-07-29 | 咪咕视讯科技有限公司 | Video synchronization method and device, electronic equipment, terminal equipment and storage medium |
| CN114827681B (en) * | 2022-04-24 | 2024-03-22 | 咪咕视讯科技有限公司 | Video synchronization method, device, electronic equipment, terminal equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113286184A (en) | 2021-08-20 |
| CN113286184B (en) | 2024-01-30 |
| CN109168059B (en) | 2021-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109168059A (en) | A kind of labial synchronization method playing audio & video respectively on different devices | |
| CN111010614A (en) | Method, device, server and medium for displaying live caption | |
| EP2328349B1 (en) | Information processing system and information processing device | |
| CN107018466B (en) | Enhanced audio recording | |
| CN109089130A (en) | A kind of method and apparatus for the timestamp adjusting live video | |
| CN102932676B (en) | Self-adaptive bandwidth transmitting and playing method based on audio and video frequency synchronization | |
| CN103888815B (en) | Method and system for real-time separation treatment and synchronization of audio and video streams | |
| CN109379619A (en) | Sound draws synchronous method and device | |
| CN109565466A (en) | More equipment room labial synchronization method and apparatus | |
| CN109379613A (en) | Audio-visual synchronization method of adjustment, TV, computer readable storage medium and system | |
| CN111147362B (en) | Multi-user instant messaging method, system, device and electronic equipment | |
| CN101887739B (en) | Method and device for synchronizing media play | |
| CN104954727B (en) | Audio-visual synchronization control device and method | |
| CN101710958A (en) | Audio and video composite device and method and device for synchronizing audio and video thereof | |
| CN101137066B (en) | Multimedia data flow synchronous control method and device | |
| CN101540871B (en) | Method and terminal for synchronously recording sounds and images of opposite ends based on circuit domain video telephone | |
| US20210227005A1 (en) | Multi-user instant messaging method, system, apparatus, and electronic device | |
| CN109525873A (en) | A kind of audio plays synchronous method and device | |
| JP2015012557A (en) | Video audio processor, video audio processing system, video audio synchronization method, and program | |
| KR102024437B1 (en) | System and method for transmitting a plurality of video image | |
| CN110177293A (en) | A kind of synchronization multicast and video retransmission method, system | |
| CN107135407B (en) | Synchronous method and system in a kind of piano video teaching | |
| CN114979783A (en) | Audio and video playing method and device and electronic equipment | |
| CN108632557B (en) | Audio and video synchronization method and terminal | |
| WO2018155910A1 (en) | Method, for supporting relay broadcast, device and non-transitory computer-readable recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |