CN105898504A - Audio and video synchronization method and apparatus - Google Patents
Audio and video synchronization method and apparatus Download PDFInfo
- Publication number
- CN105898504A CN105898504A CN201610265580.6A CN201610265580A CN105898504A CN 105898504 A CN105898504 A CN 105898504A CN 201610265580 A CN201610265580 A CN 201610265580A CN 105898504 A CN105898504 A CN 105898504A
- Authority
- CN
- China
- Prior art keywords
- error
- video
- time
- time interval
- audio
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
Abstract
The application discloses an audio and video synchronization method and apparatus. When the scheme in the application is used for audio and video playing, a first actual time interval among various video frames in the playing process will be adjusted when the accumulative time errors of the video exceed a preset first error allowable threshold; and/or a second actual time interval among various audio frames in the audio playing process is adjusted when the accumulative time errors of the audio exceed a preset second error allowable threshold. By this method, the errors between a playing frame rate required by the video and the actual playing frame rate can be reduced so that the actual playing frame rate of the video approaches the playing frame rate required this time, and/or the errors between a sampling rate required by the audio and the actual sampling rate can be reduced so that the actual playing sampling rate of the audio approaches the sampling rate required this time; and therefore, the degree of synchronization of the audio and video in the playing process is increased.
Description
Technical field
The present embodiments relate to multimedia technology field, particularly relate to a kind of audio and video synchronization method and device.
Background technology
Audio-visual generally collectively formed by Voice & Video.Wherein, Voice & Video is play jointly, and user just can be made the most glad
Reward is to sound and picture.
In audio-visual playing process, need Voice & Video to keep synchronizing, sound and picture just can be made to keep unified.But,
Inventor finds in the research process of the application, and the precision of the timer by arranging in terminal device is limited, some feelings
Under condition, the play frame rate of the sample rate of audio frequency or video is unstable, causes occurring between Voice & Video asynchronous.Such as,
If the frame per second of video playback is that 30 frames are per second, say, that every 33.33 ... 3 ... millisecond plays a frame video, but,
The precision of timer can only precision to 1 millisecond, in this case, can only every 33 milliseconds play frame videos or every
Playing frame videos, thus error occur for 34 milliseconds, this error is accumulated with reproduction time, can cause Voice & Video it
Between nonsynchronous phenomenon further obvious, the viewing having a strong impact on user is experienced.
Summary of the invention
For overcoming problem present in correlation technique, the embodiment of the present invention provides a kind of audio and video synchronization method and device.
In order to solve above-mentioned technical problem, the embodiment of the invention discloses following technical scheme:
First aspect according to embodiments of the present invention, it is provided that a kind of audio and video synchronization method, including:
According to the play frame rate needed for this video, whether detection video display process exists time error, and/or,
According to the sample rate needed for this secondary audio program, whether detection audio frequency playing process exists time error；
If described video display process exists time error, calculate the accumulated time error of described video, and/or, if
There is time error in described audio frequency playing process, calculate the accumulated time error of described audio frequency；
When the accumulated time error of described video exceedes the first default error permission threshold value, adjust in playing process each
The first real time interval between frame video, and/or, when the accumulated time error of described audio frequency exceedes default second by mistake
When difference allows threshold value, adjust the second real time interval between each audio frame of playing process sound intermediate frequency.
Preferably, described according to the play frame rate needed for this video, whether detection video display process exists the time by mistake
Difference, including:
According to the play frame rate needed for this video, calculate the first required time interval between each frame video；
If timing accuracy can not meet the first required time interval between described each frame video, determine described video playback mistake
Journey exists time error；
And/or, described according to the sample rate needed for this secondary audio program, whether detection playing process frequently exists time error,
Including:
According to the sample rate needed for this secondary audio program, calculate the second required time interval between each audio frame；
If timing accuracy can not meet the second required time interval between described each audio frame, determine that described audio frequency was play
Journey exists time error.
Preferably, the accumulated time error of the described video of described calculating, including:
According to the play frame rate needed for this video, calculate the first required time interval between each frame video, and obtain institute
State in video display process, the first real time interval between each frame video；
Calculate the very first time between described first required time interval and the first real time interval poor, determine described first
Time difference is this very first time error, and this very first time error described in storage；
According to the very first time error stored before and this very first time error described, calculate and obtain the tired of described video
Long-pending time error.
Preferably, the accumulated time error of the described audio frequency of described calculating, including:
According to the sample rate needed for this secondary audio program, calculate the second required time interval between each audio frame, and obtain described
The second real time interval in audio frequency playing process, between each audio frame；
Calculate the second time difference between described second required time interval and the second real time interval, determine described second
Time difference is this second time error, and this second time error described in storage；
According to the second time error stored before and this second time error described, calculate and obtain the tired of described audio frequency
Long-pending time error.
Preferably, described when the accumulated time error of described video exceed default first error allow threshold value time, adjust and exist
In playing process, the first real time between each frame video is spaced, including:
If the first required time interval between each frame video is more than described first real time interval, heighten described first real
Border time interval；
If the first required time interval between each frame video is less than described first real time interval, turn down described first real
Border time interval；
And/or, described when the accumulated time error of described audio frequency exceed default second error allow threshold value time, adjust and exist
The second real time interval between each audio frame of playing process sound intermediate frequency, including:
If the second required time interval between each audio frame of audio frequency is more than described second real time interval, heighten described
Second real time interval；
If the second required time interval between each audio frame of audio frequency is less than described second real time interval, turn down described
Second real time interval.
Preferably, also include:
The first real time set between each frame video before not adjusting is spaced apart the first initial interval, is adjusting institute
After stating the first initial interval, continue to calculate the accumulated time error of video, and be less than in described accumulated time error
When described first error preset allows threshold value, the first real time interval is adjusted to described first initial interval；
And/or,
The second real time set between each audio frame before not adjusting is spaced apart the second initial interval, is adjusting institute
After stating the second initial interval, continue to calculate the accumulated time error of audio frequency, and be less than in described accumulated time error
When described second error preset allows threshold value, the second real time interval is adjusted to described second initial interval.
Second aspect according to embodiments of the present invention, it is provided that a kind of audio-visual synchronization device, including:
Whether error sensing module, for according to the play frame rate needed for this video, existing in detection video display process
Time error, and/or, according to the sample rate needed for this secondary audio program, whether detection audio frequency playing process exists time error；
, if there is time error, when calculating the accumulation of described video in described video display process in error calculating module
Between error, and/or, if described audio frequency playing process exists time error, calculate the accumulated time error of described audio frequency；
Time regulating module, is used for when the accumulated time error of described video exceedes the first default error permission threshold value,
Adjust the first real time interval between each frame video in playing process, and/or, when the accumulated time of described audio frequency is missed
When difference exceedes the second default error permission threshold value, adjust second between each audio frame of playing process sound intermediate frequency real
Border time interval.
Preferably, described error sensing module includes:
First computing unit, is used for according to the play frame rate needed for this video, needed for calculating first between each frame video
Time interval；
First determines unit, if the first required time interval between described each frame video can not be met for timing accuracy,
Determine in described video display process and there is time error；
And/or, described error sensing module includes:
Second computing unit, for according to the sample rate needed for this secondary audio program, second calculated between each audio frame is taken
Between be spaced；
Second determines unit, if the second required time interval between described each audio frame can not be met for timing accuracy,
Determine in described audio frequency playing process and there is time error.
Preferably, described error calculating module includes:
Very first time interval calculation unit, for according to the play frame rate needed for this video, calculates between each frame video
First required time interval, and obtain in described video display process, the first real time interval between each frame video；
Very first time difference computing unit, for calculating between described first required time interval and the first real time interval
The very first time is poor, determines that difference of the described very first time is this very first time error, and this very first time described in storage
Error；
First accumulated time error calculation unit, for according to the very first time error that stores before and described this first
Time error, calculates the accumulated time error obtaining described video.
Preferably, described error calculating module includes:
Second time interval computing unit, for according to the sample rate needed for this secondary audio program, calculates the between each audio frame
Two required time intervals, and obtain in described audio frequency playing process, the second real time interval between each audio frame；
Second time difference calculating unit, for calculating between described second required time interval and the second real time interval
Second time difference, determine that described second time difference is this second time error, and this second time described in storage
Error；
Second accumulated time error calculation unit, for according to the second time error of storing before and described this second
Time error, calculates the accumulated time error obtaining described audio frequency.
Preferably, described time regulating module includes:
First adjustment unit, if the first required time interval between each frame video is more than between described first real time
Every, heighten described first real time interval, and if the first required time interval between each frame video is less than described the
One real time interval, turns down described first real time interval；
And/or, the second adjustment unit, if the second required time interval between each audio frame of audio frequency is more than described
Second real time interval, heightens described second real time interval, and if the second institute between each audio frame of audio frequency
Take time and be spaced less than described second real time interval, turn down described second real time interval.
Preferably, it is characterised in that
The first real time set between each frame video before not adjusting is spaced apart the first initial interval, described error
Computing module is additionally operable to after adjusting described first initial interval, continues to calculate the accumulated time error of video；
Described time regulating module is additionally operable to allow threshold in described accumulated time error less than described first error preset
During value, the first real time interval is adjusted to described first initial interval；
And/or,
The second real time set between each audio frame before not adjusting is spaced apart the second initial interval, described error
Computing module is additionally operable to after adjusting described second initial interval, continues to calculate the accumulated time error of audio frequency；
Described time regulating module is additionally operable to allow threshold in described accumulated time error less than described second error preset
During value, the second real time interval is adjusted to described second initial interval.
The technical scheme that embodiments of the invention provide can include following beneficial effect:
During by the scheme playing audio-video of the application, allow when the accumulated time error of video exceedes the first default error
During threshold value, the first real time interval between each frame video in playing process can be adjusted；And/or, when the accumulation of audio frequency
When time error exceedes the second default error permission threshold value, adjust between each audio frame of playing process sound intermediate frequency
Second real time interval.With it, can reduce between play frame rate and the play frame rate of reality that video needs
Error, the play frame rate making video actual levels off to this play frame rate needed, and/or reduces the adopting of needs of audio frequency
Error between sample rate and actual sample rate, make the sample rate of audio frequency actual play level off to this need sample rate, from
And improve audio frequency and video synchronization extent in playing process.
It should be appreciated that it is only exemplary and explanatory that above general description and details hereinafter describe, can not
Limit the present invention.
Accompanying drawing explanation
Accompanying drawing herein is merged in specification and constitutes the part of this specification, it is shown that meet the enforcement of the present invention
Example, and for explaining the principle of the present invention together with specification.
Fig. 1 is the workflow schematic diagram according to a kind of audio and video synchronization method shown in the present invention one exemplary embodiment；
Fig. 2 is according in a kind of audio and video synchronization method shown in the present invention one exemplary embodiment, calculates the accumulation of video
The workflow schematic diagram of time error；
Fig. 3 is according in a kind of audio and video synchronization method shown in the present invention one exemplary embodiment, calculates the accumulation of audio frequency
The workflow schematic diagram of time error；
Fig. 4 is the structural representation according to a kind of audio-visual synchronization device shown in the present invention one exemplary embodiment.
Detailed description of the invention
Here will illustrate exemplary embodiment in detail, its example represents in the accompanying drawings.Explained below relates to attached
During figure, unless otherwise indicated, the same numbers in different accompanying drawings represents same or analogous key element.Following exemplary is implemented
Embodiment described in example does not represent all embodiments consistent with the present invention.On the contrary, they be only with such as
The example of the apparatus and method that some aspects that described in detail in appended claims, the present invention are consistent.
Disclosure one audio and video synchronization method and device, during to solve to play audio-visual in prior art, audio frequency and video are not
The problem synchronized.
The open a kind of audio and video synchronization method of embodiments herein one.Workflow schematic diagram shown in Figure 1, institute
State audio and video synchronization method to comprise the following steps:
Step S11, according to the play frame rate needed for this video, whether detection video display process exists time error,
And/or, according to the sample rate needed for this secondary audio program, whether detection audio frequency playing process exists time error.
If the described video display process of step S12 exists time error, calculate the accumulated time error of described video,
And/or, if described audio frequency playing process exists time error, calculate the accumulated time error of described audio frequency.
Such as, if in video display process, required play frame rate is 30 frames/second, then need every 33.33 ... 3 ... in the least
Second plays a frame video, and the timing accuracy of timer may be only accurate to 1 millisecond, then can only be every 33 milliseconds or 34 millis
Second broadcasting one frame video, thus time of occurrence error.
Step S13, when the accumulated time error of described video exceed default first error allow threshold value time, adjust and broadcasting
During putting, the first real time between each frame video is spaced, and/or, when the accumulated time error of described audio frequency exceedes pre-
If the second error when allowing threshold value, adjust between the second real time between each audio frame of playing process sound intermediate frequency
Every.
In audio and video synchronization method disclosed in the present application, the first error can be preset and allow threshold value and the second error to allow
Threshold value.Further, described first error allow threshold value and the second error allow threshold value generally by staff according to the demand of collection
Arrange.It addition, described first error allows threshold value and the second error to allow threshold to be usually not less than timing accuracy.Such as, if
Timing accuracy is 1 millisecond, can set described first error and allow threshold value and the second error to allow threshold value to be 1 millisecond, when
So, it is also possible to be set to other times value, this is not limited by the application.
Step S11 of the application is to the open a kind of audio and video synchronization method of step S13, in the method, when the accumulation of video
When time error exceedes the first default error permission threshold value, the first reality between each frame video in playing process can be adjusted
Border time interval；And/or, when the accumulated time error of audio frequency exceedes the second default error permission threshold value, adjust and broadcasting
Let slip the second real time interval between each audio frame of journey sound intermediate frequency.With it, video needs can be reduced
Play frame rate and the play frame rate of reality between error, what the play frame rate making video actual leveled off to that this needs broadcasts
Put the error between frame per second, and/or sample rate and the actual sample rate of the needs reducing audio frequency, make adopting of audio frequency actual play
Sample rate levels off to this sample rate needed, thus improves audio frequency and video synchronization extent in playing process.
Further, described according to the play frame rate needed for this video, whether detection video display process exists the time
Error, including:
First, according to the play frame rate needed for this video, the first required time interval between each frame video is calculated；
Then, if timing accuracy can not meet the first required time interval between described each frame video, described video is determined
Playing process exists time error.
Wherein, if described first required time is spaced apart the integral multiple of described timing accuracy, then explanation timing accuracy can meet
The first required time interval between described each frame video, determines and there is not time error in described video display process；If
Described first required time interval is not the integral multiple of described timing accuracy, then explanation timing accuracy can not meet described each frame
The first required time interval between video, determines in described video display process and there is time error.
Such as, if the play frame rate needed for this video is 25 seconds, the most described first required time is spaced apart 40 milliseconds,
The timing accuracy of timer is 1 millisecond, then the first required time is spaced apart 40 times of described timing accuracy, timing accuracy energy
Meet the first required time interval between described each frame video, determine and described video display process does not exist time error；
If the play frame rate needed for this video is 30 frames/second, the most described first required time is spaced apart 33.33 ... 3 ... millisecond,
The timing accuracy of timer is 1 millisecond, then the first required time is spaced the integral multiple of the most described timing accuracy, then illustrate
Timing accuracy can not meet between described each frame video first required time interval, in described video display process in the presence of
Between error.
It addition, described according to the sample rate needed for this secondary audio program, whether detection frequency playing process exists time error, bag
Include:
First, according to the sample rate needed for this secondary audio program, the second required time interval between each audio frame is calculated；
Then, if timing accuracy can not meet the second required time interval between described each audio frame, described audio frequency is determined
Playing process exists time error.
Accordingly, if described second required time is spaced apart the integral multiple of described timing accuracy, then explanation timing accuracy can be full
The second required time interval between the described each audio frame of foot, determines in described audio frequency playing process and there is not time error；
If described second required time interval is not the integral multiple of described timing accuracy, then explanation timing accuracy can not meet described respectively
The second required time interval between audio frame, determines in described audio frequency playing process and there is time error.
Further, workflow schematic diagram shown in Figure 2, in the application, the accumulation of the described video of described calculating
Time error, comprises the following steps:
Step S21, according to the play frame rate needed for this video, calculate the first required time interval between each frame video,
And obtain in described video display process, the first real time interval between each frame video.
Step S22, calculate described first required time interval with the first real time be spaced between the very first time poor, really
Fixed difference of the described very first time is this very first time error, and this very first time error described in storage.
Step S23, according to the very first time error that stores before and this very first time error described, calculate and obtain institute
State the accumulated time error of video.
Such as, if the play frame rate needed for this video is 30 frames/second, the most described first required time is spaced apart 33.33 ...
3 ... millisecond.In this case, if the precision of timer is 1 millisecond, then in video display process, between each frame video
First real time was spaced apart 33 milliseconds or 34 milliseconds.
In this case, if the first real time was spaced apart 33 milliseconds, described first required time interval and the first reality
Very first time difference between time interval is 0.33 ... 3 ... millisecond, say, that this very first time error is 0.33 ...
3 ... millisecond.
Further, generally before storage very first time error and and this very first time error, calculate the accumulation of video
Error, therefore, if the first real time was spaced apart 33 milliseconds, the first required time is spaced apart 33.33 ... 3 ... millisecond,
After then playing the first frame video, the accumulated time error of video is 0.33 ... 3 ... millisecond, after playing the second frame video, and institute
The accumulated time error stating video is 0.66 ... 6 ... millisecond.In this case, after the 4th time gathers video image, institute
State the accumulated time error of video more than 1 millisecond, if described first error allows threshold value to be 1 millisecond, then be currently needed for adjusting
Whole first real time interval.
Accordingly, workflow schematic diagram shown in Figure 3, the accumulated time of the described audio frequency of calculating described herein
Error, comprises the following steps:
Step S31, according to the sample rate needed for this secondary audio program, calculate the second required time interval between each audio frame,
And obtain in described audio frequency playing process, the second real time interval between each audio frame；
Step S32, the second time difference calculated between described second required time interval and the second real time interval, really
Fixed described second time difference is this second time error, and this second time error described in storage；
Step S33, according to the second time error of storing before and this second time error described, calculate and obtain institute
State the accumulated time error of audio frequency.
It is to say, in audio frequency playing process, after each sampling operation, this second time error can be stored, and
Second time error of storage and this sum of the second time error described before calculating, the will stored before described
Two time errors and this sum of the second time error described, as the accumulated time error of described audio frequency.
It addition, in step S13 of the application, disclose adjustment first in playing process between each frame video actual
Time interval, and/or the operation at the second real time interval that adjustment is between each audio frame of playing process sound intermediate frequency.
Wherein, described when the accumulated time error of described video exceed default first error allow threshold value time, adjust and broadcasting
During putting, the first real time between each frame video is spaced, including:
If the first required time interval between each frame video is more than described first real time interval, heighten described first real
Border time interval；If the first required time interval between each frame video is less than described first real time interval, turn down institute
State the first real time interval.
Wherein, the adjustment amplitude at very first time interval is generally equal to the precision of timer.Such as, if between the first required time
Be divided into 33.33 ... 3 ... millisecond, and the first real time was spaced apart 33 milliseconds, then the first required time interval is more than described
First real time interval, needs to heighten described first real time interval, and timing accuracy is 1 millisecond, i.e. adjusts width
Degree is 1 millisecond, then generally described first real time interval is adjusted to 34 milliseconds.
If it addition, the first required time is spaced apart 33.33 ... 3 ... millisecond, and the first real time was spaced apart 34 milliseconds,
Then the first required time interval is less than described first real time interval, needs to turn down described first real time interval, and
Timing accuracy is 1 millisecond, and the amplitude that i.e. adjusts is 1 millisecond, then generally described first real time interval is adjusted to 33
Millisecond.
Accordingly, described when the accumulated time error of described audio frequency exceed default second error allow threshold value time, adjust and exist
The second real time interval between each audio frame of playing process sound intermediate frequency, including:
If the second required time interval between each audio frame of audio frequency is more than described second real time interval, heighten described
Second real time interval；If the second required time interval between each audio frame of audio frequency is less than described second real time
Interval, turns down described second real time interval.Wherein, the adjustment amplitude at the second real time interval, the most fixed
Time device timing accuracy.
Further, audio and video synchronization method disclosed in the present application also includes:
The first real time set between each frame video before not adjusting is spaced apart the first initial interval, is adjusting institute
After stating the first initial interval, continue to calculate the accumulated time error of video, and be less than in described accumulated time error
When described first error preset allows threshold value, the first real time interval is adjusted to described first initial interval；
And/or, the second real time set between each audio frame before not adjusting is spaced apart the second initial interval,
After adjusting described second initial interval, continue to calculate the accumulated time error of audio frequency, and in described accumulated time error
When allowing threshold value less than described second error preset, the second real time interval is adjusted to described second initial time
Interval.
In this application, before the accumulated time error of video allows threshold value not less than the first default error, by each frame
Very first time error after video playback is added, the sum of the very first time error after the described each frame video playback that will get
Accumulated time error as video.And when the accumulated time error determining video allows threshold value more than the first error, adjust
Whole first real time interval.
It addition, the accumulated time error at described video exceedes described first error permission threshold value, and it is real to adjust described first
After the time interval of border, continue to calculate the accumulated time error of video.In accumulated time error less than described first error
When allowing threshold value, then the first real time interval is adjusted back the time interval that this video is initial.
Wherein, after adjusting described first initial interval, when continuing the accumulated time error calculating video, if according to
After a frame video is play at the first real time interval after adjustment, the accumulated time error of the video generally equal to last time calculates
The accumulated time error of the video obtained deducts the difference of the adjustment amplitude at the first real time interval, if this difference is the biggest
Allow threshold value in described first error, then continue to play video, until calculating according to the first real time interval after adjusting
Determine that this difference allows threshold value less than described first error, then the first real time interval is adjusted to described first initial time
Between be spaced.
Such as, the first real time was spaced apart 33 milliseconds, and the first required time is spaced apart 33.33 ... 3 ... millisecond, then
After playing the first frame video, the accumulated time error of video is 0.33 ... 3 ... millisecond, after playing the 4th frame video,
The accumulated time error of video is 1.33 ... 3 ... millisecond, more than 1 millisecond.If the first error preset allows threshold value to be 1
Millisecond, then, after playing the 4th frame video, the accumulated time error of video exceedes the first default error and allows threshold value, needs
Adjust the first current real time interval.The first actual time if the adjustment amplitude of video is 1 millisecond, then after adjusting
Between be spaced apart 34 milliseconds, playing after the 5th frame video, the accumulated time error of video is (1.33 ... 3 ...-1) millisecond,
It is 0.33 ... 3 ... millisecond, say, that the accumulated time error of video allows threshold value less than the error preset.This
In the case of, after playing the 5th frame video, very first time time interval is adjusted to 33 milliseconds,
It is to say, in scheme disclosed in the present application, can realize in playing process the according to the accumulated time error of video
The adjustment at one real time interval, makes video in playing process, and actual play frame rate levels off to required play frame rate,
Reduce the error of play frame rate, improve the synchronization extent of audio frequency and video.
Accordingly, in the application, before the accumulated time error of audio frequency allows threshold value not less than the second default error,
Corresponding for each audio frame the second time error is added, by the second corresponding for described each audio frame of getting time error
With the accumulated time error as audio frequency, and when determining that the accumulated time error of audio frequency allows threshold value more than the second error,
Adjust the second real time interval.
It addition, the accumulated time error at described audio frequency exceedes described second error permission threshold value, and it is real to adjust described second
After the time interval of border, continue to calculate the accumulated time error of audio frequency.In accumulated time error less than described second error
When allowing threshold value, then the second real time interval is adjusted back the time interval that this audio frequency is initial.
Wherein, after adjusting described second initial interval, when continuing the accumulated time error calculating audio frequency, if according to
After the second real time interval after adjustment carries out playing for the first time, the accumulated time error of audio frequency generally equal to last time meter
The accumulated error of the audio frequency obtained deducts the difference of adjustment amplitude at the second real time interval, if this difference still above
Described second error allows threshold value, then continue to play out, until calculating really according to the second real time interval after adjusting
This difference fixed allows threshold value less than described second error, then adjusts the second real time interval to described second initial time
Interval.
It is to say, in scheme disclosed in the present application, can realize in playing process the according to the accumulated time error of audio frequency
The adjustment at two real times intervals, makes audio frequency in playing process, plays sample rate and level off to required sample rate, reduce
Required sample rate and the inconsistent error caused of actual sample rate, improve the synchronization extent of audio frequency and video.
It addition, during actual acquisition, accuracy value when calculating the accumulated time error of Voice & Video can be according to reality
Demand is arranged, such as, if required precision is higher, after the accumulated time error of described Voice & Video can be accurate to decimal point
Five or higher precision, if required precision is relatively low, the accumulated time error of described Voice & Video can be accurate to decimal point
The precision of latter four or lower, this is not limited by the application.
Accordingly, the open a kind of audio-visual synchronization device of the second embodiment of the application, structural representation shown in Figure 4
Figure, described audio-visual synchronization device includes: error sensing module 100, error calculating module 200 and time regulating module
300。
Wherein, described error sensing module 100, for according to the play frame rate needed for this video, detects video playback
During whether there is time error, and/or, according to the sample rate needed for this secondary audio program, detection audio frequency playing process is
No there is time error；
Described error calculating module 200, if there is time error in described video display process, calculates described video
Accumulated time error, and/or, if there is time error in described audio frequency playing process, when calculating the accumulation of described audio frequency
Between error；
Described time regulating module 300, for allowing when the accumulated time error of described video exceedes the first default error
During threshold value, adjust the first real time interval between each frame video in playing process, and/or, when tiring out of described audio frequency
When long-pending time error exceedes the second default error permission threshold value, adjust between each audio frame of playing process sound intermediate frequency
Second the real time interval.
Disclosure one audio-visual synchronization device, by device disclosed in the present application, when the accumulated time error of video
Exceed default first error when allowing threshold value, can adjust in playing process between the first real time between each frame video
Every；And/or, when the accumulated time error of audio frequency exceedes the second default error permission threshold value, adjust in playing process
The second real time interval between each audio frame of audio frequency.By this device, it is possible to reduce the broadcasting frame that video needs
Error between rate and the play frame rate of reality, the play frame rate making video actual levels off to this play frame rate needed,
And/or the error between sample rate and the actual sample rate of the needs reducing audio frequency, make the sample rate convergence of audio frequency actual play
In this sample rate needed, thus improve audio frequency and video synchronization extent in playing process.
Further, in scheme disclosed in the present application, described error sensing module 100 includes:
First computing unit, is used for according to the play frame rate needed for this video, needed for calculating first between each frame video
Time interval；
First determines unit, if the first required time interval between described each frame video can not be met for timing accuracy,
Determine in described video display process and there is time error；
And/or, described error sensing module includes:
Second computing unit, for according to the sample rate needed for this secondary audio program, second calculated between each audio frame is taken
Between be spaced；
Second determines unit, if the second required time interval between described each audio frame can not be met for timing accuracy,
Determine in described audio frequency playing process and there is time error.
Wherein, if described first required time is spaced apart the integral multiple of described timing accuracy, then explanation timing accuracy can meet
The first required time interval between described each frame video, determines and there is not time error in described video display process；If
Described first required time interval is not the integral multiple of described timing accuracy, then explanation timing accuracy can not meet described each frame
The first required time interval between video, determines in described video display process and there is time error.
Such as, if the play frame rate needed for this video is 25 seconds, the most described first required time is spaced apart 40 milliseconds,
The timing accuracy of timer is 1 millisecond, then the first required time is spaced apart 40 times of described timing accuracy, timing accuracy energy
Meet the first required time interval between described each frame video, determine and described video display process does not exist time error；
If the play frame rate needed for this video is 30 frames/second, the most described first required time is spaced apart 33.33 ... 3 ... millisecond,
The timing accuracy of timer is 1 millisecond, then the first required time is spaced the integral multiple of the most described timing accuracy, then illustrate
Timing accuracy can not meet between described each frame video first required time interval, in described video display process in the presence of
Between error.
Accordingly, if described second required time is spaced apart the integral multiple of described timing accuracy, then explanation timing accuracy can be full
The second required time interval between the described each audio frame of foot, determines in described audio frequency playing process and there is not time error；
If described second required time interval is not the integral multiple of described timing accuracy, then explanation timing accuracy can not meet described respectively
The second required time interval between audio frame, determines in described audio frequency playing process and there is time error.
In audio-visual synchronization device disclosed in the present application, described error calculating module 200 includes:
Very first time interval calculation unit, for according to the play frame rate needed for this video, calculates between each frame video
First required time interval, and obtain in described video display process, the first real time interval between each frame video；
Very first time difference computing unit, for calculating between described first required time interval and the first real time interval
The very first time is poor, determines that difference of the described very first time is this very first time error, and this very first time described in storage
Error；
First accumulated time error calculation unit, for according to the very first time error that stores before and described this first
Time error, calculates the accumulated time error obtaining described video.
Further, in audio-visual synchronization device disclosed in the present application, described error calculating module includes:
Second time interval computing unit, for according to the sample rate needed for this secondary audio program, calculates the between each audio frame
Two required time intervals, and obtain in described audio frequency playing process, the second real time interval between each audio frame；
Second time difference calculating unit, for calculating between described second required time interval and the second real time interval
Second time difference, determine that described second time difference is this second time error, and this second time described in storage
Error；
Second accumulated time error calculation unit, for according to the second time error of storing before and described this second
Time error, calculates the accumulated time error obtaining described audio frequency.
Further, described time regulating module includes:
First adjustment unit, if the first required time interval between each frame video is more than between described first real time
Every, heighten described first real time interval, and if the first required time interval between each frame video is less than described the
One real time interval, turns down described first real time interval；
And/or, the second adjustment unit, if the second required time interval between each audio frame of audio frequency is more than described
Second real time interval, heightens described second real time interval, and if the second institute between each audio frame of audio frequency
Take time and be spaced less than described second real time interval, turn down described second real time interval.
Wherein, the adjustment amplitude of very first time interval and the second time interval is generally equal to the precision of timer.
Such as, if the first required time is spaced apart 33.33 ... 3 ... millisecond, and the first real time was spaced apart 33 milliseconds,
Then the first required time interval is more than described first real time interval, needs to heighten described first real time interval, and
Timing accuracy is 1 millisecond, and the amplitude that i.e. adjusts is 1 millisecond, then generally described first real time interval is adjusted to 34
Millisecond.
If it addition, the first required time is spaced apart 33.33 ... 3 ... millisecond, and the first real time was spaced apart 34 milliseconds,
Then the first required time interval is less than described first real time interval, needs to turn down described first real time interval, and
Timing accuracy is 1 millisecond, and the amplitude that i.e. adjusts is 1 millisecond, then generally described first real time interval is adjusted to 33
Millisecond.
Further, the first real time set between each frame video before not adjusting is spaced apart between the first initial time
Every, described error calculating module is additionally operable to after adjusting described first initial interval, when continuing the accumulation calculating video
Between error；
Described time regulating module is additionally operable to allow threshold in described accumulated time error less than described first error preset
During value, the first real time interval is adjusted to described first initial interval；
And/or, the second real time set between each audio frame before not adjusting is spaced apart the second initial interval,
Described error calculating module is additionally operable to after adjusting described second initial interval, continues to calculate the accumulated time of audio frequency by mistake
Difference；
Described time regulating module is additionally operable to allow threshold in described accumulated time error less than described second error preset
During value, the second real time interval is adjusted to described second initial interval.
In scheme disclosed in the present application, can realize the first real time in playing process according to the accumulated time error of video
The adjustment at interval, makes video in playing process, and actual play frame rate levels off to required play frame rate, reduces and plays
The error of frame per second, improves the synchronization extent of audio frequency and video.
Further, in scheme disclosed in the present application, can realize in playing process second real according to the accumulated time error of audio frequency
The adjustment of border time interval, makes audio frequency in playing process, plays sample rate and level off to required sample rate, reduce required
Sample rate and the inconsistent error caused of actual sample rate, improve audio frequency and video synchronization extent.
It addition, in scheme disclosed in the present application, accuracy value when calculating the accumulated time error of Voice & Video can basis
Actual demand is arranged, such as, if required precision is higher, the accumulated time error of described Voice & Video can be accurate to decimal
Five or higher precision after Dian, if required precision is relatively low, the accumulated time error of described Voice & Video can be accurate to little
The precision of four or lower after several points, this is not limited by the application.
About the device in above-described embodiment, wherein modules performs the concrete mode of operation in relevant the method
Embodiment is described in detail, explanation will be not set forth in detail herein.
Those skilled in the art, after considering specification and putting into practice invention disclosed, will readily occur to other enforcement of the present invention
Scheme.The application is intended to any modification, purposes or the adaptations of the present invention, these modification, purposes or
Adaptations follow the present invention general principle and include the undocumented common knowledge in the art of the present invention or
Conventional techniques means.Description and embodiments is considered only as exemplary, and true scope and spirit of the invention are by following
Claim is pointed out.
It should be appreciated that the invention is not limited in precision architecture described above and illustrated in the accompanying drawings, and
Various modifications and changes can carried out without departing from the scope.The scope of the present invention is only limited by appended claim.
Claims (12)
1. an audio and video synchronization method, it is characterised in that including:
According to the play frame rate needed for this video, whether detection video display process exists time error, and/or, root
According to the sample rate needed for this secondary audio program, whether detection audio frequency playing process exists time error；
If described video display process exists time error, calculate the accumulated time error of described video, and/or, if institute
State in audio frequency playing process and there is time error, calculate the accumulated time error of described audio frequency；
When the accumulated time error of described video exceedes the first default error permission threshold value, adjust in playing process each
The first real time interval between frame video, and/or, when the accumulated time error of described audio frequency exceedes default second by mistake
When difference allows threshold value, adjust the second real time interval between each audio frame of playing process sound intermediate frequency.
Audio and video synchronization method the most according to claim 1, it is characterised in that described according to this video needed for
Whether play frame rate, exist time error in detection video display process, including:
According to the play frame rate needed for this video, calculate the first required time interval between each frame video；
If timing accuracy can not meet the first required time interval between described each frame video, determine described video playback mistake
Journey exists time error；
And/or, described according to the sample rate needed for this secondary audio program, whether detection frequency playing process exists time error, bag
Include:
According to the sample rate needed for this secondary audio program, calculate the second required time interval between each audio frame；
If timing accuracy can not meet the second required time interval between described each audio frame, determine that described audio frequency was play
Journey exists time error.
Audio and video synchronization method the most according to claim 1, it is characterised in that the accumulation of the described video of described calculating
Time error, including:
According to the play frame rate needed for this video, calculate the first required time interval between each frame video, and obtain institute
State in video display process, the first real time interval between each frame video；
Calculate the very first time between described first required time interval and the first real time interval poor, determine described first
Time difference is this very first time error, and this very first time error described in storage；
According to the very first time error stored before and this very first time error described, calculate and obtain the tired of described video
Long-pending time error.
Audio and video synchronization method the most according to claim 1, it is characterised in that the accumulation of the described audio frequency of described calculating
Time error, including:
According to the sample rate needed for this secondary audio program, calculate the second required time interval between each audio frame, and obtain described
The second real time interval in audio frequency playing process, between each audio frame；
Calculate the second time difference between described second required time interval and the second real time interval, determine described second
Time difference is this second time error, and this second time error described in storage；
According to the second time error stored before and this second time error described, calculate and obtain the tired of described audio frequency
Long-pending time error.
Audio and video synchronization method the most according to claim 1, it is characterised in that described when the accumulation of described video
Between error exceed default first error when allowing threshold value, adjust the first actual time between each frame video in playing process
Between be spaced, including:
If the first required time interval between each frame video is more than described first real time interval, heighten described first real
Border time interval；
If the first required time interval between each frame video is less than described first real time interval, turn down described first real
Border time interval；
And/or, described when the accumulated time error of described audio frequency exceed default second error allow threshold value time, adjust and broadcasting
Let slip the second real time interval between each audio frame of journey sound intermediate frequency, including:
If the second required time interval between each audio frame of audio frequency is more than described second real time interval, heighten described
Second real time interval；
If the second required time interval between each audio frame of audio frequency is less than described second real time interval, turn down described
Second real time interval.
Audio and video synchronization method the most according to claim 1, it is characterised in that also include:
The first real time set between each frame video before not adjusting is spaced apart the first initial interval, is adjusting institute
After stating the first initial interval, continue to calculate the accumulated time error of video, and be less than in described accumulated time error
When described first error preset allows threshold value, the first real time interval is adjusted to described first initial interval；
And/or,
The second real time set between each audio frame before not adjusting is spaced apart the second initial interval, is adjusting institute
After stating the second initial interval, continue to calculate the accumulated time error of audio frequency, and be less than in described accumulated time error
When described second error preset allows threshold value, the second real time interval is adjusted to described second initial interval.
7. an audio-visual synchronization device, it is characterised in that including:
Whether error sensing module, for according to the play frame rate needed for this video, existing in detection video display process
Time error, and/or, according to the sample rate needed for this secondary audio program, whether detection audio frequency playing process exists time error；
, if there is time error, when calculating the accumulation of described video in described video display process in error calculating module
Between error, and/or, if described audio frequency playing process exists time error, calculate the accumulated time error of described audio frequency；
Time regulating module, is used for when the accumulated time error of described video exceedes the first default error permission threshold value,
Adjust the first real time interval between each frame video in playing process, and/or, when the accumulated time of described audio frequency is missed
When difference exceedes the second default error permission threshold value, adjust second between each audio frame of playing process sound intermediate frequency real
Border time interval.
Audio-visual synchronization device the most according to claim 7, it is characterised in that described error sensing module includes:
First computing unit, is used for according to the play frame rate needed for this video, needed for calculating first between each frame video
Time interval；
First determines unit, if the first required time interval between described each frame video can not be met for timing accuracy,
Determine in described video display process and there is time error；
And/or, described error sensing module includes:
Second computing unit, for according to the sample rate needed for this secondary audio program, second calculated between each audio frame is taken
Between be spaced；
Second determines unit, if the second required time interval between described each audio frame can not be met for timing accuracy,
Determine in described audio frequency playing process and there is time error.
Audio-visual synchronization device the most according to claim 7, it is characterised in that described error calculating module includes:
Very first time interval calculation unit, for according to the play frame rate needed for this video, calculates between each frame video
First required time interval, and obtain in described video display process, the first real time interval between each frame video；
Very first time difference computing unit, for calculating between described first required time interval and the first real time interval
The very first time is poor, determines that difference of the described very first time is this very first time error, and this very first time described in storage
Error；
First accumulated time error calculation unit, for according to the very first time error that stores before and described this first
Time error, calculates the accumulated time error obtaining described video.
Audio-visual synchronization device the most according to claim 7, it is characterised in that described error calculating module includes:
Second time interval computing unit, for according to the sample rate needed for this secondary audio program, calculates the between each audio frame
Two required time intervals, and obtain in described audio frequency playing process, the second real time interval between each audio frame；
Second time difference calculating unit, for calculating between described second required time interval and the second real time interval
Second time difference, determine that described second time difference is this second time error, and this second time described in storage
Error；
Second accumulated time error calculation unit, for according to the second time error of storing before and described this second
Time error, calculates the accumulated time error obtaining described audio frequency.
11. audio-visual synchronization devices according to claim 7, it is characterised in that described time regulating module includes:
First adjustment unit, if the first required time interval between each frame video is more than between described first real time
Every, heighten described first real time interval, and if the first required time interval between each frame video is less than described the
One real time interval, turns down described first real time interval；
And/or, the second adjustment unit, if the second required time interval between each audio frame of audio frequency is more than described the
Two the real times interval, heighten described second the real time interval, and if between each audio frame of audio frequency second needed for
Time interval, less than described second real time interval, turns down described second real time interval.
12. audio-visual synchronization devices according to claim 7, it is characterised in that
The first real time set between each frame video before not adjusting is spaced apart the first initial interval, described error
Computing module is additionally operable to after adjusting described first initial interval, continues to calculate the accumulated time error of video；
Described time regulating module is additionally operable to allow threshold value in described accumulated time error less than described first error preset
Time, the first real time interval is adjusted to described first initial interval；
And/or,
The second real time set between each audio frame before not adjusting is spaced apart the second initial interval, described error
Computing module is additionally operable to after adjusting described second initial interval, continues to calculate the accumulated time error of audio frequency；
Described time regulating module is additionally operable to allow threshold value in described accumulated time error less than described second error preset
Time, the second real time interval is adjusted to described second initial interval.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610265580.6A CN105898504A (en) | 2016-04-26 | 2016-04-26 | Audio and video synchronization method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610265580.6A CN105898504A (en) | 2016-04-26 | 2016-04-26 | Audio and video synchronization method and apparatus |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105898504A true CN105898504A (en) | 2016-08-24 |
Family
ID=56704648
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610265580.6A Pending CN105898504A (en) | 2016-04-26 | 2016-04-26 | Audio and video synchronization method and apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105898504A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107959874A (en) * | 2017-12-12 | 2018-04-24 | 青岛海信电器股份有限公司 | Sound draws synchronous auto-correction method and device |
| CN111464256A (en) * | 2020-04-14 | 2020-07-28 | 北京百度网讯科技有限公司 | Time stamp correction method and device, electronic equipment and storage medium |
| CN111556467A (en) * | 2020-03-31 | 2020-08-18 | 惠州市德赛西威汽车电子股份有限公司 | Mobile phone interconnection audio playing processing method |
| CN112887772A (en) * | 2017-03-14 | 2021-06-01 | 上海兆芯集成电路有限公司 | Audio synchronization method for video streaming |
| CN113382300A (en) * | 2021-06-08 | 2021-09-10 | 三星电子（中国）研发中心 | Audio and video playing method and device |
| CN114827696A (en) * | 2021-01-29 | 2022-07-29 | 华为技术有限公司 | Method for synchronously playing cross-device audio and video data and electronic device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104618737A (en) * | 2013-11-01 | 2015-05-13 | 深圳中兴力维技术有限公司 | Streaming media system clock slow synchronization method and streaming media system clock slow synchronization device |
| JP2015099346A (en) * | 2013-11-19 | 2015-05-28 | 韓國電子通信研究院Ｅｌｅｃｔｒｏｎｉｃｓ ａｎｄ Ｔｅｌｅｃｏｍｍｕｎｉｃａｔｉｏｎｓ Ｒｅｓｅａｒｃｈ Ｉｎｓｔｉｔｕｔｅ | Multi-screen display system, and image signal correcting method for the same |
-
2016
- 2016-04-26 CN CN201610265580.6A patent/CN105898504A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104618737A (en) * | 2013-11-01 | 2015-05-13 | 深圳中兴力维技术有限公司 | Streaming media system clock slow synchronization method and streaming media system clock slow synchronization device |
| JP2015099346A (en) * | 2013-11-19 | 2015-05-28 | 韓國電子通信研究院Ｅｌｅｃｔｒｏｎｉｃｓ ａｎｄ Ｔｅｌｅｃｏｍｍｕｎｉｃａｔｉｏｎｓ Ｒｅｓｅａｒｃｈ Ｉｎｓｔｉｔｕｔｅ | Multi-screen display system, and image signal correcting method for the same |
Non-Patent Citations (1)
| Title |
|---|
| 梅永: "中国学术期刊(光盘版)电子杂志社", 《中国优秀博硕士学位论文全文数据库(硕士) 工程科技II辑》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112887772A (en) * | 2017-03-14 | 2021-06-01 | 上海兆芯集成电路有限公司 | Audio synchronization method for video streaming |
| CN107959874A (en) * | 2017-12-12 | 2018-04-24 | 青岛海信电器股份有限公司 | Sound draws synchronous auto-correction method and device |
| CN107959874B (en) * | 2017-12-12 | 2020-08-25 | 海信视像科技股份有限公司 | Method and device for automatically correcting sound and picture synchronization |
| CN111556467A (en) * | 2020-03-31 | 2020-08-18 | 惠州市德赛西威汽车电子股份有限公司 | Mobile phone interconnection audio playing processing method |
| CN111556467B (en) * | 2020-03-31 | 2023-08-08 | 惠州市德赛西威汽车电子股份有限公司 | Mobile phone interconnection audio playing processing method |
| CN111464256A (en) * | 2020-04-14 | 2020-07-28 | 北京百度网讯科技有限公司 | Time stamp correction method and device, electronic equipment and storage medium |
| CN114827696A (en) * | 2021-01-29 | 2022-07-29 | 华为技术有限公司 | Method for synchronously playing cross-device audio and video data and electronic device |
| CN113382300A (en) * | 2021-06-08 | 2021-09-10 | 三星电子（中国）研发中心 | Audio and video playing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105898504A (en) | Audio and video synchronization method and apparatus | |
| CN105142021B (en) | Display control program, display control method and display device | |
| CN101808202B (en) | Method, system and computer for realizing sound-and-caption synchronization in video file | |
| US8223269B2 (en) | Closed caption production device, method and program for synthesizing video, sound and text | |
| US8111327B2 (en) | Method and apparatus for audio/video synchronization | |
| KR20160010507A (en) | Method and apparatus for processing video images | |
| US8798151B2 (en) | Video display device, interpolated image generation circuit and interpolated image generation method | |
| CN109089130A (en) | A kind of method and apparatus for the timestamp adjusting live video | |
| CN110740338B (en) | Bullet screen processing method and device, electronic equipment and storage medium | |
| US20150071618A1 (en) | Annotating media content for automatic content understanding | |
| US20130041998A1 (en) | DELIVERY QUALITY OF EXPERIENCE (QoE) IN A COMPUTER NETWORK | |
| CN101998029A (en) | Image processing device and image processing method | |
| KR20150057421A (en) | Multi-screen display apparatus and synchronication tuning method for the same | |
| CN108449617A (en) | A kind of method and device of control audio-visual synchronization | |
| CN104038839B (en) | Picture quality of television method of adjustment | |
| CN105516818B (en) | The method for previewing and device of video playing progress | |
| CN109275008A (en) | A kind of method and apparatus of audio-visual synchronization | |
| CN110290411A (en) | Sound is automatically corrected in video display process draws nonsynchronous method | |
| JP2012034234A (en) | Video reproduction device and video reproduction method | |
| CN101887739B (en) | Method and device for synchronizing media play | |
| CN104618737B (en) | The slow synchronous method of stream media system clock and its device | |
| CN103581730A (en) | Method for achieving synchronization of audio and video on digital set top box | |
| CN103596033A (en) | Method for solving problem of audio and video non-synchronization in multimedia system terminal playback | |
| US9432708B1 (en) | System and method for automatically determining latency of components in a video system and synchronizing components | |
| TW201308983A (en) | Display apparatus for displaying 3D images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20160824 |