CN101119461B - System and method for maintaining video frame and audio frame synchronous broadcasting - Google Patents
System and method for maintaining video frame and audio frame synchronous broadcasting Download PDFInfo
- Publication number
- CN101119461B CN101119461B CN200610108410A CN200610108410A CN101119461B CN 101119461 B CN101119461 B CN 101119461B CN 200610108410 A CN200610108410 A CN 200610108410A CN 200610108410 A CN200610108410 A CN 200610108410A CN 101119461 B CN101119461 B CN 101119461B
- Authority
- CN
- China
- Prior art keywords
- video
- frame
- audio
- time
- play
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The present invention discloses a method that is used for maintaining synch play of the video frame and the audio frame in video player system and consists of the following procedures: (a) read the pre-set play time of audio in the audio frame and obtain the real audio play time of the audio frame; (b) calculate synch excursion time according to the pre-set video play time and the actual audio play time; (c) calculate adjusting video play time corresponding with the video frame according to the synch excursion time, pre-set play time of the video frame and pre-set video play time; and(d) optically display the video frame according to the adjusting video play time and present time. Thus, the video and the audio frame can maintain synch play.
Description
Technical field
The invention relates to a kind of video-audio playing system (Audio/video player system), particularly about a kind of video-audio playing system and method thereof in order to maintenance frame of video (Video frame) and audio frame (Audio frame) synchronous playing.
Background technology
A video-audio playing system must can be play the audio/video file of depositing in the storage device, if the video-audio data of this audio/video file has through encoding process, after video-audio playing system also must use suitable audio or video decoder decode audio frequency and video data, audio plays and video data again.Concerning a video-audio playing system, keeping between video and audio frequency is a very big challenge synchronously.
Usually keeping between video and audio frequency is according to being recorded in after predetermined reproduction time of audio frame in the audio/video file and the predetermined reproduction time of frame of video make comparisons with time at present and calculate synchronously, broadcast between in due course audio frame or frame of video keep between video and audio frequency synchronously.In general, the broadcast of audio frequency can not can not be later than video above 45 milliseconds early than the broadcast that video surpasses 15 milliseconds and audio frequency.If this condition is not when satisfying, the beholder just can clearly experience asynchronous between video and audio frequency.
The tradition video-audio playing system is to read from storage device and displaying video and audio frequency with single integrated process, and this program can be kept the synchronous of video track and audio track.Yet, maturation along with the computing environment of multiplex (MUX) or many threads, most computer user wishes can be when watching digital picture, also can in computer, carry out other program or function, so above-mentioned unique sequence play-back technology is eliminated gradually, the substitute is video data and voice data branch are parted and independently both decoded, handle and play to meet the demand of multiplex's environment.
Yet such function but causes extra multimedia stationary problem to take place.Particularly video data and voice data are divided and part and independently with both decodings, when handling and playing, can make at an easy rate that just video data and voice data can't synchronous playing.
In known technology, proposition is arranged in order to keep the method for frame of video and audio frame synchronous playing, announce the 6th as United States Patent (USP), 510, No. 279 (' No. 279 patents), United States Patent (USP) announce the 6th, 262, and No. 776 (' No. 776 patents) and United States Patent (USP) announce the 6th, 016, No. 166 (' No. 166 patents).But these known technologies but can't solve following problems.
In general, can cause the reason that video data and voice data can't synchronous playing to have following two kinds: (1) when display (Monitor) displaying video frame (Video frame), the display delay time of being caused (Display Delay Time); (2) inaccuracy of audio output device output sampling rate (Inaccuracy of 0utput Sample Rate), detailed as following.
See also Fig. 1, when Fig. 1 is display displaying video frame, cause the schematic diagram of display delay time.As shown in Figure 1, when display in time T
V(T
V0, T
V1, T
V2) displaying video frame V (V
0, V
1, V
2) time, frame of video V (V
0, V
1, V
2) time of really being shown on the indicator screen is T
V' (T
V0', T
V1', T
V2'), that is display needs through handling after a while, just can be with frame of video V ' (V
0', V
1', V
2') be shown on the screen.This reproduction time difference is the display delay time (Display Delay Time) D
LCD, as shown in Figure 1.
See also Fig. 2 A and Fig. 2 B, Fig. 2 A is that the audio frame under the ideal state is play schematic diagram. Fig. 2 B is that the audio frame under the actual state is play schematic diagram. and the t among Fig. 2 A and Fig. 2 B
An(n=1,2 ...) representative is recorded in file sound intermediate frequency frame A
nThe time that should play, T
AnRepresent audio frame A
nVia the time of audio playing apparatus actual play.Under ideal state, as audio frame A
nWhen playing in time, be recorded in file sound intermediate frequency frame A
nThe time t that should play
AnCan just equal the time T of actual play
An, shown in Fig. 2 A.Yet, because the inaccuracy of audio frequency output sampling rate after audio playing apparatus is play a period of time, is recorded in file sound intermediate frequency frame A
nPredetermined reproduction time t
AnWith actual play time T
AnJust can begin to produce difference, shown in Fig. 2 B.
With AMR is example, and each AMR audio frame comprises 160 audio samples (Sample), and the audio frequency output sampling rate of broadcast (Output Sample Rate) is 8000 hertz (Hz).But because the clock inaccuracy of the audio output device that adopts, actual output sampling rate may be 7999 hertz.The audio samples number that can play normal each second just is reduced to 7999 by 8000.So after playing 1000 seconds, the audio samples number that the audio samples number of actual play just should be play has more in theory lacked 1000* (8000-7999)=1000.Push away through after 1000 seconds thus, audio frame A
nThe time t that should play
AnTime T with actual play
AnJust poor 1000* (1/8000*100)=125 millisecond.By nonsynchronous condition between video of mentioning before and audio frequency, the user will discover asynchronous between audio ﹠ video.
In the known technology, ' No. 279 patents and ' No. 166 patents all needleless are improved the uncertainty of display delay time and audio frequency output sampling rate.Though ' No. 776 patent has been improved the display delay time, does not still consider the uncertainty of audio frequency output sampling rate.
Therefore, main purpose of the present invention is to provide a kind of video-audio playing system and method thereof, to address the above problem.
Summary of the invention
A purpose of the present invention is to provide a kind of video-audio playing system and method thereof, in order to the synchronous playing of maintenance frame of video and audio frame, and then promotes audio-visual play quality.
According to a preferred embodiment of the present invention, video-audio playing system (Audio/Video playersystem) comprises memory (Memory), processor (Processor), audio decoder (Audiodecoder), Video Decoder (Video decoder), bus (Bus), memory interface (Storageinterface), storage device (Storage device), audio output interface, audio output device (Audio output device), video output interface and video output device (Video outputdevice).
In the above embodiments, memory is in order to storing software program (Software program) and temporary audio frame and video requency frame data.Bus is used for the communication between each interface and memory, processor, audio decoder and the Video Decoder.Storage device is in order to deposit the audio/video file of compression, comprise audio frame packed data and predetermined audio reproduction time and video frame compression data and predetermined video playback time and utilize memory interface with other bus on the element communication.Audio decoder and Video Decoder are in order to decoding and coding audio frame and frame of video later.Audio output interface and video output interface deliver audio output device in order to will decode audio frame later and frame of video and video output device is play.Processor is stored in software program in the memory in order to execution, and controls each element, in reasonable time audio plays and video data.
Audio frame after processor will be encoded and video requency frame data take out from storage device via memory interface and are temporary in the memory, simultaneously and obtain the predetermined audio reproduction time and the video playback time of this audio frame and frame of video, and control audio decoder and video decoder decodes be temporary in audio frame and video requency frame data in the memory.Audio frame after its preprocessor will be decoded and video requency frame data deliver audio output device via audio output interface and video output interface respectively and video play device is play, and obtains the realAudio reproduction time.Predetermined audio reproduction time and this realAudio reproduction time according to this audio frame, processor calculates the simultaneous bias time (Synchronization offset time). and according to the predetermined video playback time and the display delay time (DisplayDelay Time) of this simultaneous bias time, this frame of video, processor calculates the adjustment video playback time corresponding to this frame of video.Adjust video playback time and present time according to this, processor selection ground is delivered this frame of video to video play device and is play.By this, frame of video and audio frame can keep synchronous playing.
The present invention also provides a kind of method that keeps frame of video and audio frame synchronous playing in video-audio playing system, this method comprises the following step: (a) read the audio frame in these a plurality of audio frames, obtain the predetermined audio reproduction time, and this audio frame of decoding, play this audio frame, obtain the realAudio reproduction time of this audio frame; (b) according to this predetermined audio reproduction time and this realAudio reproduction time of this audio frame, calculate the simultaneous bias time; (c) read frame of video in these a plurality of frame of video, obtain the predetermined video playback time; (d) according to predetermined video playback time of being somebody's turn to do of this simultaneous bias time, this frame of video and display delay time, calculate the adjustment video playback time corresponding to this frame of video; And (e) this frame of video of decoding, play this frame of video according to this adjustment video playback time.
Therefore, by video-audio playing system of the present invention and method thereof, not only considered the display delay time that display causes, and further considered the inaccuracy of audio frequency output sampling rate, thereby can keep the synchronous playing of frame of video and audio frame at any time, and then promote audio-visual play quality.
Can be about the advantages and spirit of the present invention by following detailed Description Of The Invention and appended graphic being further understood.
Description of drawings
When Fig. 1 is display displaying video frame, cause the schematic diagram of display delay time.
Fig. 2 A is that the audio frame under the ideal state is play schematic diagram.
Fig. 2 B is that the audio frame under the actual state is play schematic diagram.
Fig. 3 is the functional block diagram of the video-audio playing system of first preferred embodiment according to the present invention.
Fig. 4 is the presentation of video frames schematic diagram of first preferred embodiment according to the present invention.
Fig. 5 is the flow chart of the audio-visual synchronous broadcast method of first preferred embodiment according to the present invention.
[main element label declaration]
10: video-audio playing system 12: memory
14: processor 16: audio decoder
18: Video Decoder 20: bus
22: memory interface 24: audio output interface
26: video output interface 28: storage device
30: audio output device 32: video output device
A
n: audio frame V
i: frame of video
t
An: predetermined audio reproduction time t
Vi: the predetermined video playback time
T
An: realAudio reproduction time D
Sync: the simultaneous bias time
t
Adj: adjust the video playback time D
LCD: the display delay time
T: present time S100～S128: process step
Embodiment
See also Fig. 3, Fig. 3 is the functional block diagram of the video-audio playing system 10 of first preferred embodiment according to the present invention. video-audio playing system (Audio/video player system) 10 is in order to keep the synchronous playing of all frame of video (Video frame) and audio frame (Audio frame).As shown in Figure 3, video-audio playing system 10 comprises memory 12, processor 14, audio decoder 16, Video Decoder 18, bus 20, memory interface 22, storage device 28, audio output interface 24, audio output device 30, video output interface 26, video output device 32.Audio output device 30 can be loud speaker or other similar device, and video output device 32 can be LCD (Liquid crystal display, LCD) or other similar device.It should be noted that processor 14, memory 12, audio decoder 16, Video Decoder 18, memory interface 28, audio output interface 24, video output interface 26, with bus 20 can be integrated in system single chip (System on a Chip, SoC) in.If the usefulness height of processor 14, the function of alternative audio decoder 12 is carried out the audio frame that decoding has been compressed by processor 14.If the usefulness of processor 14 is high, even also alternative Video Decoder 18, carry out the frame of video that decoding has been compressed by processor 14.
Please consult Fig. 2 B and Fig. 4 more in the lump, Fig. 4 is the presentation of video frames schematic diagram of first preferred embodiment according to the present invention.In the above embodiments, audio frame and predetermined reproduction time or frame of video and the predetermined reproduction time thereof of processor 14 in order to read compression from storage device 28.If obtain compressed audio frame A earlier
nAnd predetermined audio reproduction time t
An, processor 14 utilizes audio decoder 16 that this audio frame An decoding is broadcasted from audio output device 30 via audio output interface 24 again, at the same time and obtain this audio frame A
nRealAudio reproduction time T
AnAccording to this audio frame A
nPredetermined audio reproduction time t
AnAnd realAudio reproduction time T
An, processor 14 calculates simultaneous bias time (Synchronizationoffset time) D
Sync(t
An-T
An), shown in Fig. 2 B.If obtain compressed video frame V earlier
i(i=1,2 ...) and predetermined video playback time t
Vi, according to the simultaneous bias time D
SyncAnd display delay time D LCD, processor 14 calculates corresponding to this frame of video V
iAdjustment video playback time t
Adj(t
Vi-D
LCD+ D
Sync), as shown in Figure 4.Adjust video playback time t according to this
AdjAnd present time T, processor 14 is optionally with frame of video V
iDelivering video output device 32 via video output interface 26 plays.By this, frame of video and audio frame can keep synchronous playing.In this embodiment, predetermined video playback shift time D
LCDBe meant when video play device displaying video frame the display delay time of being caused.Time T is meant that present system shows the time at present.
In the above embodiments, processor 14 is according to advancing optionally displaying video frame of rank delay and drop policy (Advanced Delayand Drop Policy).If this adjusts video playback time t
AdjAfter this present time T, that is present system shows that the time does not reach this adjustment video playback time t as yet
Adj, processor 14 can postpone (Delay) this present time T and adjust video playback time t to this
AdjThe time, just play this frame of video V
iIf this adjusts video playback time t
AdjBefore this present time T, that is present system shows that the time has surpassed this and adjusted video playback time t
Adj, 14 of processors can further be judged this adjustment video playback time t
AdjWith the difference of this present time T whether greater than predetermined critical (Threshold).If this adjusts video playback time t
AdjDuring greater than this predetermined critical, processor 14 can abandon (Drop) frame of video V with the difference of this present time T
i, that is this frame of video V
iCan not be played, failing processor 14 can be play this frame of video V in present time T
iBy this, not only frame of video and audio frame can keep synchronous playing, and can under the principle that does not influence synchronous playing decoded frame of video be played back as far as possible, to promote video playing quality.In this embodiment, predetermined critical can design it according to different demands.For example, in a preferred embodiment, predetermined critical can be set at t
Vi-2D
LCD
See also Fig. 5, Fig. 5 is the flow chart of the audio-visual synchronous broadcast method of first preferred embodiment according to the present invention.According to above-mentioned preferred embodiment, method of the present invention is used to keep in the video-audio playing system synchronous playing of frame of video and audio frame.This method comprises the following step:
Step S100: beginning;
Step S102: from storage device, read video-audio data, obtain predetermined reproduction time of audio frame and audio frequency or frame of video and predetermined video playback time;
Step S104: as if what read in step S102 is audio frame, then carries out step S106; If not, carry out step S112;
Step S106: the audio frame that the control audio decoder decode has compressed;
Step S108: with decoded audio frame number certificate, be sent to audio output interface, play by audio output device;
Step S110: calculate the simultaneous bias time according to realAudio reproduction time and the predetermined reproduction time of audio frequency;
Step S112: what read in step S102 is frame of video, then the frame of video compressed of control of video decoder decode;
Step S114:, calculate the video playback time of adjusting according to the predetermined video broadcast time and the display delay time (Display Delay Time) of simultaneous bias time and this frame of video;
Step S116: confirm to adjust the video playback time whether before the time at present; If not, carry out step S120; If carry out step S118;
Step S118: whether confirm to adjust the video playback time greater than critical value; If then carry out step S122; If not, then carry out step S124;
Step S120: postpone in adjusting video playback displaying video frame during the time;
Step S122: abandon frame of video, do not play;
Step S124: in displaying video frame immediately of present time;
Step S126: all audio frames and frame of video are all processed; If carry out step S128; If not, carry out step S102;
Step S128: finish.
Compared to known technology, video-audio playing system of the present invention and method thereof have not only been considered the display delay time that display causes, and further considered the inaccuracy of audio frequency output sampling rate, thereby can keep the synchronous playing of frame of video and audio frame at any time, and then promoted audio-visual play quality.In addition, enter rank and postpone and drop policy by of the present invention, not only frame of video and audio frame can keep synchronous playing, and can under the principle that does not influence synchronous playing decoded frame of video be played back as far as possible, to promote video playing quality.
By the above detailed description of preferred embodiments, be to wish to know more to describe feature of the present invention and spirit, and be not to come category of the present invention is limited with above-mentioned disclosed preferred embodiment.On the contrary, its objective is that hope can contain in the category of claim scope of being arranged in of various changes and tool equality institute of the present invention desire application.
Claims (2)
1. method that in video-audio playing system, keeps frame of video and audio frame synchronous playing, this method comprises the following step:
(a) read audio frame in a plurality of audio frames, obtain the predetermined audio reproduction time, and this audio frame of decoding, play this audio frame, obtain the realAudio reproduction time of this audio frame;
(b) according to this predetermined audio reproduction time and this realAudio reproduction time of this audio frame, calculate the simultaneous bias time;
(c) read frame of video in a plurality of frame of video, obtain the predetermined video playback time;
(d) according to predetermined video playback time of being somebody's turn to do of this simultaneous bias time, this frame of video and display delay time, calculate the adjustment video playback time corresponding to this frame of video; And
(e) this frame of video of decoding is play this frame of video according to this adjustment video playback time.
2. method according to claim 1, wherein step (e) also comprises the following step:
(e1) judge that this adjustment video playback time is whether before the time at present; If not, execution in step (e2) then; If, execution in step (e3) then;
(e2) postpone during the time, to play this frame of video in this adjustment video playback; And
(e3) judge that whether the difference of this adjustment video playback time and this present time is greater than critical value; If then abandon this frame of video; If not, then play this frame of video in the present time immediately.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200610108410A CN101119461B (en) | 2006-08-02 | 2006-08-02 | System and method for maintaining video frame and audio frame synchronous broadcasting |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200610108410A CN101119461B (en) | 2006-08-02 | 2006-08-02 | System and method for maintaining video frame and audio frame synchronous broadcasting |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101119461A CN101119461A (en) | 2008-02-06 |
| CN101119461B true CN101119461B (en) | 2010-05-12 |
Family
ID=39055360
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200610108410A Expired - Fee Related CN101119461B (en) | 2006-08-02 | 2006-08-02 | System and method for maintaining video frame and audio frame synchronous broadcasting |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101119461B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102063923B (en) * | 2009-11-18 | 2015-05-27 | 新奥特（北京）视频技术有限公司 | Adaptive playing method and device of animation |
| CN102082774B (en) * | 2009-12-01 | 2015-02-18 | 新奥特（北京）视频技术有限公司 | Stream media data playing method and system |
| EP2534833B1 (en) * | 2010-02-12 | 2016-04-27 | Thomson Licensing | Method for synchronized content playback |
| CN101877791A (en) * | 2010-06-04 | 2010-11-03 | 浙江工业大学 | MPEG4-based audio/video synchronization control method in video network transmission |
| CN103220058A (en) * | 2012-01-20 | 2013-07-24 | 旭扬半导体股份有限公司 | Audio frequency data and vision data synchronizing device and method thereof |
| CN104954727B (en) * | 2014-03-25 | 2018-08-07 | 南宁富桂精密工业有限公司 | Audio-visual synchronization control device and method |
| CN104104987B (en) * | 2014-07-29 | 2018-05-18 | 小米科技有限责任公司 | Picture and synchronous sound method and device in video playing |
| CN104243850B (en) * | 2014-09-15 | 2018-04-17 | 苏州佳世达电通有限公司 | The method that data is played simultaneously |
| CN106658133B (en) * | 2016-10-26 | 2020-04-14 | 广州市百果园网络科技有限公司 | Audio and video synchronous playing method and terminal |
| CN109379619B (en) * | 2018-11-20 | 2021-05-18 | 海信视像科技股份有限公司 | Sound and picture synchronization method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1201335A (en) * | 1997-03-19 | 1998-12-09 | 索尼公司 | Synchronous deviation control apparatus and method thereof |
| JP2005341256A (en) * | 2004-05-27 | 2005-12-08 | Yamaha Corp | Audio video amplifier |
| WO2005117431A1 (en) * | 2004-05-26 | 2005-12-08 | Vividas Technologies Pty Ltd | Method for synchronising video and audio data |
| CN1758772A (en) * | 2005-11-04 | 2006-04-12 | 无敌科技(西安)有限公司 | Method for synchronous playing video and audio of medium document and its system |
-
2006
- 2006-08-02 CN CN200610108410A patent/CN101119461B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1201335A (en) * | 1997-03-19 | 1998-12-09 | 索尼公司 | Synchronous deviation control apparatus and method thereof |
| WO2005117431A1 (en) * | 2004-05-26 | 2005-12-08 | Vividas Technologies Pty Ltd | Method for synchronising video and audio data |
| JP2005341256A (en) * | 2004-05-27 | 2005-12-08 | Yamaha Corp | Audio video amplifier |
| CN1758772A (en) * | 2005-11-04 | 2006-04-12 | 无敌科技(西安)有限公司 | Method for synchronous playing video and audio of medium document and its system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101119461A (en) | 2008-02-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101119461B (en) | System and method for maintaining video frame and audio frame synchronous broadcasting | |
| US20220182720A1 (en) | Reception apparatus, reception method, and program | |
| JP3053541B2 (en) | Synchronous variable speed playback of digitally recorded audio and video | |
| US7738767B2 (en) | Method, apparatus and program for recording and playing back content data, method, apparatus and program for playing back content data, and method, apparatus and program for recording content data | |
| KR100862630B1 (en) | System and method for synchronizing video frames and audio frames | |
| US6185369B1 (en) | Apparatus and method for synchronously reproducing multi-angle data | |
| JP5903924B2 (en) | Receiving apparatus and subtitle processing method | |
| CN102215429B (en) | Recording method for mobile TV | |
| US20030095790A1 (en) | Methods and apparatus for generating navigation information on the fly | |
| CN102113339B (en) | Digital broadcast reproduction device and digital broadcast reproduction method | |
| CN103190092A (en) | A system and method for synchronized playback of streaming digital content | |
| US20040086263A1 (en) | System for maintaining history of multimedia content and method thereof | |
| KR100744594B1 (en) | Content reproduce system, reproduce device, reproduce method, and distribution server | |
| CN101290790B (en) | Synchronous playing method and device for both audio and video | |
| JP2006324914A (en) | Broadcast program receiver | |
| WO2001018658A1 (en) | Method and apparatus for sending slow motion video-clips from video presentations to end viewers upon request | |
| JP2004040579A (en) | Digital broadcast reception device and synchronous reproduction method for digital broadcast | |
| JP2006148679A (en) | Data processing device | |
| US8213778B2 (en) | Recording device, reproducing device, recording medium, recording method, and LSI | |
| CN102231853B (en) | Program processing method and mobile terminal | |
| JP2003219367A5 (en) | Television broadcast processing apparatus, television broadcast processing system, and computer program | |
| CN103404162A (en) | Broadcast receiving device and method | |
| JPH11134804A (en) | Image and audio synchronization system | |
| JP2008193616A (en) | Program distribution system and program | |
| JP3825920B2 (en) | Recording / playback device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20100512 Termination date: 20180802 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |