CN103596033B - Method for solving problem of audio and video non-synchronization in multimedia system terminal playback - Google Patents
Method for solving problem of audio and video non-synchronization in multimedia system terminal playback Download PDFInfo
- Publication number
- CN103596033B CN103596033B CN201310556881.0A CN201310556881A CN103596033B CN 103596033 B CN103596033 B CN 103596033B CN 201310556881 A CN201310556881 A CN 201310556881A CN 103596033 B CN103596033 B CN 103596033B
- Authority
- CN
- China
- Prior art keywords
- video
- audio
- time point
- broadcasting
- stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention belongs to the technical field of multimedia audio and video image processing and discloses a method for solving the problem of audio and video non-synchronization in multimedia system terminal playback. The method comprises the steps that 1) according to overall time and overall video frames of recording, the video mean frame rate is computed; 2) a timer is turned on for preparing video stream playing; 3) an audio stream sent by a RTSP server is directly played; 4) a video stream sent by the RTSP server is stored in a cache mode; 5) according to the computed video mean frame rate, the video stream is played in a timing mode; 6) after terminal playback, the course is finished; otherwise, the step 3), the step 4) and the step 5) are carried out again, and the audio stream and the video stream are played continuously. According to the method, the video stream is not directly played during terminal playback, the video stream is stored in the cache mode, according to the overall time and the overall video frames of recording, the video mean frame rate is computed, the video stream is played according to the video mean frame rate, and audio and video synchronization is basically achieved.
Description
Technical field
The invention belongs to multimedia audio-video technical field of image processing, particularly to one many
The method solving the asynchronous problem of audio frequency and video in the playback of media system terminal.
Background technology
Along with the fast development of communication industry, IMS (IP Multimedia Subsystem) multimedia system
Occurring various types of terminal in system, the real-time audio and video communication of these terminals is the most important,
But being as the proposition of backup communication demand, the audio frequency and video playback function of terminal also becomes more and more important.
In IMS multimedia system, the sound that video and audio recording server is responsible for recording in communication process regards
Frequency also preserves into file, when terminal proposes replay request, and RTSP (Real Time Streaming
Protocol) file recorded is fluidized by server, and by network, audio/video flow is sent
Play back for terminal to terminal.Terminal playback is typically all the direct fluidisation according to RTSP server
Result carries out audio frequency and video playback.At this moment, it is possible to occur that audio frequency and video are different during playback
Step phenomenon, basic reason be some terminal when real-time communication the timestamp of video flowing with per second
The video frame number sent does not mates, and shows as the actual video frame number sent per second less than the time
Stabbing the corresponding video frame number that should send per second, this can't go wrong in real-time communication,
But during RTSP server fluidizes again, but there will be the nonsynchronous problem of audio frequency and video,
Because RTSP server fluidizes according to timestamp information, the video flowing i.e. played back
Can be faster than the audio stream of playback, video flowing is play in audio stream in advance and is terminated.
Summary of the invention
For the shortcoming of above-mentioned prior art, the present invention proposes one and returns in multimedia system terminal
The method putting the middle asynchronous problem of solution audio frequency and video, it is characterised in that specifically comprise the following steps that
(1) according to the total duration recorded and total video frame number, the average frame per second of video is calculated；
(2) intervalometer is opened in case the broadcasting of video flowing；
(3) audio stream that RTSP server sends over directly is play；
(4) video flowing that caching RTSP server sends over；
(5) broadcasting of video flowing is carried out time according to the video average frame calibration calculated；
(6) terminate when terminal playback, then terminate process；Otherwise repeat step (3)～(5), continue
The continuous broadcasting carrying out audio/video flow.
When terminal playback is suspended, suspending the broadcasting of audio stream, pause timer is to stop simultaneously
The only broadcasting of video flowing；When terminal playback time-out is cancelled, then recovers intervalometer, then weigh
Multiple step (3)～(5), carry out the broadcasting of audio/video flow.
When terminal progress bar during playback is dragged to time point A, then time in order to ensure
Between put the audio-visual synchronization of A, one before calculating time point A can make audio-visual synchronization
Time point B, the video flowing started to RTSP server request time point B, buffered video stream,
And play video flowing according to the average frame per second of video；Otherwise continue step (3)～(5), carry out sound
The broadcasting of video flowing.
The calculation of described time point B is as follows:
A) assume that video flowing starts the time a length of t1 to time point A；
B) the timestamp information calculating RTSP server carried according to known video stream sends and regards
The real time frame rate of frequency stream；
C) assuming that video flowing starts the time a length of t2 to time point B, wherein, time point B exists
Before time point A；
D) condition of audio-visual synchronization after ensureing to pull progress bar: the average frame per second of t1* video=
T2* real time frame rate, i.e. t2=(the average frame per second of t1* video)/real time frame rate, calculate the time with this
Point B.
The beneficial effect of the invention:
(1) present invention directly plays the audio frequency that RTSP server sends over when terminal plays back
Stream, but the most directly play video flowing, but first buffered video stream, total further according to record
Duration and total video frame number calculate the average frame per second of video, regard according to the average frame per second of this video
The broadcasting of frequency stream, realizes the synchronization of audio frequency and video substantially；
(2) calculating of the time point of audio-visual synchronization is carried out after pulling progress bar, it is ensured that sound regards
Frequency can also synchronize after progress bar pulls.
Accompanying drawing explanation
Fig. 1 is the playback flow process of audio frequency and video in the present invention；
Fig. 2 be pull progress bar after calculate the schematic diagram of time point of audio-visual synchronization.
Detailed description of the invention
Below in conjunction with the accompanying drawings, the present invention is further described by detailed description of the invention.
It is illustrated in figure 1 the playback flow process of audio frequency and video in the present invention, specifically comprises the following steps that
(1) according to the total duration recorded and total video frame number, the average frame per second of video is calculated；
(2) intervalometer is opened in case the broadcasting of video flowing；
(3) audio stream that RTSP server sends over directly is play；
(4) video flowing that caching RTSP server sends over；
(5) broadcasting of video flowing is carried out time according to the video average frame calibration calculated；
(6) terminate when terminal playback, then terminate process；Otherwise repeat step (3)～(5), continue
The continuous broadcasting carrying out audio/video flow.
When terminal playback is suspended, suspending the broadcasting of audio stream, pause timer is to stop simultaneously
The only broadcasting of video flowing；When terminal playback time-out is cancelled, then recovers intervalometer, then weigh
Multiple step (3)～(5), carry out the broadcasting of audio/video flow.
The effect of intervalometer therein be at set intervals (usually a few tens of milliseconds) play one
Frame video flowing, it is ensured that video flowing can be Tong Bu with audio stream.
Audio stream and video flowing are play simultaneously.Just enter as long as terminal receives audio stream from server
Row is play, but the video flowing also timing that caching receives from server is play, thus ensures sound
Audio video synchronization.
It is illustrated in figure 2 the signal of the time point calculating audio-visual synchronization after pulling progress bar
Figure.Where it is assumed that progress bar is dragged to time point A, then in order to ensure the sound of time point A
Audio video synchronization, then need the time point B before RTSP server request time point A to open
The video flowing begun, buffered video stream, and play video flowing according to the average frame per second of video.
The calculation of time point B is as follows:
A) assume that video flowing starts the time a length of t1 to time point A；
B) timestamp information carried according to known video stream calculates RTSP server transmission
The real time frame rate of video flowing；
C) assume that video flowing starts the time a length of t2 to time point B, wherein, time point B
Before time point A；
D) condition of audio-visual synchronization after guarantee pulls progress bar: t1* video average frame
Rate=t2* real time frame rate, i.e. t2=(the average frame per second of t1* video)/real time frame rate,
Time point B is calculated with this.
If progress bar does not has dragged, then continue step (3)～(5), carry out audio/video flow
Broadcasting
The above, the only present invention preferably one of detailed description of the invention, but the guarantor of the present invention
The scope of protecting is not limited thereto, and any those familiar with the art the invention discloses
Technical scope in, the change that can readily occur in or replacement, all should contain the protection in the present invention
Within the scope of.Therefore, protection scope of the present invention should be as the criterion with scope of the claims.
Claims (2)
1. the method solving the asynchronous problem of audio frequency and video in multimedia system terminal plays back, it is characterised in that specifically comprise the following steps that
(1) according to the total duration recorded and total video frame number, the average frame per second of video is calculated；
(2) intervalometer is opened in case the broadcasting of video flowing；
(3) audio stream that RTSP server sends over directly is play；
(4) video flowing that caching RTSP server sends over；
(5) broadcasting of video flowing is carried out time according to the video average frame calibration calculated；
(6) terminate when terminal playback, then terminate process；Otherwise repeat step (3)～(5), proceed the broadcasting of audio/video flow；
If terminal progress bar during playback is dragged to time point A, then in order to ensure the audio-visual synchronization of time point A, calculate a time point B that can make audio-visual synchronization before time point A, the video flowing started to RTSP server request time point B, buffered video stream, and play video flowing according to the average frame per second of video；Otherwise continue step (3)～(5), carry out the broadcasting of audio/video flow；
The calculation of described time point B is as follows:
A) assume that video flowing starts the time a length of t1 to time point A；
B) timestamp information carried according to known video stream calculates RTSP server and sends the real time frame rate of video flowing；
C) assuming that video flowing starts the time a length of t2 to time point B, wherein, time point B is before time point A；
D) condition of audio-visual synchronization after ensureing to pull progress bar: the average frame per second=t2 × real time frame rate of t1 × video, i.e. t2=(the average frame per second of t1 × video)/real time frame rate, calculates time point B with this.
The method solving the asynchronous problem of audio frequency and video in multimedia system terminal plays back the most according to claim 1, it is characterised in that when terminal playback is suspended, suspends the broadcasting of audio stream, and pause timer is to stop the broadcasting of video flowing simultaneously；When terminal playback time-out is cancelled, then recovers intervalometer, then repeat step (3)～(5), carry out the broadcasting of audio/video flow.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310556881.0A CN103596033B (en) | 2013-11-11 | 2013-11-11 | Method for solving problem of audio and video non-synchronization in multimedia system terminal playback |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310556881.0A CN103596033B (en) | 2013-11-11 | 2013-11-11 | Method for solving problem of audio and video non-synchronization in multimedia system terminal playback |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103596033A CN103596033A (en) | 2014-02-19 |
| CN103596033B true CN103596033B (en) | 2017-01-11 |
Family
ID=50085988
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310556881.0A Expired - Fee Related CN103596033B (en) | 2013-11-11 | 2013-11-11 | Method for solving problem of audio and video non-synchronization in multimedia system terminal playback |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103596033B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105262979B (en) * | 2014-06-27 | 2018-09-28 | 浙江大华技术股份有限公司 | A kind of multi-video-stream render method and device |
| CN110446113A (en) * | 2019-07-23 | 2019-11-12 | 广州易方信息科技股份有限公司 | The method for playing back ppt and video flowing |
| CN110753201B (en) * | 2019-11-12 | 2022-02-08 | 维沃移动通信有限公司 | Video recording method and device, electronic equipment and storage medium |
| CN110944225B (en) * | 2019-11-20 | 2022-10-04 | 武汉长江通信产业集团股份有限公司 | HTML 5-based method and device for synchronizing audio and video with different frame rates |
| CN111683334B (en) * | 2020-05-29 | 2023-03-28 | 厦门盈趣科技股份有限公司 | Self-adaptive playback system, method and device |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003092761A (en) * | 2001-09-18 | 2003-03-28 | Toshiba Corp | Moving picture reproducing device, moving picture reproducing method and audio reproducing device |
| CN101076099B (en) * | 2007-06-14 | 2010-06-09 | 北京中星微电子有限公司 | Method and device for controlling video record and synchronized-controlling unit |
| CN101827271B (en) * | 2009-03-04 | 2012-07-18 | 联芯科技有限公司 | Audio and video synchronized method and device as well as data receiving terminal |
| US8599310B2 (en) * | 2010-12-14 | 2013-12-03 | Ati Technologies Ulc | Audio and video clock synchronization |
| CN102316276B (en) * | 2011-09-26 | 2013-06-19 | 成都索贝数码科技股份有限公司 | Signal switching matrix system |
-
2013
- 2013-11-11 CN CN201310556881.0A patent/CN103596033B/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| CN103596033A (en) | 2014-02-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103596033B (en) | Method for solving problem of audio and video non-synchronization in multimedia system terminal playback | |
| CN201781583U (en) | Multichannel server video playback synchronous control system | |
| CN106385620B (en) | A kind of data smoothing output method based on Streaming Media | |
| CN107231563A (en) | Method for processing video frequency and device | |
| TWI424747B (en) | System and method for connection to a network and for reproducing an audio/video stream received from a server on the network | |
| CN109089130A (en) | A kind of method and apparatus for the timestamp adjusting live video | |
| CN104410807A (en) | Method and device for synchronously replaying multi-channel video | |
| JP2015515208A (en) | Buffer management method for synchronization of correlated media presentations | |
| CN103607664B (en) | A kind of audio and video synchronization method of embedded multimedia playing system | |
| US8526501B2 (en) | Decoder and decoding method based on video and audio time information | |
| CN103200461A (en) | Multiple-player-terminal synchronized playing system and playing method | |
| WO2020215453A1 (en) | Video recording method and system | |
| CN105141810B (en) | It is a kind of to support slow-motion multichannel presentation system and method in real time | |
| CN109257631A (en) | Video carousel playback method, device, computer equipment and storage medium | |
| CN102623032B (en) | Method for realizing synchronous playback of hard disk recorder | |
| WO2008076849A3 (en) | Synchronous multi-media recording and playback with end user control | |
| US9653117B2 (en) | Interconnected multimedia systems with synchronized playback of media streams | |
| CN110446113A (en) | The method for playing back ppt and video flowing | |
| KR20160022307A (en) | System and method to assist synchronization of distributed play out of control | |
| US20080022007A1 (en) | System and method of audio/video streaming | |
| CN104618673B (en) | A kind of multichannel video recording synchronized playback control method and device based on NVR | |
| CN109905763B (en) | Method for optimizing audio playing pause of player | |
| CN105791735A (en) | Method and system for dynamically adjusting video call code streams | |
| CN110225385B (en) | Audio and video synchronization adjustment method and device | |
| JP2012222503A (en) | Information processing device and method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20170111 Termination date: 20211111 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |