CN103596033A - Method for solving problem of audio and video non-synchronization in multimedia system terminal playback - Google Patents
Method for solving problem of audio and video non-synchronization in multimedia system terminal playback Download PDFInfo
- Publication number
- CN103596033A CN103596033A CN201310556881.0A CN201310556881A CN103596033A CN 103596033 A CN103596033 A CN 103596033A CN 201310556881 A CN201310556881 A CN 201310556881A CN 103596033 A CN103596033 A CN 103596033A
- Authority
- CN
- China
- Prior art keywords
- video
- audio
- time point
- playback
- broadcasting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention belongs to the technical field of multimedia audio and video image processing and discloses a method for solving the problem of audio and video non-synchronization in multimedia system terminal playback. The method comprises the steps that 1) according to overall time and overall video frames of recording, the video mean frame rate is computed; 2) a timer is turned on for preparing video stream playing; 3) an audio stream sent by a RTSP server is directly played; 4) a video stream sent by the RTSP server is stored in a cache mode; 5) according to the computed video mean frame rate, the video stream is played in a timing mode; 6) after terminal playback, the course is finished; otherwise, the step 3), the step 4) and the step 5) are carried out again, and the audio stream and the video stream are played continuously. According to the method, the video stream is not directly played during terminal playback, the video stream is stored in the cache mode, according to the overall time and the overall video frames of recording, the video mean frame rate is computed, the video stream is played according to the video mean frame rate, and audio and video synchronization is basically achieved.
Description
Technical field
The invention belongs to multimedia audio-video technical field of image processing, particularly a kind of method that solves the asynchronous problem of audio frequency and video in the playback of multimedia system terminal.
Background technology
Fast development along with communication industry, in IMS (IP Multimedia Subsystem) multimedia system, there are various dissimilar terminals, the real-time audio and video communication of these terminals is no doubt important, but along with the proposition of backup communication demand, the audio frequency and video playback function of terminal also becomes more and more important.
In IMS multimedia system, video and audio recording server is responsible for recording the audio frequency and video in communication process and is preserved into file, when terminal proposes replay request, RTSP (Real Time Streaming Protocol) server carries out fluidisation to the file of recording, and sends to terminal for terminal playback audio/video flow by network.Terminal playback is all generally directly according to the fluidisation result of RTSP server, to carry out audio frequency and video playback.At this moment, in the process of playback, just likely there is the nonsynchronous phenomenon of audio frequency and video, basic reason is that the timestamp of some terminal video flowing when real-time communication and the video frame number of transmission per second do not mate, show as the actual video frame number sending per second and be less than the video frame number that should send per second that timestamp is corresponding, this can't go wrong in real-time communication, but in the process of RTSP server again fluidisation, but there will be the nonsynchronous problem of audio frequency and video, because RTSP server carries out fluidisation according to timestamp information, the video flowing that is playback can be faster than the audio stream of playback, video flowing is play and is finished in audio stream in advance.
Summary of the invention
For the shortcoming of above-mentioned prior art, the present invention proposes a kind of method that solves the asynchronous problem of audio frequency and video in the playback of multimedia system terminal, it is characterized in that, concrete steps are as follows:
(1), according to total duration and the total video frame number recorded, calculate the average frame per second of video;
(2) open a timer in order to the broadcasting of video flowing;
(3) play-over the audio stream that RTSP server sends over;
(4) video flowing that buffer memory RTSP server sends over;
(5) according to calculate video average frame calibration time carry out the broadcasting of video flowing;
(6) when terminal playback finishes, finish process; Otherwise repeating step (3)～(5), proceed the broadcasting of audio/video flow.
When terminal playback is suspended, suspend the broadcasting of audio stream, pause timer is to stop the broadcasting of video flowing simultaneously; When terminal playback time-out is cancelled, recover timer, then repeating step (3)～(5), carry out the broadcasting of audio/video flow.
In the process of terminal in playback, progress bar is dragged to time point A, in order to guarantee the audio-visual synchronization of time point A, put an A time point B that can make audio-visual synchronization before computing time, the video flowing starting to RTSP server request time point B, buffer memory video flowing, and according to the average frame per second displaying video stream of video; Otherwise continue step (3)～(5), carry out the broadcasting of audio/video flow.
The account form of described time point B is as follows:
A) suppose that it is t1 that video flowing starts to the duration of time point A;
B) timestamp information carrying according to known video stream calculates the real time frame rate that RTSP server sends video flowing;
C) suppose that it is t2 that video flowing starts to the duration of time point B, wherein, time point B is before time point A;
D) guarantee to pull the progress bar condition of audio-visual synchronization afterwards: the average frame per second=t2* of t1* video real time frame rate, t2=(the average frame per second of t1* video)/real time frame rate, calculates time point B with this.
The beneficial effect of the invention:
(1) the present invention play-overs the audio stream that RTSP server sends over when terminal playback, but do not play-over video flowing, but buffer memory video flowing first, according to total duration of recording and total video frame number, calculate the average frame per second of video again, according to the average frame per second of this video, carry out the broadcasting of video flowing, substantially realize the synchronous of audio frequency and video;
(2) pull the calculating that progress bar carries out the time point of audio-visual synchronization afterwards, guarantee that audio frequency and video can also be synchronous after progress bar pulls.
Accompanying drawing explanation
Fig. 1 is the playback flow process of audio frequency and video in the present invention;
Fig. 2 pulls the schematic diagram that progress bar calculates the time point of audio-visual synchronization afterwards.
Embodiment
Below in conjunction with accompanying drawing, by embodiment, further describe the present invention.
The playback flow process that is illustrated in figure 1 audio frequency and video in the present invention, concrete steps are as follows:
(1), according to total duration and the total video frame number recorded, calculate the average frame per second of video;
(2) open a timer in order to the broadcasting of video flowing;
(3) play-over the audio stream that RTSP server sends over;
(4) video flowing that buffer memory RTSP server sends over;
(5) according to calculate video average frame calibration time carry out the broadcasting of video flowing;
(6) when terminal playback finishes, finish process; Otherwise repeating step (3)～(5), proceed the broadcasting of audio/video flow.
When terminal playback is suspended, suspend the broadcasting of audio stream, pause timer is to stop the broadcasting of video flowing simultaneously; When terminal playback time-out is cancelled, recover timer, then repeating step (3)～(5), carry out the broadcasting of audio/video flow.
The effect of timer is wherein that (being generally a few tens of milliseconds) plays a frame video flowing at set intervals, and assurance video flowing can be synchronizeed with audio stream.
Audio stream and video flowing are play simultaneously.Terminal is as long as receiving audio stream from server just plays, but the video flowing that buffer memory receives from server fixed time airplay, thus guarantee audio-visual synchronization.
Be illustrated in figure 2 and pull the schematic diagram that progress bar calculates the time point of audio-visual synchronization afterwards.Wherein, suppose that progress bar is dragged to time point A, in order to guarantee the audio-visual synchronization of time point A, need the video flowing that starts to the time point B before RTSP server request time point A, buffer memory video flowing, and according to the average frame per second displaying video stream of video.
The account form of time point B is as follows:
A) suppose that it is t1 that video flowing starts to the duration of time point A;
B) timestamp information carrying according to known video stream calculates the real time frame rate that RTSP server sends video flowing;
C) suppose that it is t2 that video flowing starts to the duration of time point B, wherein, time point B is before time point A;
D) guarantee to pull the progress bar condition of audio-visual synchronization afterwards: the average frame per second=t2* of t1* video real time frame rate, t2=(the average frame per second of t1* video)/real time frame rate, calculates time point B with this.
If progress bar is not pulled, continue step (3)～(5), carry out the broadcasting of audio/video flow
The above; it is only preferably one of embodiment of the present invention; but protection scope of the present invention is not limited to this; anyly be familiar with those skilled in the art in the technical scope that the present invention discloses; the variation that can expect easily or replacement, within all should being encompassed in protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion with the protection range of claim.
Claims (4)
1. in the playback of multimedia system terminal, solve the method for the asynchronous problem of audio frequency and video, it is characterized in that, concrete steps are as follows:
(1), according to total duration and the total video frame number recorded, calculate the average frame per second of video;
(2) open a timer in order to the broadcasting of video flowing;
(3) play-over the audio stream that RTSP server sends over;
(4) video flowing that buffer memory RTSP server sends over;
(5) according to calculate video average frame calibration time carry out the broadcasting of video flowing;
(6) when terminal playback finishes, finish process; Otherwise repeating step (3)～(5), proceed the broadcasting of audio/video flow.
2. a kind of method that solves the asynchronous problem of audio frequency and video in the playback of multimedia system terminal according to claim 1, is characterized in that, when terminal playback is suspended, suspends the broadcasting of audio stream, and pause timer is to stop the broadcasting of video flowing simultaneously; When terminal playback time-out is cancelled, recover timer, then repeating step (3)～(5), carry out the broadcasting of audio/video flow.
3. a kind of method that solves the asynchronous problem of audio frequency and video in the playback of multimedia system terminal according to claim 1, it is characterized in that, if terminal progress bar in the process of playback is dragged to time point A, in order to guarantee the audio-visual synchronization of time point A, put an A time point B that can make audio-visual synchronization before computing time, the video flowing starting to RTSP server request time point B, buffer memory video flowing, and according to the average frame per second displaying video stream of video; Otherwise continue step (3)～(5), carry out the broadcasting of audio/video flow.
4. a kind of method that solves the asynchronous problem of audio frequency and video in the playback of multimedia system terminal according to claim 3, is characterized in that, the account form of described time point B is as follows:
A) suppose that it is t1 that video flowing starts to the duration of time point A;
B) timestamp information carrying according to known video stream calculates the real time frame rate that RTSP server sends video flowing;
C) suppose that it is t2 that video flowing starts to the duration of time point B, wherein, time point B is before time point A;
D) guarantee to pull the progress bar condition of audio-visual synchronization afterwards: the average frame per second=t2* of t1* video real time frame rate, t2=(the average frame per second of t1* video)/real time frame rate, calculates time point B with this.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310556881.0A CN103596033B (en) | 2013-11-11 | 2013-11-11 | Method for solving problem of audio and video non-synchronization in multimedia system terminal playback |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310556881.0A CN103596033B (en) | 2013-11-11 | 2013-11-11 | Method for solving problem of audio and video non-synchronization in multimedia system terminal playback |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103596033A true CN103596033A (en) | 2014-02-19 |
| CN103596033B CN103596033B (en) | 2017-01-11 |
Family
ID=50085988
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310556881.0A Expired - Fee Related CN103596033B (en) | 2013-11-11 | 2013-11-11 | Method for solving problem of audio and video non-synchronization in multimedia system terminal playback |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103596033B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105262979A (en) * | 2014-06-27 | 2016-01-20 | 浙江大华技术股份有限公司 | Method and device for multi-channel video playback |
| CN110446113A (en) * | 2019-07-23 | 2019-11-12 | 广州易方信息科技股份有限公司 | The method for playing back ppt and video flowing |
| CN110753201A (en) * | 2019-11-12 | 2020-02-04 | 维沃移动通信有限公司 | Video recording method and device, electronic equipment and storage medium |
| CN110944225A (en) * | 2019-11-20 | 2020-03-31 | 武汉长江通信产业集团股份有限公司 | HTML 5-based method and device for synchronizing audio and video with different frame rates |
| CN111683334A (en) * | 2020-05-29 | 2020-09-18 | 厦门盈趣科技股份有限公司 | Self-adaptive playback system, method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030058224A1 (en) * | 2001-09-18 | 2003-03-27 | Chikara Ushimaru | Moving image playback apparatus, moving image playback method, and audio playback apparatus |
| CN101076099A (en) * | 2007-06-14 | 2007-11-21 | 北京中星微电子有限公司 | Method and device for controlling video record and synchronized-controlling unit |
| CN101827271A (en) * | 2009-03-04 | 2010-09-08 | 联芯科技有限公司 | Audio and video synchronized method and device as well as data receiving terminal |
| CN102316276A (en) * | 2011-09-26 | 2012-01-11 | 成都索贝数码科技股份有限公司 | Signal switching matrix system |
| US20120169930A1 (en) * | 2010-12-14 | 2012-07-05 | Ati Technologies Ulc | Audio and Video Clock Synchronization |
-
2013
- 2013-11-11 CN CN201310556881.0A patent/CN103596033B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030058224A1 (en) * | 2001-09-18 | 2003-03-27 | Chikara Ushimaru | Moving image playback apparatus, moving image playback method, and audio playback apparatus |
| CN101076099A (en) * | 2007-06-14 | 2007-11-21 | 北京中星微电子有限公司 | Method and device for controlling video record and synchronized-controlling unit |
| CN101827271A (en) * | 2009-03-04 | 2010-09-08 | 联芯科技有限公司 | Audio and video synchronized method and device as well as data receiving terminal |
| US20120169930A1 (en) * | 2010-12-14 | 2012-07-05 | Ati Technologies Ulc | Audio and Video Clock Synchronization |
| CN102316276A (en) * | 2011-09-26 | 2012-01-11 | 成都索贝数码科技股份有限公司 | Signal switching matrix system |
Non-Patent Citations (1)
| Title |
|---|
| 吴炜，常义林: "《在接收端实现的媒体同步控制算法》", 《系统工程与电子技术》, vol. 28, no. 10, 31 October 2006 (2006-10-31), pages 1587 - 1591 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105262979A (en) * | 2014-06-27 | 2016-01-20 | 浙江大华技术股份有限公司 | Method and device for multi-channel video playback |
| CN110446113A (en) * | 2019-07-23 | 2019-11-12 | 广州易方信息科技股份有限公司 | The method for playing back ppt and video flowing |
| CN110753201A (en) * | 2019-11-12 | 2020-02-04 | 维沃移动通信有限公司 | Video recording method and device, electronic equipment and storage medium |
| CN110753201B (en) * | 2019-11-12 | 2022-02-08 | 维沃移动通信有限公司 | Video recording method and device, electronic equipment and storage medium |
| CN110944225A (en) * | 2019-11-20 | 2020-03-31 | 武汉长江通信产业集团股份有限公司 | HTML 5-based method and device for synchronizing audio and video with different frame rates |
| CN111683334A (en) * | 2020-05-29 | 2020-09-18 | 厦门盈趣科技股份有限公司 | Self-adaptive playback system, method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103596033B (en) | 2017-01-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11627351B2 (en) | Synchronizing playback of segmented video content across multiple video playback devices | |
| CN201781583U (en) | Multichannel server video playback synchronous control system | |
| CN105247437B (en) | Synchronous multiple OTT streaming clients | |
| EP2752023B1 (en) | Method to match input and output timestamps in a video encoder and advertisement inserter | |
| CN103596033A (en) | Method for solving problem of audio and video non-synchronization in multimedia system terminal playback | |
| US8526501B2 (en) | Decoder and decoding method based on video and audio time information | |
| WO2017067489A1 (en) | Set-top box audio-visual synchronization method, device and storage medium | |
| CN102215429B (en) | Recording method for mobile TV | |
| CN103546662A (en) | Audio and video synchronizing method in network monitoring system | |
| CN103338386A (en) | Audio and video synchronization method based on simplified timestamps | |
| JP2015515208A (en) | Buffer management method for synchronization of correlated media presentations | |
| WO2018120557A1 (en) | Method and device for synchronously processing audio and video, and storage medium | |
| CN103607664B (en) | A kind of audio and video synchronization method of embedded multimedia playing system | |
| CN106686438A (en) | Cross-device audio/image synchronous playing method, equipment and system | |
| CN103888813A (en) | Audio and video synchronization realization method and system | |
| CN107018437A (en) | Method and relevant device for fast channel change | |
| KR20160022307A (en) | System and method to assist synchronization of distributed play out of control | |
| CN112929713B (en) | Data synchronization method, device, terminal and storage medium | |
| CN110446113A (en) | The method for playing back ppt and video flowing | |
| CN104333795A (en) | Real-time video bitstream play speed control method independent of timestamp | |
| CN108810656A (en) | A kind of the debounce processing method and processing system of real-time live broadcast TS streams | |
| JP6051847B2 (en) | Video information reproduction method and system | |
| CN109040818A (en) | Audio and video synchronization method, storage medium, electronic equipment and system when live streaming | |
| JP2012222503A (en) | Information processing device and method, and program | |
| JP5642319B2 (en) | DIGITAL DATA DISTRIBUTION DEVICE AND METHOD, DIGITAL DATA REPRODUCTION DEVICE AND METHOD, SYNCHRONOUS REPRODUCTION SYSTEM, PROGRAM, AND RECORDING MEDIUM |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20170111 Termination date: 20211111 |