CN103607664B - A kind of audio and video synchronization method of embedded multimedia playing system - Google Patents
A kind of audio and video synchronization method of embedded multimedia playing system Download PDFInfo
- Publication number
- CN103607664B CN103607664B CN201310603239.3A CN201310603239A CN103607664B CN 103607664 B CN103607664 B CN 103607664B CN 201310603239 A CN201310603239 A CN 201310603239A CN 103607664 B CN103607664 B CN 103607664B
- Authority
- CN
- China
- Prior art keywords
- frame
- video
- packet
- audio
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The present invention provides the audio and video synchronization method of a kind of embedded multimedia playing system, it is characterized in that: comprise the steps: that step 1. receives audio and video data streams, and decode, the decoded audio/video frames of step 2. flows into display buffer or relief area of jumping the queue, and decoded audio frame enters audio buffer；Entering display buffer for the most decoded frame of video, after being later than for previous packet decoding, the previous packet video frame of a packet decoding enters relief area of jumping the queue；Step 3. synchronizes to play: the present invention has the advantage that with good effect and is: the present invention is applied in track traffic industry PIS system, complete multimedia messages to issue and transmit from operational control center (OCC) to car, finally realizing audio-visual synchronization and be presented to the function of passenger, raising passenger takes user experience during railcar.
Description
Technical field
The invention belongs to the communications field, especially relate to the audio frequency and video of a kind of embedded multimedia playing system
Synchronous method.
Background technology
Due to factors such as the unreliability of network and propagation delay times, often meet during real-time data communication
To such as packet loss, out of order, shake, the series of problems such as repeat.In order to enable the most effectively
Transmitting real-time data, IETF (Internet Engineering Task Force, Internet engineering
Task groups) develop RTP (Real-timeTransport Protocol, RTP).RTP
The data offer for interactive audio, video etc. with real-time characteristic transmits service, end to end including two
Individual protocol RTP in close relations and RTCP (Real-time Transport ControlProtocol,
RTCP Real-time Transport Control Protocol).
RTP is a kind of host-host protocol on Internet for multimedia data stream, but RTP is originally
Body can not provide reliable transfer mechanism for transmitting packet in order, does not the most provide flow-control or gathers around
Plug controls, and it relies on RTCP to provide these to service.Flow-control is provided together with RTCP with RTP and gathers around
Plug controls service.Server by utilizing RTCP information dynamically changes transfer rate, even changes and effectively carries
Lotus type.RTP and RTCP with the use of, they can make transmission with effective feedback and minimum expense
Efficiency optimization, thus be particularly suitable for transmitting online real time data.
In prior art, in traditional PIS system, the transmission of multimedia messages and issue are by simulation
Or the mode of Semi-digital realizes, along with network technology and the development of computer technology, prior art
Cannot meet user to the requirement in terms of HD video.
Summary of the invention
The problem to be solved in the present invention is to provide the audio-visual synchronization of a kind of embedded multimedia playing system
Method.
For solving above-mentioned technical problem, the technical solution used in the present invention is:
The audio and video synchronization method of a kind of embedded multimedia playing system, comprises the steps:
Step 1. receives audio and video data streams, and decodes:
Start the thread receiving RTP audio frequency and video bag, receive RTP bag, afterwards it is resolved, resolve
Its data type carried, i.e. RTP audio pack or RTP video bag, if RTP audio pack enters audio frequency
Receive relief area, after be sent to audio decoder and be decoded；If RTP video bag enters buffering of queuing up
District, sequence before being first decoded RTP video bag processes, after be sent to Video Decoder and solve
Code；
Frame of video has just enter into the processing procedure in the queuing buffer:
RTP bag is directly entered the queuing buffer, proceeds by queuing decoding, first according to RTP video bag
Entrained packet number, the most first finds the position of other bags of packet belonging to this packet number, then
In this packet, utilize self-contained serial number in this packet from big to small, start relatively to come one by one
Quickly determine oneself position in this packet；
During the decoding of RTP frame of video:
When the i-th packet will decode, it is judged that whether i+1 is grouped complete I frame:
If so, it is decoded；
If it is not, this packet is moved to relief area, judge i-th+2 is grouped whether there is I frame afterwards:
If being decoded, when will decode, it may be judged whether playing the i-th+2 grouped data；If
It is to abandon i+1 packet, if otherwise starting i+1 packet decoding；
If no-trump i-th+2 packet moves on to relief area, after judge whether to receive the frame of video of i+1 packet:
If having, i+1 packet removal relief area is decoded；
If without proceeding to judge i-th+3 is grouped whether there is I frame:
If without abandoning the i+1 packet in relief area；
If being decoded, after judge whether to play the i-th+3 packet according to the feedback of display buffer:
If playing, then abandon i+1 packet and the i-th+2 packet；
If not playing, then return and judge whether i+1 packet has complete I frame, cycle criterion.
Step 2.
Decoded audio frame enters audio buffer；
Display buffer is entered for the most decoded frame of video, late for previous packet decoding
Previous packet video frame in a rear packet decoding enters relief area of jumping the queue.
Step 3. synchronizes to play:
Audio frame in audio buffer is play continuously；
Frame of video in display buffer carries out played in order, and the frame of video of relief area of jumping the queue carries out jumping the queue broadcasting
Put；
Wherein the reproduction time of frame of video is that to utilize NTP with RTP timestamp to calculate current video frame corresponding
Audio frame play time and obtain, it is judged that the time labelling of foremost one frame of current buffer with
Time ± △ relation, wherein Time ± △ represents and allows the time range that plays out of frame of video:
If in the range of the time of the foremost of current buffer one frame is marked at Time ± △, then normal play；
If the time labelling of the foremost of current buffer one frame is less than Time-△, (frame is late) loses
Abandon this frame of video；
If the time labelling of the foremost of current buffer one frame is more than Time+ △, (frame comes early) weight
A former frame is put in replay, the most again compares, judges:
(1) if in the range of the time is marked at Time ± △, then normal play；
(2) if still greater than; would forward jitter-buffer to, find forward lower frame, until former frame weighs altogether
Broadcast 3 times, existing 3 frames of jitter-buffer, compare for the last time；If the time is marked at Time ± △ scope
In, then normal play；
Otherwise it is switched to video as standard, disposably adjusts audio frequency, the most again switch back to audio frequency
For standard.
Step 4. display buffer feeds back information to Video Decoder.
Further, in step 1, if first bag of this packet of RTP bag for just having received,
First determine whether the frame type of oneself:
If not I frame, directly determine the packet positions of oneself according to the size of packet number, in comparison procedure
In, find there is no the packet number the same with oneself, then move on to first and open up Xin Kuai district；
If I frame, finding the block district oneself being grouped, be made whether that I frame judges, Ruo Gaikuai district has
Have I frame, then this bag abandons automatically；It is otherwise ordinary circumstance, normal process.
Further, in step 2, the initial value size of display buffer and relief area of jumping the queue is according to regarding
Frequently compressed format dynamically adjusts.
Further, in step 3, take to use the audio frame received based on playing the same of time limit
One step process, is compensated the delay variation of network, is come by display buffer with the audio frame of receiving terminal caching
Realize audio playback.
The present invention has the advantage that with good effect: the present invention is applied to track traffic industry PIS system
In, complete multimedia messages and issue and transmit from operational control center (OCC) to car, finally realize
Audio-visual synchronization is presented to the function of passenger, and raising passenger takes user experience during railcar.
Accompanying drawing explanation
Fig. 1 is the structural representation of the present invention；
Fig. 2 is the process chart that the frame of video of the present invention has just enter into the queuing buffer；
Fig. 3 is the frame of video decoding process flow chart of the present invention；
Fig. 4 is the frame of video process chart in the display buffer of the present invention.
Detailed description of the invention
As shown in Figures 1 to 4:
The audio and video synchronization method of a kind of embedded multimedia playing system of the present invention, comprises the steps:
Step 1. receives audio and video data streams, decodes after sequence:
Start the thread receiving RTP audio frequency and video bag, receive RTP bag, afterwards it is resolved, resolve
Its data type carried, i.e. RTP audio pack or RTP video bag, if RTP audio pack enters audio frequency
Receive relief area, after be sent to audio decoder and be decoded；If RTP video bag enters buffering of queuing up
District, sequence before being first decoded RTP video bag processes, after be sent to Video Decoder and solve
Code；
As in figure 2 it is shown, just entered the processing procedure in the queuing buffer for frame of video:
RTP video bag is directly entered the queuing buffer, first according to the packet number entrained by RTP video bag,
Judge,
If there is no the packet number the same with oneself, open up a new chunk；
If having the position of other bags the most first finding packet belonging to this packet number, then in this packet
In, utilize self-contained serial number in this packet from big to small, start to compare one by one and quickly determine
Oneself is in the position of this packet；Determining whether I frame afterwards, no if, compare in organizing and determine
Position in group；
It is for I frame, then judges whether this block district has I frame, then abandon this frame if being, if not
It is to compare in organizing and determine position in group.
As it is shown on figure 3, during the decoding of RTP frame of video:
When the i-th packet will decode, it is judged that whether i+1 is grouped complete I frame:
If so, it is decoded；
If it is not, this packet is moved to relief area, judge i-th+2 is grouped whether there is I frame afterwards:
If being decoded, when will decode, it may be judged whether playing the i-th+2 grouped data；If
It is to abandon i+1 packet, if otherwise starting i+1 packet decoding；
If no-trump i-th+2 packet moves on to relief area, after judge whether to receive the frame of video of i+1 packet:
If having, i+1 packet removal relief area is decoded；
If without proceeding to judge i-th+3 is grouped whether there is I frame:
If without abandoning the i+1 packet in relief area；
If being decoded, after judge whether to play the i-th+3 packet according to the feedback of display buffer:
If playing, then abandon i+1 packet and the i-th+2 packet；
If not playing, then return and judge whether i+1 packet has complete I frame, cycle criterion.
Step 2.
Decoded audio frame enters audio buffer；
Display buffer is entered for the most decoded frame of video, late for previous packet decoding
Previous packet video frame in a rear packet decoding enters relief area of jumping the queue；
Step 3. synchronizes to play:
Audio frame in audio buffer is play continuously；
Frame of video in display buffer carries out played in order, and the frame of video of relief area of jumping the queue carries out jumping the queue broadcasting
Put；
Wherein the reproduction time of frame of video is that to utilize NTP with RTP timestamp to calculate current video frame corresponding
Audio frame play time and obtain, it is judged that the time labelling of foremost one frame of current buffer with
Time ± △ relation, wherein Time ± △ represents the time range allowing frame of video to play out, such as Fig. 4
Shown in:
If in the range of the time of the foremost of current buffer one frame is marked at Time ± △, then normal play；
If the time labelling of the foremost of current buffer one frame is less than Time-△, (frame is late) loses
Abandon this frame of video；
If the time labelling of the foremost of current buffer one frame is more than Time+ △, (frame comes early) weight
A former frame is put in replay, the most again compares, judges:
(1) if in the range of the time is marked at Time ± △, then normal play；
(2) if still greater than; would forward jitter-buffer to, find forward lower frame, until former frame weighs altogether
Broadcast 3 times, existing 3 frames of jitter-buffer, compare for the last time；If the time is marked at Time ± △ scope
In, then normal play；
Otherwise it is switched to video as standard, disposably adjusts audio frequency, the most again switch back to audio frequency
For standard；
Step 4. display buffer feeds back information to Video Decoder.
In step 1, following process done for some special circumstances:
If for first bag of this packet of the RTP bag that just arrived, first determine whether the frame type of oneself:
If not I frame, directly determine the packet positions of oneself according to the size of packet number, in comparison procedure
In, find there is no the packet number the same with oneself, then move on to first and open up Xin Kuai district；
If I frame, finding the block district oneself being grouped, be made whether that I frame judges, Ruo Gaikuai district has
Have I frame, then this bag abandons automatically；It is otherwise ordinary circumstance, normal process.
In step 2, display buffer and relief area initial value size of jumping the queue are moved according to video code model
State is adjustable, display buffer be one normal range size, i.e. can deposit 12 frame video frame size
Display queue, the upper limit is 15 frames, and lower limit is 3 frames, when less than 3 or more than 15 frame time, represent underflow
Or overflow；
Due to the stationary problem of display buffer audio frequency and video to be considered, audio frequency is broadcast continuously
Put, by Video processing being reached synchronization with voice data；
Specifically take to use based on the synchronous method playing the time limit, with reception to the voice medium received
The audio medium stream of end caching compensates the delay variation of network, is realized by the design of display buffer
Playback.
Above one embodiment of the present of invention is described in detail, but described content has been only the present invention's
Preferred embodiment, it is impossible to be considered the practical range for limiting the present invention.All according to the present patent application scope
Impartial change and the improvement etc. made, within all should still belonging to the patent covering scope of the present invention.
Claims (4)
1. the audio and video synchronization method of an embedded multimedia playing system, it is characterised in that: include as
Lower step:
Step 1. receives audio and video data streams, and decodes:
Start the thread receiving RTP audio frequency and video bag, receive RTP bag, afterwards it is resolved, resolve
Its data type carried, i.e. RTP audio pack or RTP video bag, if RTP audio pack enters audio frequency
Receive relief area, after be sent to audio decoder and be decoded；If RTP video bag enters buffering of queuing up
District, sequence before being first decoded RTP video bag processes, after be sent to Video Decoder and solve
Code；
Frame of video has just enter into the processing procedure in the queuing buffer:
RTP bag is directly entered queuing area decoder, proceeding by queuing decoding, first according to RTP video bag institute
The packet number carried, the most first finds the position of other bags of packet belonging to this packet number, then exists
In this packet, utilize self-contained serial number in this packet from big to small, start relatively to come one by one fast
Speed determines oneself position in this packet；
During RTP frame of video decodes:
When the i-th packet will decode, it is judged that whether i+1 is grouped complete I frame:
If so, it is decoded；
If it is not, this packet is moved to relief area, judge i-th+2 is grouped whether there is I frame afterwards:
If being decoded, when will decode, it may be judged whether playing the i-th+2 grouped data；If
It is to abandon i+1 packet, if otherwise starting i+1 packet decoding；
If no-trump i-th+2 packet moves on to relief area, after judge whether to receive the frame of video of i+1 packet:
If having, i+1 packet removal relief area is decoded；
If without proceeding to judge i-th+3 is grouped whether there is I frame:
If without abandoning the i+1 packet in relief area；
If being decoded, after judge whether to play the i-th+3 packet according to the feedback of display buffer:
If playing, then abandon i+1 packet and the i-th+2 packet；
If not playing, then return and judge whether i+1 packet has complete I frame, cycle criterion；
Step 2.
Decoded audio frame enters audio buffer；
Display buffer is entered for the most decoded frame of video, late for previous packet decoding
Previous packet video frame in a rear packet decoding enters relief area of jumping the queue；
Step 3. synchronizes to play:
Audio frame in audio buffer is play continuously；
Frame of video in display buffer carries out played in order, and the frame of video of relief area of jumping the queue carries out jumping the queue broadcasting
Put；
Wherein the reproduction time of frame of video is that to utilize NTP with RTP timestamp to calculate current video frame corresponding
Audio frame play time and obtain, it is judged that the time labelling of foremost one frame of current buffer with
Time ± △ relation, wherein Time ± △ represents and allows the time range that plays out of frame of video:
If in the range of the time of the foremost of current buffer one frame is marked at Time ± △, then normal play；
If the time labelling of the foremost of current buffer one frame is less than Time-△, then abandon this frame of video；
If the time labelling of the foremost of current buffer one frame is more than Time+ △, then before repeating playing once
One frame, compares the most again, judges:
(1) if in the range of the time is marked at Time ± △, then normal play；
(2) if still greater than; would forward jitter-buffer to, find forward lower frame, until former frame is replayed altogether
3 times, existing 3 frames of jitter-buffer, compare for the last time；If in the range of the time is marked at Time ± △,
Then normal play；
Otherwise it is switched to video as standard, disposably adjusts audio frequency, the most again switch back to audio frequency
For standard；
Step 4. display buffer feeds back information to Video Decoder.
The audio and video synchronization method of embedded multimedia playing system the most according to claim 1, its
It is characterised by: in step 1, if first bag of this packet of RTP bag for just having arrived, first
Judge the frame type of oneself:
If not I frame, directly determine the packet positions of oneself according to the size of packet number, in comparison procedure
In, find there is no the packet number the same with oneself, then move on to first and open up Xin Kuai district；
If I frame, finding the block district oneself being grouped, be made whether that I frame judges, Ruo Gaikuai district has
Have I frame, then this bag abandons automatically；It is otherwise ordinary circumstance, normal process.
The audio and video synchronization method of embedded multimedia playing system the most according to claim 1, its
Being characterised by: in step 2, the initial value size of display buffer and relief area of jumping the queue is according to video pressure
Contracting form carries out the most adjustable.
The audio and video synchronization method of embedded multimedia playing system the most according to claim 1, its
It is characterised by: in step 3, takes to use the audio frame received based on the synchronization side playing the time limit
Method, is compensated the delay variation of network, is realized by display buffer with the audio frame of receiving terminal caching
Playback.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310603239.3A CN103607664B (en) | 2013-11-22 | 2013-11-22 | A kind of audio and video synchronization method of embedded multimedia playing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310603239.3A CN103607664B (en) | 2013-11-22 | 2013-11-22 | A kind of audio and video synchronization method of embedded multimedia playing system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103607664A CN103607664A (en) | 2014-02-26 |
| CN103607664B true CN103607664B (en) | 2016-09-14 |
Family
ID=50125861
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310603239.3A Active CN103607664B (en) | 2013-11-22 | 2013-11-22 | A kind of audio and video synchronization method of embedded multimedia playing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103607664B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105376613B (en) | 2015-12-10 | 2019-05-10 | 华为技术有限公司 | A kind of fast channel switching method, server and IPTV system |
| CN107040748A (en) * | 2016-02-03 | 2017-08-11 | 北京机电工程研究所 | One kind monitoring and video conference application integration platform and method |
| CN105847862A (en) * | 2016-03-30 | 2016-08-10 | 乐视控股（北京）有限公司 | Cloud program directing station and cutoff continued broadcasting method thereof |
| CN106791271B (en) * | 2016-12-02 | 2019-08-13 | 福建星网智慧科技股份有限公司 | A kind of audio and video synchronization method |
| CN108495164B (en) * | 2018-04-09 | 2021-01-29 | 珠海全志科技股份有限公司 | Audio and video synchronization processing method and device, computer device and storage medium |
| CN110417715A (en) * | 2018-04-28 | 2019-11-05 | 北京视联动力国际信息技术有限公司 | A kind of framing method and device based on view networking |
| CN113014586B (en) * | 2021-03-04 | 2022-09-13 | 即时匹配(上海)网络科技有限公司 | RTP data packet out-of-order processing and framing method and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102665141A (en) * | 2012-05-16 | 2012-09-12 | 哈尔滨工业大学深圳研究生院 | AVS (audio video standard) audio and video presynchronizing method based on RTP (real time protocol) package |
| CN102932676A (en) * | 2012-11-14 | 2013-02-13 | 武汉烽火众智数字技术有限责任公司 | Self-adaptive bandwidth transmitting and playing method based on audio and video frequency synchronization |
-
2013
- 2013-11-22 CN CN201310603239.3A patent/CN103607664B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102665141A (en) * | 2012-05-16 | 2012-09-12 | 哈尔滨工业大学深圳研究生院 | AVS (audio video standard) audio and video presynchronizing method based on RTP (real time protocol) package |
| CN102932676A (en) * | 2012-11-14 | 2013-02-13 | 武汉烽火众智数字技术有限责任公司 | Self-adaptive bandwidth transmitting and playing method based on audio and video frequency synchronization |
Non-Patent Citations (3)
| Title |
|---|
| 基于RTP/RTCP协议的音视频同步算法;漆峰;《世界科技研究与发展》;20121031;第34卷(第5期);732-734 * |
| 基于嵌入式操作系统的流媒体的研究与应用;邓丽娜;《中国优秀硕士学位全文数据库》;20060815;42-48 * |
| 基于嵌入式流媒体播放系统的研究与实现;魏立强;《中国优秀硕士学位论文全文库》;20111215;33-42 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103607664A (en) | 2014-02-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103607664B (en) | A kind of audio and video synchronization method of embedded multimedia playing system | |
| CN103338386B (en) | Based on the audio and video synchronization method simplifying timestamp | |
| CN101271720B (en) | Synchronization process for mobile phone stream media audio and video | |
| KR101927145B1 (en) | Decoder and method at the decoder for synchronizing the rendering of contents received through different networks | |
| CN104618786B (en) | Audio and video synchronization method and device | |
| CN105247437B (en) | Synchronous multiple OTT streaming clients | |
| CN201781583U (en) | Multichannel server video playback synchronous control system | |
| FI116816B (en) | Streaming media | |
| CN102742249B (en) | Method, system and device for synchronization of media streams | |
| CN101030938B (en) | System and method for organizing and transmitting QoS self-adaptive P2P flow medium data | |
| CN102761776B (en) | Video and audio synchronizing method of P2PVoD (peer-to-peer video on demand) system based on SVC (scalable video coding) | |
| CN109906613A (en) | The multi-modal synchronization of audio and video renders | |
| CN101827271B (en) | Audio and video synchronized method and device as well as data receiving terminal | |
| CN103546662A (en) | Audio and video synchronizing method in network monitoring system | |
| WO2006006980A3 (en) | Maintaining synchronization of streaming audio and video using internet protocol | |
| TW200820777A (en) | System and method of audio/video streaming | |
| WO2007095478A3 (en) | Method and system for streaming digital video content to a client in a digital video network | |
| EP1919217A3 (en) | Apparatus and method for providing in a terminal a pause function for a broadcast stream | |
| JP6516767B2 (en) | MMTP decapsulation buffer signaling and operation | |
| CN113099310A (en) | Real-time media internal video and audio coordination method based on android platform | |
| CN103888813A (en) | Audio and video synchronization realization method and system | |
| CN101682562A (en) | Method and apparatus for modifying playback timing of talkspurts within a sentence without affecting intelligibility | |
| CN107438192A (en) | The synchronous method of audio and video playing and related system and multimedia play terminal | |
| CN102404650A (en) | Method for synchronously controlling audio and video of online video | |
| CN101202613B (en) | Terminal for clock synchronization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: An Audio Video Synchronization Method for Embedded Multimedia Playback System Effective date of registration: 20230921 Granted publication date: 20160914 Pledgee: Bank of Beijing Limited by Share Ltd. Tianjin branch Pledgor: TIANJIN BEIHAI COMMUNICATION TECHNOLOGY Co.,Ltd. Registration number: Y2023120000081 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |