CN102231828A - Delay processing method for decoding, synchronizing and outputting image - Google Patents
Delay processing method for decoding, synchronizing and outputting image Download PDFInfo
- Publication number
- CN102231828A CN102231828A CN2011101728671A CN201110172867A CN102231828A CN 102231828 A CN102231828 A CN 102231828A CN 2011101728671 A CN2011101728671 A CN 2011101728671A CN 201110172867 A CN201110172867 A CN 201110172867A CN 102231828 A CN102231828 A CN 102231828A
- Authority
- CN
- China
- Prior art keywords
- time
- frame number
- transmitting terminal
- mark
- delay
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention provides a delay processing method for decoding, synchronizing and outputting an image, which comprises the following steps that: (S1) a buffer area is arranged at a decoder end to receive a remote end image data packet, and a high point marker and a low point marker of a data packet number are arranged in the buffer area; (S2) when the data packet number in the buffer area is located at a warning point of the low point marker, timing is started; when the data packet number reaches to a warning point of the high point marker, time length T and received frame quantity M are recorded; (S3) frame quantity N that are excessively sent out per second by a camera sending end is calculated in this period, wherein N=M/T; (S4) the frame quantity N excessively sent out per second by a feedback camera sending end is set to the camera sending end, after receiving the feedback N, the camera sending end actively reduces the sending frame rate. With the adoption of the method, image delay can be modified, the produced delay can be controlled in a receivable range, therefore, normality and work efficiency of video monitoring can be ensured.
Description
Technical field
The present invention relates to a kind of at the synthetic delay process method of exporting of multiplexer channel picture decoding that with video (television set, video wall) is the H.264 hardware video decoder of display terminal.
Background technology
Along with the fast development of Internet and multimedia technology, the network real-time Transmission of video becomes one of research focus of network application.H.264 standard becomes the main form and the standard of multimedia transmission on present and the next generation network with its high compression rate, high-quality, low code check.The time delay of the real-time Transmission requirement transmission of video is little, packet loss is low.
As based on H.264 video server, show to decoding terminals by network transmitted image packet in front end hardware coding back.Terminal can be based on the computer of client software or output on the video (television set, video wall) by after the Video Decoder hardware decoding.Yet these two kinds of terminals there are differences on image refreshing.It is real-time processing that computer client software is accepted behind the image data decoding, and promptly the decoder receiving terminal receives how many frames how many packets just can separate, and refreshes in real time.It then is that 40ms refreshes a frame that video decoder decodes outputs on the video (television set, video wall).Particularly when decoder carries out the picture decoding of multiplexer channel CIF form (video standard form commonly used), receiving the image data decoding that the video encoder end transmits, is that one-period refreshes a frame with 40ms.When decoder receives the image data decoding that far-end transmits, if the packet frame number that decoder decode receives is too fast, will cause in each cycle, the packet frame number is faster than every 40ms decoding and displaying cycle.In one-period at 40ms, when time-delay takes place in a certain road or multiway images, also can have influence on next cycle, when this time-delay is accumulated to one long period, will cause video (television set, video wall) image and image scene that obvious time-delay situation takes place, and this image time-delay is expendable.Significantly performance characteristic is: the image that TV, video wall show lags behind the camera actual monitored time, such as: 10 o'clock what also show is 9: 30 monitoring image.
And the display end standard of shooting head end and the decoding of monitoring client hardware is inconsistent, and just two kinds of standard whiles also can produce picture delay when a TV (video wall) is gone up demonstration.Camera and display end may be PAL or TSC-system formula, and wherein the standard of PAL is that per second shows 25 frames, and per second shows 30 frames during the standard of NTSC.Transmitting terminal when camera is a pal mode so, and when the Rcv decoder end was the TSC-system formula, the picture delay of being given birth to was particularly serious.After time-delay accumulation after a while, these delays will cause between video (television set, video wall) image and image scene very big time-delay being arranged, and this image time-delay is expendable, has lost the original intention and the effect of video monitoring, makes real-time monitoring become an empty verbiage.
Characteristics such as video (television set, video wall) is extensive, the most the most frequently used solution as the monitor terminal of output, and it is low to have a cost, and effect is directly perceived, and is easy to operate.Because the more uncertainty that the network real-time Transmission of video faces, the fluctuation ratio of Network Transmission quality is more common.And the nonsynchronous situation of the frame per second of transmitting terminal and receiving terminal exists always, can not be synchronously.Fig. 1 has shown very intuitively why the frame per second meeting is asynchronous.
So, when with video (television set, video wall) being the terminal demonstration, must consider that its 40ms is the characteristic that refreshes of one-period.If after the situation of time-delay takes place in certain passage in the one-period at 40ms, will there be always and has influence on the cycle of back in this time-delay, can cause between certain passage of image of video terminal (television set, video wall) and the image scene sizable time-delay being arranged after accumulation after a while, it is all undesired to cause image to show.Video Decoder in the case has very urgent requirement for the delay process of image data packets.Therefore, along with further developing of streaming media service, this class processing method just is being subjected to increasing attention.
Summary of the invention
Technical problem to be solved by this invention is: the delay process method that the synthetic output of a kind of picture decoding is provided.Adopt this method correction image to delay time, the time-delay that is produced is controlled within the acceptable scope, guarantee the normal and operating efficiency of real-time video monitoring.
The technical solution adopted in the present invention is: the delay process method of the synthetic output of picture decoding may further comprise the steps:
S1) in decoder end a buffering area is set and receives the far-end image data packets, the height point mark and the low spot mark of a number-of-packet is set in buffering area;
S2) number-of-packet is in the warning point of low spot mark in buffering area, picks up counting; When number-of-packet arrives the warning point of high some mark, record time span T and the frame number M that receives;
S3) in calculating during this period of time, the frame number N that camera transmitting terminal per second pilosity goes out, wherein N=M/T;
S4) the frame number N that goes out of feedback camera transmitting terminal per second pilosity gives the camera transmitting terminal, and the camera transmitting terminal initiatively reduces sending frame per second after receiving feedback N.
Further, step S2 specifically comprises following process:
When buffer data bag number is in the warning point of low spot mark, start timer, time this moment is T
0, and note down current frame number M
0When number-of-packet arrives the warning point of high some mark, stop timer, time this moment is T
1, and note down current frame number M
1Time span T=T
1-T
0, the frame number M=M that receives in during this period of time
1-M
0
Further, the low spot of step S1 is labeled as 10% buffer size, and high point is labeled as 90% buffer size.
Advantage of the present invention: this transmission feeds back to transmitting terminal, and carry out active at transmitting terminal and reduce the method that frame per second solves picture delay that sends, with traditional method of giving up a part of frame number at the decoder receiving terminal relatively the time, bigger advantage is arranged, because conventional method is to keep recently normal I frame, try every possible means to give up the P frame.Giving up a part of frame number and can cause more serious ghost image and Hua Ping phenomenon like this.We's rule is dealt with problems at the source end, minimizes ghost image and Hua Ping problem, and higher efficient is arranged.
Description of drawings
Fig. 1 is that decoder outputs to and produces decoding time-delay schematic diagram when video (television set) is gone up.
Fig. 2 is that buffering area is provided with schematic diagram.
Fig. 3 is the synthetic delay process method that outputs on the video (television set) of H.264 Video Decoder multiway images decoding provided by the invention.
Embodiment
The invention provides a kind of processing method of delaying time at the picture decoding that with video (television set, video wall) is the H.264 hardware video Code And Decode device of display terminal.
Among Fig. 1, what black numbers 1～10 was represented is that TV is forced the 40ms refresh cycle, and 1～10 expression is correctly decoded 1 frame of frame per second to 10 frames.Red arrow and numeral be to send frame per second, as seen when the transmission frame per second was too fast, the two ends frame per second was inconsistent, picture decoding will be undesired.At this moment, must feed back to transmitting terminal, initiatively reduce sending frame per second, make the two ends frame per second consistent as far as possible, it is normal that picture decoding is tending towards.
Blue arrow and numeral then expression be to send frame per second to cross when slow, the two ends frame per second is inconsistent, at this moment, picture decoding is undesired equally.At this time feed back to transmitting terminal, initiatively accelerate to send frame per second, make the two ends frame per second consistent as far as possible, it is normal that picture decoding is tending towards.
Further set forth the present invention below in conjunction with accompanying drawing 2,3.
Step 1, decoder end are provided with a buffering area and receive the far-end image data packets, and one high some mark (highwater) and low spot mark (lowwater) are set in buffering area.Packet is in lowwater warning point in buffering area, and software picks up counting.When packet arrives highwater warning point, record time span T and the frame number M that receives.Promptly when the buffer data bag is in lowwater warning point, start timer, time this moment is T0, and notes down current frame number M0.When packet arrives highwater warning point, stop timer, time this moment is T1, and notes down current frame number M1.Time span T=T1-T0.M=M1-M0 notes down the frame number for receiving in during this period of time.
Step 2, calculating M/T=N.In N is during this period of time, the frame number that camera transmitting terminal per second pilosity goes out.
Step 3, feedback N give the camera transmitting terminal, and the camera transmitting terminal is after receiving feedback N, and the active reduction sends frame per second, initiatively reduce the transmission frame number and are N.So just at the source end, 2 kinds of situations that the described TV of background technology, video wall display image produce delay issue have fundamentally been solved.At this moment, buffer data is tending towards normally, and it is normal that image is tending towards.Feedback N in the step 3 is after the camera transmitting terminal receives, and M in the step 2 and T can reset, so that proofread and correct next time.
Step 3 specifically may further comprise the steps:
3.1, T and M reset in each feedback back;
Calculate M/T=N.In N is during this period of time, the frame number that camera transmitting terminal per second pilosity goes out.
3.2, feedback N gives the camera transmitting terminal.After the camera transmitting terminal receives feedback, when coding proceeds to YUV, utilize the data of feedback N, carry out reduction initiatively and send frame per second.Normal and the TV of the packet frame number change that sends to the decoder receiving terminal, the 40ms refresh cycle of video wall are matched.Fundamentally solve the picture delay problem.
This transmission feeds back to transmitting terminal, and carry out active at transmitting terminal and reduce the method that frame per second solves picture delay that sends, with traditional method of giving up a part of frame number at the decoder receiving terminal relatively the time, bigger advantage is arranged, because it is to keep recently normal I frame that tradition is given up a part of frame number method, try every possible means to give up the P frame.Giving up a part of frame number and can cause more serious ghost image and Hua Ping phenomenon like this.We's rule is dealt with problems at the source end, minimizes ghost image and Hua Ping problem, and higher efficient is arranged.
Claims (3)
1. the delay process method of the synthetic output of picture decoding is characterized in that, may further comprise the steps:
S1) in decoder end a buffering area is set and receives the far-end image data packets, the height point mark and the low spot mark of a number-of-packet is set in buffering area;
S2) number-of-packet is in the warning point of low spot mark in buffering area, picks up counting; When number-of-packet arrives the warning point of high some mark, record time span T and the frame number M that receives;
S3) in calculating during this period of time, the frame number N that camera transmitting terminal per second pilosity goes out, wherein N=M/T;
S4) the frame number N that goes out of feedback camera transmitting terminal per second pilosity gives the camera transmitting terminal, and the camera transmitting terminal initiatively reduces sending frame per second after receiving feedback N.
2. method according to claim 1 is characterized in that, step S2 specifically comprises following process:
When buffer data bag number is in the warning point of low spot mark, start timer, time this moment is T
0, and note down current frame number M0; When number-of-packet arrives the warning point of high some mark, stop timer, time this moment is T
1, and note down current frame number M
1Time span T=T
1-T
0, the frame number M=M that receives in during this period of time
1-M
0
3. method according to claim 1 and 2 is characterized in that: the low spot of step S1 is labeled as 10% buffer size, and high point is labeled as 90% buffer size.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110172867 CN102231828B (en) | 2011-06-24 | 2011-06-24 | Delay processing method for decoding, synchronizing and outputting image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110172867 CN102231828B (en) | 2011-06-24 | 2011-06-24 | Delay processing method for decoding, synchronizing and outputting image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102231828A true CN102231828A (en) | 2011-11-02 |
| CN102231828B CN102231828B (en) | 2013-05-15 |
Family
ID=44844355
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201110172867 Active CN102231828B (en) | 2011-06-24 | 2011-06-24 | Delay processing method for decoding, synchronizing and outputting image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102231828B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190164518A1 (en) * | 2017-11-28 | 2019-05-30 | Nvidia Corporation | Dynamic jitter and latency-tolerant rendering |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050033879A1 (en) * | 2001-11-22 | 2005-02-10 | Hwang In Seong | Method for providing a video data streaming service |
| CN1588909A (en) * | 2004-07-15 | 2005-03-02 | 复旦大学 | Method for controlling group broadcast transmission rate of network digital TV programme |
| CN1983907A (en) * | 2005-12-13 | 2007-06-20 | 中兴通讯股份有限公司 | Method for controlling flow media transmitting rate |
| CN101272486A (en) * | 2008-04-10 | 2008-09-24 | 清华大学 | Video transmission control method based on PID control and received frame rate stable model |
| CN101557512A (en) * | 2009-05-19 | 2009-10-14 | 武汉长江通信产业集团股份有限公司 | Method for processing time delay when video terminal receives video data |
-
2011
- 2011-06-24 CN CN 201110172867 patent/CN102231828B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050033879A1 (en) * | 2001-11-22 | 2005-02-10 | Hwang In Seong | Method for providing a video data streaming service |
| CN1588909A (en) * | 2004-07-15 | 2005-03-02 | 复旦大学 | Method for controlling group broadcast transmission rate of network digital TV programme |
| CN1983907A (en) * | 2005-12-13 | 2007-06-20 | 中兴通讯股份有限公司 | Method for controlling flow media transmitting rate |
| CN101272486A (en) * | 2008-04-10 | 2008-09-24 | 清华大学 | Video transmission control method based on PID control and received frame rate stable model |
| CN101557512A (en) * | 2009-05-19 | 2009-10-14 | 武汉长江通信产业集团股份有限公司 | Method for processing time delay when video terminal receives video data |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190164518A1 (en) * | 2017-11-28 | 2019-05-30 | Nvidia Corporation | Dynamic jitter and latency-tolerant rendering |
| US10741143B2 (en) * | 2017-11-28 | 2020-08-11 | Nvidia Corporation | Dynamic jitter and latency-tolerant rendering |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102231828B (en) | 2013-05-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9456209B2 (en) | Method of multiplexing H.264 elementary streams without timing information coded | |
| JP5788101B2 (en) | Network streaming of media data | |
| CN101505316B (en) | Method and device for reordering and multiplexing multimedia packets from multimedia streams pertaining to interrelated sessions | |
| US20150101003A1 (en) | Data transmission apparatus, system and method | |
| CN101557512B (en) | Method for processing time delay when video terminal receives video data | |
| CN102014286B (en) | Video coding and decoding method and device | |
| US20130263201A1 (en) | Data transmission apparatus, system and method | |
| US10862940B1 (en) | Low latency live video on a communication session | |
| US9699229B2 (en) | Robust live operation of dash | |
| EP2498495A1 (en) | Decoder and method at the decoder for synchronizing the rendering of contents received through different networks | |
| US20170034589A1 (en) | Adaptive profile switching system and method for media streaming over ip networks | |
| CN101605252B (en) | Method and system for converting program stream into transport stream | |
| JP5896596B2 (en) | Method and apparatus for multiplexing video elementary streams without coded timing information | |
| CN102118270A (en) | Method and device for measuring user QoE (Quality of Experience) | |
| CN101951387A (en) | Method and device for transmitting stream media | |
| TWI423674B (en) | System and method for reducing the zapping time | |
| US9049481B2 (en) | Fine-tuning the time for leaving/joining a multicast session during channel changes | |
| CN102223539A (en) | Processing method for splash screen caused by picture coding frame loss | |
| WO2012116558A1 (en) | Method and device for evaluating video quality | |
| CN115119009B (en) | Video alignment method, video encoding device and storage medium | |
| CN104270594A (en) | Data packet sending and receiving method and device | |
| WO2009103351A1 (en) | Method and apparatus for obtaining media over a communications network | |
| CN102231828B (en) | Delay processing method for decoding, synchronizing and outputting image | |
| WO2016090182A1 (en) | Location agnostic media control room and broadcasting facility | |
| CN104158804A (en) | Multi-screen interaction method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20111102 Assignee: WUHAN YANGTZE COMMUNICATIONS ZHILIAN TECHNOLOGY CO., LTD. Assignor: Wuhan Yangtze Communications Industry Group Co., Ltd. Contract record no.: 2014420000177 Denomination of invention: Delay processing method for decoding, synchronizing and outputting image Granted publication date: 20130515 License type: Exclusive License Record date: 20140926 |
|
| LICC | Enforcement, change and cancellation of record of contracts on the licence for exploitation of a patent or utility model |