CN106993197B - A kind of frame losing method based on encoder - Google Patents
A kind of frame losing method based on encoder Download PDFInfo
- Publication number
- CN106993197B CN106993197B CN201710228767.3A CN201710228767A CN106993197B CN 106993197 B CN106993197 B CN 106993197B CN 201710228767 A CN201710228767 A CN 201710228767A CN 106993197 B CN106993197 B CN 106993197B
- Authority
- CN
- China
- Prior art keywords
- frame
- video
- losing
- frame losing
- queue length
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 15
- 238000005516 engineering process Methods 0.000 abstract description 4
- 238000004891 communication Methods 0.000 abstract description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 210000005252 bulbus oculi Anatomy 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 210000001508 eye Anatomy 0.000 description 1
- 230000002045 lasting effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The present invention relates to field of communication technology more particularly to a kind of frame losing methods based on encoder, include the following steps: S1: judging whether the output frame rate of video source is greater than frame per second actually required, if it is, carrying out frame losing processing；S2: judge to encode the output frame rate whether frame per second is greater than video source, if it is, carrying out frame losing processing；S3: according to flow queue length and specified plug-flow queue length is pushed forward, frame losing processing is carried out.The beneficial effects of the present invention are: frame losing processing can be carried out respectively to video source, Video coding and audio/video pushing stage, influence of the reduction frame losing to audio and video playing quality as far as possible.
Description
Technical field
The present invention relates to field of communication technology more particularly to a kind of frame losing methods based on encoder.
Background technique
Net cast refers to be broadcast live using internet and stream media technology, and video is because having merged image, text, sound
Etc. abundant element, excellent both in sound and shape, excellent is increasingly becoming the mainstream expression way of internet.Video passes through true, lively
It propagates, builds strong presence, attract eyeball, reach communication effect with deep impression, that memory is lasting.
Traditional net cast is made of audio-source, video source, AAC encoder, H264 encoder and RTMP impeller.
Audio-source can come from DV or connect other microphones of equipment, primary to read a frame, is packaged into a packet, is sent to later
AAC encoder is encoded.Video source is other cameras from DV or connection equipment, and the image that capture camera takes is shown
On the screen, it once shows a frame, is packaged into a packet, is sent to H264 encoder later and is encoded.AAC encoder is to sound
Frequency is according to being compressed, to use the more data of less bandwidth for transmission.There is an audio to be processed inside AAC encoder
Frame queue, AAC encoder take out a frame from queue every time and are handled.The effect of H264 encoder is compression video, so as to
In the case where not losing clarity, with less bandwidth for transmission video data, H264 encoder takes from video frame queue every time
A frame is handled out.The effect of RTMP impeller is to send server for data such as video frame or audio frames, and RTMP is pushed away
There is a queue in the inside of stream device, and audio frame and video frame are all in this queue.
During net cast, due to audio, video data generate the too fast or encoder scramble time it is too long and by
The limitation of the computing resources such as CPU and GPU or network condition are undesirable, are required to carry out frame losing processing.Existing frame losing method master
Will be undesirable for network condition and make frame losing processing, broadly abandon whole B frame or P frame, this has resulted in receiving end view
Frequency quality is decreased obviously.
Summary of the invention
The shortcomings that for above scheme, the present invention propose a kind of frame losing method based on encoder, and this method can be use up can
Influence of the frame losing to audio and video playing quality can be reduced.
The technical scheme is that: a kind of frame losing method based on encoder includes the following steps:
S1: judging whether the output frame rate of video source is greater than frame per second actually required, if it is, carrying out frame losing processing；
S2: judge to encode the output frame rate whether frame per second is greater than video source, if it is, carrying out frame losing processing；
S3: according to flow queue length and specified plug-flow queue length is pushed forward, frame losing processing is carried out.
Preferably, in step S1, frame losing processing method particularly includes:
S101: in upper a period of time T, calculating the frame number C in video source, and calculating its frame per second is C/T；
S102: frame per second actually required is F, and calculating frame loss rate is C/T-F, and frame losing accounting is=1-F*T/C；
S103: in any time t in this period of time T, calculating video source frame number be c, then should frame losing number be c*
(1-F*T/C)；
S104: in upper a period of time T, the frame number for counting video source is Cin, and the frame number of output is Cout, then practical
Frame losing number is Cin-Cout；
S105: judging whether frame losing should be selected when next frame arrives in this period of time T, if c* (1-F*T/C) >
Cin-Cout, i.e. judgement should frame losing number it is whether bigger than practical frame losing number, then should frame losing, Cin numerical value adds 1, and Cout numerical value
It is constant；If c* (1-F*T/C)≤Cin-Cout, Cin numerical value adds 1, and Cout adds 1；
S106: continuing to repeat step S105, and until this end cycle, repetition starts next period.
Preferably, in step S2, frame losing processing method particularly includes: lose any of the removing trailing end in coding queue
One frame.
Preferably, in step S3, frame losing processing includes: to lose non-key video frame；Lose all video frames；Lose all videos and
Audio frame.
Preferably, in step S3, if the current plug-flow queue length L specified plug-flow queue length * 4/5 of >, loses all views
Frequency and audio frame, empty queue.
Preferably, in step S3, if the current plug-flow queue length L specified plug-flow queue length * 3/4 of >, loses all
Video frame.
Preferably, in step S3, if the current plug-flow queue length L specified plug-flow queue length * 2/3 of >, loses all
Non-key video frame.
Preferably, in step S3, if the current plug-flow queue length L specified plug-flow queue length * 1/2 of >, loses half
Non-key video frame.
The beneficial effects of the present invention are: can be lost respectively to video source, Video coding and audio/video pushing stage
Frame processing, as far as possible influence of the reduction frame losing to audio and video playing quality.
Detailed description of the invention
The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
Fig. 1 is the overall flow schematic diagram of the embodiment of the present invention.
Specific embodiment
Below in conjunction with attached drawing, technical scheme of the present invention will be further described, but the present invention is not limited to these realities
Apply example.
Fig. 1 is the flow diagram of the embodiment of the present invention, as shown in Figure 1, comprising the following steps:
S1: judging whether the output frame rate of video source is greater than frame per second actually required, if it is, carrying out frame losing processing.
Audio, video data generates too fast, wastes massive band width, but human eye and do not think more smooth.Such as: video source produces
The frame per second of raw video data is 50 frames, and the frame per second that we need is 25 frames, then needs frame losing.Frame losing strategy in video source
It is: calculates the accounting of the frame lost, carrys out row frame losing according to the queue of accounting.Specific steps are as follows:
S101: in upper a period of time T, calculating the frame number C in video source, and calculating its frame per second is C/T；
S102: frame per second actually required is F, and calculating frame loss rate is C/T-F, and frame losing accounting is=1-F*T/C；
S103: in any time t in this period of time T, calculating video source frame number be c, then should frame losing number be c*
(1-F*T/C)；
S104: in upper a period of time T, the frame number for counting video source is Cin, and the frame number of output is Cout, then practical
Frame losing number is Cin-Cout；
S105: judging whether frame losing should be selected when next frame arrives in this period of time T, if c* (1-F*T/C) >
Cin-Cout, i.e. judgement should frame losing number it is whether bigger than practical frame losing number, then should frame losing, Cin numerical value adds 1, and Cout numerical value
It is constant；If c* (1-F*T/C)≤Cin-Cout, Cin numerical value adds 1, and Cout adds 1；
S106: continuing to repeat step S105, and until this end cycle, repetition starts next period.
This method in cycle T frame loss rate and practical frame losing number constantly update.
S2: judge to encode the output frame rate whether frame per second is greater than video source, if it is, carrying out frame losing processing.
, can be because cpu resource be inadequate because it is more that encoder occupies cpu resource, coding frame per second does not catch up with video source
The frame per second of output, then need frame losing.It can also draw badly because of CPU, overheat causes code efficiency to reduce.When cpu performance reduces
When, in order to not influence the fluency of video, random frame losing strategy is taken to omit.Coding queue length is N, when queue size is N-1
When, random number a P, P are generated 1 between N-1, losing the frame on P position.
S3: according to flow queue length and specified plug-flow queue length is pushed forward, frame losing processing is carried out.
Impeller just frame losing later is arrived, cost is expensive, because the frame for reaching impeller all carried out losing several times
Frame is encoded, and cpu resource is occupied.But under the conditions of low network speed, the speed for sending audio-video is slower, also can only frame losing.
The video frame having after coding for reaching impeller, also there is the audio frame after coding.Audio frame is smaller, frame losing solution
Never how many problems, audio frame is also important, so losing audio frame is not first choice, first choice is to lose video frame.
Inside video frame, there are key frame and non-key frame, if losing key frame, will lead to colored screen.Frame losing at this time handles packet
It includes: losing non-key video frame；Lose all video frames；Lose all videos and audio frame.
If the current plug-flow queue length L specified plug-flow queue length * 4/5 of >, loses all videos and audio frame, empties team
Column.If the current plug-flow queue length L specified plug-flow queue length * 3/4 of >, loses all video frames.If current plug-flow queue
The specified plug-flow queue length * 2/3 of length L >, then lose all non-key video frames.If current plug-flow queue length L > is specified
Plug-flow queue length * 1/2 then loses the non-key video frame of half.
It carries out different frame losings from specified plug-flow queue length according to plug-flow queue length L to handle, reduction frame losing as far as possible
Influence to audio and video playing quality.
Specific embodiment described herein is only an example for the spirit of the invention.The neck of technology belonging to the present invention
The technical staff in domain can make various modifications or additions to the described embodiments or replace by a similar method
In generation, however, it does not deviate from the spirit of the invention or beyond the scope of the appended claims.
Claims (2)
1. a kind of frame losing method based on encoder, which comprises the steps of:
S1: judging whether the output frame rate of video source is greater than frame per second actually required before video source is not encoded, if it is,
Frame losing processing is carried out to video source, the specific steps are as follows:
S101: in upper a period of time T, calculating the frame number C in video source, and calculating its frame per second is C/T；
S102: frame per second actually required is F, and calculating frame loss rate is C/T-F, and frame losing accounting is=1-F*T/C；
S103: in any time t in this period of time T, calculating video source frame number is c, then frame losing number should be c* (1-F*T/
C)；
S104: in upper a period of time T, the frame number for counting video source is Cin, and the frame number of output is Cout, then practical frame losing
Number is Cin-Cout；
S105: judge whether frame losing should be selected when next frame arrives in this period of time T, if c* (1-F*T/C) > (Cin-
Cout), that is, judge that frame losing number is bigger than practical frame losing number, then should frame losing, Cin numerical value adds 1, and Cout numerical value is constant；If c*
(1-F*T/C)≤(Cin-Cout), then Cin numerical value adds 1, and Cout adds 1；
S106: continuing to repeat step S105, and until this end cycle, repetition starts next period；
S2: judging to encode the output frame rate whether frame per second is greater than video source in the stage for encoding video, if it is,
Frame losing processing is carried out to video；
S3: if sending audio-video under conditions of low network speed in the audio/video pushing stage, basis is needed to be pushed forward flow queue length
With specified plug-flow queue length, frame losing processing is carried out to the audio-video after coding, main frame losing mode, which has, loses non-key view
Frequency frame loses all video frames and loses all video frames and audio frame, specific frame losing method are as follows:
If the current plug-flow queue length L specified plug-flow queue length * 1/2 of >, loses the non-key video frame of half；
If the current plug-flow queue length L specified plug-flow queue length * 2/3 of >, loses all non-key video frames；
If the current plug-flow queue length L specified plug-flow queue length * 3/4 of >, loses all video frames；
If the current plug-flow queue length L specified plug-flow queue length * 4/5 of >, loses all videos and audio frame, empties queue.
2. according to the frame losing method described in claim 1 based on encoder, which is characterized in that in step S2, the tool of frame losing processing
Body method are as follows: lose any one frame of the removing two ends in coding queue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710228767.3A CN106993197B (en) | 2017-04-10 | 2017-04-10 | A kind of frame losing method based on encoder |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710228767.3A CN106993197B (en) | 2017-04-10 | 2017-04-10 | A kind of frame losing method based on encoder |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106993197A CN106993197A (en) | 2017-07-28 |
| CN106993197B true CN106993197B (en) | 2019-05-24 |
Family
ID=59414916
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710228767.3A Active CN106993197B (en) | 2017-04-10 | 2017-04-10 | A kind of frame losing method based on encoder |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106993197B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108600794B (en) * | 2018-05-21 | 2021-03-02 | 深圳市梦网科技发展有限公司 | Frame loss rate correction method, device and terminal |
| CN110177308A (en) * | 2019-04-15 | 2019-08-27 | 广州虎牙信息科技有限公司 | Mobile terminal and its audio-video frame losing method in record screen, computer storage medium |
| CN110166808B (en) * | 2019-06-13 | 2021-09-07 | 深圳市拔超科技有限公司 | Method and device for solving video asynchronism caused by crystal oscillator error and decoding equipment |
| CN111277836B (en) * | 2020-01-19 | 2021-10-22 | 苏州浪潮智能科技有限公司 | Video extraction frame loss control method, system, terminal and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101778426A (en) * | 2010-01-21 | 2010-07-14 | 深圳市同洲电子股份有限公司 | Method and equipment for video data stream transmission in mobile wireless network |
| CN101883240A (en) * | 2010-06-09 | 2010-11-10 | 广东威创视讯科技股份有限公司 | Video conference system based on MCU (Microprogrammed Control Unit) and video transmission lost packet processing method thereof |
| CN103152576A (en) * | 2013-03-21 | 2013-06-12 | 浙江宇视科技有限公司 | Video coding and decoding device suitable for multicast packet loss resistance |
| CN103475902A (en) * | 2013-09-06 | 2013-12-25 | 同观科技（深圳）有限公司 | Video coding and network transmission method and video forwarding server |
| CN104639978A (en) * | 2015-02-13 | 2015-05-20 | 上海小蚁科技有限公司 | Multimedia processing method and device |
| CN104702968A (en) * | 2015-02-17 | 2015-06-10 | 华为技术有限公司 | Frame loss method for video frame and video sending device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102074148B1 (en) * | 2011-08-30 | 2020-03-17 | 엔엘디 엘엘씨 | Systems and methods for encoding and streaming video encoded using a plurality of maximum bitrate levels |
-
2017
- 2017-04-10 CN CN201710228767.3A patent/CN106993197B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101778426A (en) * | 2010-01-21 | 2010-07-14 | 深圳市同洲电子股份有限公司 | Method and equipment for video data stream transmission in mobile wireless network |
| CN101883240A (en) * | 2010-06-09 | 2010-11-10 | 广东威创视讯科技股份有限公司 | Video conference system based on MCU (Microprogrammed Control Unit) and video transmission lost packet processing method thereof |
| CN103152576A (en) * | 2013-03-21 | 2013-06-12 | 浙江宇视科技有限公司 | Video coding and decoding device suitable for multicast packet loss resistance |
| CN103475902A (en) * | 2013-09-06 | 2013-12-25 | 同观科技（深圳）有限公司 | Video coding and network transmission method and video forwarding server |
| CN104639978A (en) * | 2015-02-13 | 2015-05-20 | 上海小蚁科技有限公司 | Multimedia processing method and device |
| CN104702968A (en) * | 2015-02-17 | 2015-06-10 | 华为技术有限公司 | Frame loss method for video frame and video sending device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106993197A (en) | 2017-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106993197B (en) | A kind of frame losing method based on encoder | |
| US11323136B2 (en) | Method and apparatus for processing a received sequence of data packets by removing unsuitable error correction packets from the sequence | |
| CN104096362B (en) | The Rate Control bit distribution of video flowing is improved based on player's region-of-interest | |
| CN110430441B (en) | Cloud mobile phone video acquisition method, system, device and storage medium | |
| CN106713947A (en) | Method and device for reducing live broadcasting time delay and standstill as well as live broadcasting system | |
| CN102905128B (en) | Codec processor is the method for Rate Control during wireless video transmission | |
| CN102547375A (en) | Transmission method for quickly previewing H.264 coded picture | |
| CN106791899A (en) | The live middle method and device and client for reducing viewer end picture interim card | |
| WO2023005140A1 (en) | Video data processing method, apparatus, device, and storage medium | |
| JP7265623B2 (en) | Content-adaptive quantization intensity and bitrate modeling | |
| CN106993190A (en) | Software-hardware synergism coding method and system | |
| CN106792133A (en) | Virtual reality server, method of transmitting video data and system | |
| CN103780908B (en) | Efficient H264 decoding method | |
| CN102223539A (en) | Processing method for splash screen caused by picture coding frame loss | |
| CN106792247A (en) | Reduce the live TV stream method for pushing and device, live broadcast system of time delay | |
| CN108471540B (en) | High-definition video smooth live broadcast method and device based on ultralow code stream | |
| CN106231319A (en) | A kind of method alignd frame by frame based on software and hardware combining | |
| CN111901678B (en) | Anti-jitter smoothing method and system for TCP real-time video stream | |
| CN103873873B (en) | A kind of image coding/decoding method under non-Reliability Channel | |
| WO2024078066A1 (en) | Video decoding method and apparatus, video encoding method and apparatus, storage medium, and device | |
| CN105263021B (en) | A kind of HEVC video encoding/decoding methods based on UVD | |
| CN103139528B (en) | The processing method of a kind of audio, video data and device | |
| CN110365997A (en) | A kind of the interactive teaching live broadcasting method and system of low latency | |
| CN106534889A (en) | Real-time streaming media multipath forwarding control method based on loop buffer | |
| CN102413333B (en) | Video compression coding/decoding system and method based on underdetermined blind signal separation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A frame loss method based on encoder Granted publication date: 20190524 Pledgee: Guotou Taikang Trust Co.,Ltd. Pledgor: HANGZHOU XINGXI TECHNOLOGY Co.,Ltd. Registration number: Y2024980020954 |