CN103702119B - Code rate control method based on variable frame rate in low delay video coding - Google Patents

Abstract

The invention provides a code rate control method based on a variable frame rate in low delay video coding. The code rate control method comprises the following steps: adjusting the current coding frame rate FC through the current quantization parameter Q: when the value of the quantization parameter Q is in a low section range, the current coding frame rate FC is increased in a range of being not higher than the upper limit of frame rate; when the value of the quantization parameter Q is in a high section range, the current coding frame rate FC is decreased in a range of being not lower than the lower limit of frame rate; when the value of the quantization parameter Q is in a middle section range, the current coding frame rate FC is invariable. According to the code rate control method, active frame rate adjustment is adopted, and the code rate control during the transmission process can be ensured so that the value of the quantization parameter is adjusted in the middle section range, thus the visual quality keeps smooth and excellent.

Description

Bit rate control method of the low latency Video coding based on variable frame rate

Technical field

The present invention relates to video image compression coding technology.

Background technology

Rate Control is functional component important in video encoder.Low latency (the Low- in Video coding Delay) refer in particular to during inter prediction encoding only comprising infra-frame prediction I- frames and single directional prediction P- frame, do not use bi-directional predicted B- frame structures are not adopted to encode.Low latency coding is the sequential encoding for quickly carrying out, and feature is coded sequence and playing sequence Unanimously.Modal low latency is encoded to " IPP.. " or " IPP..IPP.. " structure.

The adjustment of coding bit rate is realized by controlling coding input quantization parameter Q, quantization parameter Q is a non-negative Integer.In MPEG-1, MPEG-2, MPEG-4, H.261, H.263, it rationally takes in the encoder such as WMV1, WMV2, RV10, RV20 Value scope in [2,31], H.264/AVC, H.264/SVC, in the encoder such as HEVC its reasonable value scope in [0,51].

Frame per second (F) is the quantity of the frame of broadcasting per second, unit frame (fps) per second.Limited by image sampling, information source frame per second (F_S) given before Video coding.Actual coding frame per second (F_C) fixed value is usually, it is standing to put and F_SIt is consistent, but Can also be adjusted being set to and F_SUnequal, this is thus referred to as variable frame rate regulation.

The content of the invention

The technical problem to be solved be to provide it is a kind of can reasonable adjusting quantization parameter Q based on variable frame rate Bit rate control method.

The present invention for solve above-mentioned technical problem institute just employing technical scheme be, in low latency Video coding be based on it is variable The bit rate control method of frame per second, including：

Present encoding frame per second F is adjusted by current quantization parameter Q_C：When the value of quantization parameter Q is in low section of interval, Increase present encoding frame per second F in not higher than frame per second upper range_C；When the value of quantization parameter Q is interval in high section, it is being not less than Present encoding frame per second F is reduced in frame per second lower range_C；When the value of quantization parameter Q is interval in stage casing, present encoding frame per second is kept F_CIt is constant.

In Rate Control, output code flow data are affected substantially by quantization parameter Q-value, but while are affected by picture material It is very big.Scene texture is complicated, the video source that motion change is violent will consume more bits.In order to balance code consumes bit Few and coding consumes the extremely many video sources of bit, and the frame rate adjustment taken the initiative both can ensure that the code check control in transmitting procedure System.When quantization parameter Q-value frequently falls in low section of interval, suitably increase frame per second, will directly reduce frame coding output bit number, It is interval to stage casing that Rate Control will adjust follow-up QP values；When quantization parameter Q-value frequently falls when high section is interval, suitably reduce frame Rate, will directly increase frame coding output bit number, it is also possible to so that follow-up QP values are adjusted to stage casing interval, it is ensured that visual quality is protected Maintain an equal level sliding and excellent.

Specifically, by frame per second changed factorTo adjust present encoding frame per second F_C,Wherein ← represent to The parameter assignment of the direction of arrow, F_SIt is the sampling frame per second obtained from video source；

Increase present encoding frame per second F_CForReduce present encoding frame per second F_CFor

Frame per second lower limit be 10Hz, frame per second upper limit 100Hz.

Specifically, current quantization parameter Q is expressed as used in the nearest 1 second coding till present encoding moment t The meansigma methodss of quantization parameter

Further, the present invention can be to control code check by adjusting quantization parameter, and the inventive method also includes,

In current coding moment t, when such as current volume frame is I- frames, using quantization parameter Q：

In current coding moment t, when such as current volume frame is P- frames, using quantization parameter Q：

Wherein, Q_tFor quantization parameter Q, Q that current coding moment t is used_t-1For the amount that upper one coding moment t-1 is used Change parameter Q, R_t-1Represent the frame coding output bit of a coding moment t-1, C_tFor the coded image of current coding moment t Texture complication, C_t-1For the coded image Texture complication of upper one coding moment t-1；Δ R needs amendment for coding moment t The buffer pool size of current coding moment t, B (t-1) is the buffer pool size of upper coding moment t-1, is α, and β is referred to as controlled Device parameter, μ be prevent and kill off 0 empirical parameter；

Distortion rate model is lnR=alnQ+blnC+c when current volume frame is I- frames, and Q represents quantization parameter, and R represents frame Coding output bit, C presentation code image texture complexities, a, b, c be distortion rate model parameter, distortion rate model parameter a, b Value by the way that how far linear regression is updated；Current to compile distortion rate model when frame is P- frames be lnR=alnQ+c, distortion The value of rate model parameter a is updated by one-variable linear regression；

Coding frame per second F in upper one coding moment_CAfter change, need according to new coding frame per second F_CRedefine pixel Point target bit Tbpp,So as to buffer pool size B (t) at present encoding moment, B (t)=B (t-1)+ R_t-1- Tbpp, R_t-1A coding moment t-1 time frame coding output bit is represented, TBR is target bit rate, and W is image pixel Width, is the high H of image pixel.

By the rate-distortion model set up, it was found that quantization parameter, frame coding output bit, coding in encoding video pictures The linear relationship for existing two-by-two between image texture complexity, gives a kind of new method of regulation quantization parameter.Also, it is The regulation of quantization parameter is reasonably carried out, adjusting strength Ipt (t) is introduced, when the quantization parameter value of continuous some frames codings The frequency and intensity of change are bigger, then will reduce adjusting strength Ipt (t)；The quantization parameter value changes of continuous some frame codings Frequency and intensity it is less, then will increase adjusting strength Ipt (t).

Further, the present invention can be with while by adjusting quantization parameter to control code check, by adjusting image Organize GOP to control code check, comprise the following steps：

1）Calculate the grey level histogram of present frame；

2）The index of similarity of two consecutive frames is calculated by the grey level histogram of present frame and the grey level histogram of previous frame； The index of similarity is represented by cosine similarity：

Wherein, cos (θ_t-1,t) it is the histogrammic cosine similarity of two consecutive frames, t represents present encoding moment, H_t[i] table Show the pixel sum that the video frame image grey scale pixel value in coding moment t is i, H_t-1[i] to be represented and encode moment t- upper one 1 video frame image grey scale pixel value is total for the pixel of i, and the scope of video frame image grey scale pixel value is 1 to n, and n is ash Total element number when degree rectangular histogram one-dimension array is represented；

3）When the index of similarity of two consecutive frames is less than threshold value, then it represents that occurrence scene switches, into step 4）；Otherwise set Put present frame type and be set to P- frames, extract return to step 1 after next frame data）；

4）Whether frame per second cycle value is reached with the P- frames for encoding in statistics present image group GOP, in this way, then into step 5）, present frame type is otherwise set and is set to P- frames, extract return to step 1 after next frame data）；

5）Present frame type is set and is set to I- frames, start a new GOP, extract return to step after next frame data 1）.

So, scene change detection and variable GOP length be combined with each other, and will cause for long video to be encoded Source, encode output result present GOP length it is different the characteristics of.The starting of each GOP correspond to a new scene.

Further, linearly dependent coefficient is introduced, comes together to characterize index of similarity with cosine similarity；

Wherein, r_t-1,tLinearly dependent coefficient is crossed for two consecutive frames are histogrammic,It is the video image in coding moment t Frame grey scale pixel value average,For in the video frame image grey scale pixel value average of coding moment t-1.

The invention has the beneficial effects as follows, the frame rate adjustment taken the initiative ensures that visual quality keeps smooth and excellent, enters one Step, code check can also be controlled by adjusting present encoding frame per second, adjustment image sets GOP simultaneously.

Description of the drawings

Fig. 1 is position of the integrated bit rate controller of embodiment in whole video coding system.

Fig. 2 is the integrated bit rate controller fundamental diagram of embodiment.

Fig. 3 is that embodiment scene change detection adjusts flow chart with dynamic GOP.

Fig. 4 is the regulation flow process figure of embodiment variable frame rate.

Fig. 5 is the control flow chart of embodiment variable bit rate.

Fig. 6 is test result relief area of the embodiment in different sequence groups, similarity and scene change detection state diagram.

Fig. 7 is that embodiment controls the state diagram after Deadline sequential codings with full I- frames variable bit rate.

Specific embodiment

Rate Control is completed by encoding video pictures bit rate controller in the present embodiment, specifically comprising three portions The function of dividing：Variable bit rate control function, scene change detection and dynamic GOP adjustment functions, the regulatory function of variable frame rate, Variable bit rate control, the regulatory function of variable frame rate and scene change detection and dynamic GOP adjustment functions, the tune of variable frame rate Section function can be carried out simultaneously, except a next time has been affected it by the regulation result of variable frame rate on variable bit rate control Outward, these three functions are substantially independent of one another.

First, variable bit rate control

The resolution of video image is i.e. given before coding starts, with pixel width (W) and the product form institute of high (H) Represent.Due to the presence that may have various sample modes in video image source sampling, two resolution on its colourity direction can There can be different sizes.Without loss of generality, the true resolution of the image is represented with the resolution of pixel gray level component.

For target bit rate (TBR) per bit (bps) per second is set when coding starts, in an encoding process Any time can be written over.After TBR initial setting up, situation about not being written over is referred to as constant bit rate (CBR), is written over The situation of once or more is referred to as variable bit rate (VBR).In the case where not causing and obscuring, TBR is considered as can be with The |input paramete of variation, when TBR values keep in an encoding process constant, then it is assumed that be CBR patterns, when TBR values are in coding During change (even if only once), then it is assumed that be VBR patterns.Frame per second (F) is the quantity of the frame of broadcasting per second, unit Frame (fps) per second.Limited by image sampling, information source frame per second (F_S) given before Video coding.Actual coding frame per second (F_C) logical Often it is initially set to and F_SIt is consistent, but it is also possible to be arranged to and F_SIt is unequal.Video frequency coding rate can uniformly by Enter row index per pixel bit (bpp) to be unified, then, target bit rate specification can be turned to pixel target ratio Special (Tbpp), is calculated by formula (1) and is obtained.

Wherein, the value of W and H is fixation, TBR and F_CIn an encoding process probable value is changed, when coding frame per second F_CAdjusted After whole, Tbpp needs to update.

Coding input quantization parameter (QP) is represented with Q, is a nonnegative integer.MPEG-1, MPEG-2, MPEG-4, H.261, H.263, in the encoder such as WMV1, WMV2, RV10, RV20 its reasonable value scope in [2,31], H.264/AVC, H.264/SVC, in the encoder such as HEVC its reasonable value scope in [0,51].

Image frame level Texture complication represents that the details for characterizing picture material itself enriches degree, with pixel gray level with C The average of gradient is portraying.As shown in formula (2), wherein l_i,jThe pixel gray value of (i, j) is designated as under expression.For image The calculating of frame level Texture complication is not limited to formula (2), and being currently used for calculating image frame level Texture complication algorithm can fit With.

The distortion of decoding video is represented with D, to recover the pixel average variance (MSE) of image after original image and coding To portray.The calculating of common picture engraving distortion factor value peak signal ratio (PSNR) is exactly to be converted to get by D, such as formula (3) shown in.

Wherein, digit when K represents pixel gray value binary representation.For example, 1 is represented with 1 byte (8) When pixel grey scale is 256 color, the value of K is 8；When representing a pixel gray value with 10-bit, the value of K is 10.

Frame coding output bit is represented with R, shown in the rate-distortion model such as formula (4) of foundation, it is also possible to by formula (4) weight It is written as such as formula (5).Implicit relation is three variable lnR, lnQ in encoding video pictures, there is binary primary line between lnC Sexual intercourse.

lnR=a·lnQ+b·lnC+c (4)

R=Q^a·C^b·e^c (5)

Binary once linear relationship (4) will be directly used in Rate Control, and a, b, c are three model parameters, and Q and C is from change Amount, R is dependent variable.Can all cause a, b, c for different information sources, using the different coding structures of different encoders, configuration The value of three model parameters is differed.But, it is above-mentioned for the information source for determining, the encoder of determination, the coding structure for determining The value of three model parameters is just highly stable.What binary linearity relation (5) was represented is the relation between distortion and bit, directly Ground is seen, monotonic functional relationship is presented between the bit number of input and distortion.

To a in I- frames coding rate-distortion model (4), the value of tri- parameters of b, c introduces multiple linear regression and is updated. Shown in the input matrix of construction such as formula (6).In coding moment t, Q, C and the R of the s frame data of nearest continuous I- frames before collection Construct the matrix of the row of s rows 3.Output result in formula (6) after one I- frames coding of data correspondence of the matrix per a line.

When P- frame Rate Controls are carried out, parameter b is forced to be set as 0, not consider shadow of the Texture complication to coding Ring.Now only have a, two parameters of c are retained, introduce one-variable linear regression and be updated.The input matrix of construction such as formula (7) It is shown.Similarly, in coding moment t, the Q and R of the s frame data of nearest continuous P-frame constructs the square of the row of s rows 2 before collection Battle array.Output result in formula (7) after one P- frames coding of data correspondence of the matrix per a line.

Linear regression is modal statistics and analysis instrument, it is easy to obtain its realization.Unitary used by the present invention and Multiple linear regression is fitted using least square approximation.Certainly, it is likely to be fitted with method for distinguishing, such as minimum absolute Error recurrence etc..Input matrix line number s shown in formula (6) and (7) is the sample number of multiple linear regression, it is also possible to claimed For window size.In the present invention, the value of s is the reasonable maximum between minimum 5, maximum 30.

Illusion reference decoder relief area (abbreviation relief area afterwards) is set, with B (t) current t buffer pool size is represented, it is single Position is every pixel bit (bpp).Its initial value and desired value are all fixedly installed as 0, i.e. B (0)=0.It is encoded in each frame Afterwards, buffer pool size will be updated, such as shown in formula (8).

B(t)=B(t-1)+R_t-1-Tbpp (8)

For formula (8), as B (t)>When 0, expression excessively uses bit；As B (t)<When 0, represent not enough using bit. The target of Rate Control is exactly to make the close desired value 0 of B (t) values as much as possible.The bit of frame level coding distributes in specified Tbpp On the basis of must account for the feedback of B (t).For the change of quick response deviation, amount to be regulated is made within the most short time Target is reached, the present invention corrects B (t) from PD control device.Shown in makeover process such as formula (9).

Wherein,Represent B (t) be corrected after result, parameter alpha, β is referred to as controller parameter its value can rely on Jing Test value selection.The present invention recommend value be α=0.45, β=0.55.

For the regulation for reasonably carrying out QP value changes in Rate Control, the factor of adjusting strength (Ipt) is introduced, based on this The consideration of sample：The frequency and intensity of continuous some frame coding QP value changes are bigger, then will reduce the intensity for adjusting；It is continuous some The frequency and intensity of frame coding QP value changes are less, then the intensity for adjusting increase.In present encoding moment t, 1 frame is calculated The absolute difference of consecutive frame coding QP values is with the calculating such as formula (10) of Ipt (t) is shown in the rate cycle.

So, need the bit increment Δ R of amendment to be calculated by formula (11) at coding moment t to obtain.

Wherein, μ is empirical parameter, and its value is generally fallen in interval [0.5,1.0], and value is more little, and the intensity for adjusting is more Greatly, value is more big, and the intensity for adjusting is less.The μ values that the present invention recommends are golden section point 0.618.

For the adjustment of existing measure, typically directly adoptOr the form of Δ R=B (t), Δ R The frequency of adjustment measure has been reacted, it is excessively frequent for the adjustment of measure when Δ R=B (t), and When, can not be adjusted the long period again, Ipt (t) amendments are introduced in the present inventionSo that encoding efficiency is more preferably, fluctuation is more It is little.

Total differential derivation, such as formula (12) are carried out to lnR=alnQ+blnC+c parts in rate-distortion model formula (4) It is shown.

In current coding moment t, Δ Q=Q is defined_t-Q_t-1、ΔC=C_t-C_t-1, then calculate the quantization parameter of current I- frames Q, relying on formula (13) is carried out.For P- frames coding b is forced to be set as 0, then quantization parameter Q relies on formula (14) and carries out.

The bit increment Δ R introduced in formula (13) and formula (14) is calculated by formula (11) and obtained.

2nd, scene change detection and dynamic GOP methods of adjustment

Video image is frequently run onto the situation of scene switching in broadcasting, and the generally content of switching and the frequency of switching is all It is irregular.In terms of encoding and decoding, due to extensive application of the inter prediction mode in Video coding, subsequent frame will be referred to Decoding frame in the early time, then, in the moment of scene switching, current encoded frame will be unable to directly obtain from decoding frame in the early time effectively Refer to image (block).From from the perspective of application, scene should be just when switching image carry out rationally being grouped again when Wait.The unnecessary distortion (mosaic phenomenon) that the independent packet of each image sets can both cause packet loss in data transmission procedure Blocking is in a GOP, it is also possible to provide the user the playing function for supporting random selection time point.In a word, switch in scene Shi Jinhang new GOP initialization, all has a clear superiority in terms of video objective visual quality and service quality is improved.

The present invention proposes two kinds of scene change detection computational methods.Both approaches are dependent on the intensity histogram of video image Figure statistics, both can individually implement, it is also possible to Joint Implementation.

The grey level histogram of image is represented with one-dimension array H, K represents position during pixel gray value binary representation Number, then total element number of array H is n=2^K, such as one byte (8bit) of common use represents a grey scale pixel value When, total element number of H is n=256.Define H_t[i] represents that in coding moment t video frame image grey scale pixel value be i Pixel sum.The relation such as public affairs of two continuous frames image grey level histogram are represented with karr Pearson linearly dependent coefficient Shown in formula (15).

In formula (16), correlation coefficient r_t-1,tSpan is [- 1,1], and on the occasion of positive correlation is represented, negative value represents negative Close.To r_t-1,tCarry out square, thenValue can be fallen in [0,1] scope,Closer to 1 rectangular histogram for representing two continuous frames Dependency is stronger；Represent that the rectangular histogram dependency of two continuous frames is weaker closer to 0.Generally, it is considered thatWhen value is more than 0.8, Linear dependence is notable.The calculating time complexity of formula (16) is O (n²)。

The dependency of two high dimension vectors, referred to as cosine similarity can be described with high dimension vector included angle cosine value.Pass through The cosine value of the angle in two inner product of vectors spaces of measurement is measuring the similarity between them.Formula (16) is given with Nogata Figure statistical value is the cosine similarity computational methods of high dimension vector.

In formula (17), due in statistics of histogram, H_t[i] value is always non-negative, then H_t-1And H_tVector it is total It is the first quartile fallen in higher dimensional space, then their angle theta_t-1,tFall between 0 ° and 90 °.Therefore, cosine similarity cos (θ_t-1,t) value be 0 to 1 between.θ_t-1,tValue more meets 0 ° of so cos (θ_t-1,t) closer to 1, this represents that two vector correlations are got over By force, it is otherwise then weaker.Similar with the computation complexity of formula (16), the time complexity of formula (17) is O (n²)。

Define Sim_tShown in similarity between adjacent two frames of presentation code moment t, such as formula (17).

Judge that the detection method that scene switches is to work as Sim_t>=ξ represents non-occurrence scene switching；Work as Sim_t<ξ represents scene Switching occurs.Here ξ is an empirical value, there is shown the sensitivity to scene detection.The value of ξ is too big (such as 0.95), So testing result is it is possible that flase drop；The value too little (such as 0.5) of ξ, then testing result is it is possible that missing inspection.This The value of invention suggestion is 0.85.

If scene change detection to occur, immediately present frame type is set to into I- frames, by GOP with regard to encoded P- The enumerator of frame sets to 0, and relevant environment makees the initialization of GOP, that is, restart the coding of a GOP.In the present invention, scene is cut Change detection and variable GOP length be combined with each other, the result for long video source to be encoded being encoded to output will be caused to present The characteristics of GOP length is different.The starting of each GOP correspond to a new scene.

3rd, the control method of variable frame rate

The frame per second (F) of Video coding is a scalar-unit, represents the quantity of the frame for showing within the unit interval or refreshing, Conventional unit is display frame number (frames per second, fps or Hz) per second.Frame per second is limited by sampling, and common takes Value has film：24fps, TV (PAL)：25fps, TV (NTSC)：29.97fps, CRT monitor：60Hz-85Hz, liquid crystal Show device：60Hz, 3D display：120Hz.From for video compress source, 20fps, 24fps, 30fps, 50fps are generally comprised, Several typical frame per second types such as 60fps.

From sampling frame per second F that video source is obtained_SIt is usually fixed, may has if changing frame per second in encoding-decoding process Two methods.First, newly-generated some frames, expand frame per second.Motion compensation class frame per second is lifted and relies on large-scale computing, and interpolation is calculated Method class frame per second is lifted will cause moving object edge blurry or still frame jitter phenomenon.Second, actively give up some frames, i.e., it is right Selected frame is not encoded.

In Rate Control, output code flow data are affected substantially by QP values, but while are affected also very big by picture material. Scape texture is complicated, the video source that motion change is violent will consume more bits.In order to balance code consume bit it is few and Coding consumes the extremely many video sources of bit, and the frame rate adjustment taken the initiative both can ensure that the Rate Control in transmitting procedure, and Can keep smooth and excellent on visual quality.

The legal QP values of coding are segmented.With 20% as experience cut-point, i.e., low section 20%, stage casing 60%, high section 20%.For QP spans are for the encoder of [0,31], its be segmented into low section [0,8], stage casing [9,24], high section [25, 31].For QP spans are for the encoder of [0,51], it is segmented into low section [0,10], stage casing [11,40], high section [41,51]。

The thought of the regulation of variable frame rate is when frame coding QP values frequently fall in low section of interval, suitably to increase frame per second, Frame coding output bit number will directly be reduced, it is interval to stage casing that Rate Control will adjust follow-up QP values；When frame encodes QP values frequently Fall when high section is interval, suitably reduce frame per second, will directly increase frame coding output bit number, it is also possible to so that follow-up QP values are adjusted It is whole interval to stage casing.

In the initial phase of coding, actual coding frame per second is arranged to F_C←F_S, i.e., it is equal with sampling frame per second.Arrange Frame per second changed factorThe frame per second modification strategy of the present invention is, in present encoding moment t, in seeking nearest 1 second coding, to try to achieve QP What is be worth is average as shown in formula (18).

JudgeSpan, it would be possible to there are three kinds of situations：

The first situation, ifFall in low section of interval, then

Second situation, ifFall interval in stage casing, then frame per second keepsIt is constant；

The third situation, ifFall interval in high section, then

Finally, actual coding frame per second is modified toCertainly, in order to ensure the continuous of visual experience and play The ability of equipment, F_CModification will be strictly limited between 10Hz-100Hz.

In the first case, the double of frame per second will need the frame of new insertion respective numbers.Present invention provide that with closest to The video source data of frame that is, in the case where frame per second is double, repeats same frame and compiles as the video data of new insertion frame Code is twice.This method can be prevented effectively from the unnecessary distortion brought using interpolation class and motion compensation class method.

In the third situation, reducing by half for frame per second means that the frame for having half quantity will not be encoded, i.e., between fixation Encoded every the frame of step-length.The video source data being skipped will not be introduced in encoder, then the frame in cataloged procedure Between coded portion will not be using to frame-skipping data.Therefore, will not experience because of frame losing when user watches video The mutation of visual quality.

First and the 3rd both of these case modification frame per second after, be required for new F_CValue substitutes into formula (1) and recalculates Pixel target bits Tbpp.After the modification of conducting frame rate, next code just can enter after must again completing a frame per second cycle Row frame per second next time judges and changes.

Embodiment

Fig. 1 is the key step flow chart that coding realizes process, is specifically included：

Step 101：Select/determine encoder.The standard of encoding video pictures has many, common are：MPEG-1、 MPEG-2, MPEG-4, H.261, H.263, WMV1, WMV2, RV10, RV20, H.264/AVC, H.264/SVC, HEVC etc..From matching somebody with somebody Put selection and the code stream encapsulation format that encoder is read in file.

Step 102：Initialization code check control parameter.The target bit rate of input bit rate control, information source from configuration file The frame per second of video, resolution, GOP preset lengths.According to specified file encapsulation format (container), set up output file after coding and connect Mouthful.

Step 103：Loop coding starts, and reads the frame data of information source video.Actual coding frame per second is relied on from information source video A frame video original data is obtained in file or data flow.

Step 104：The integrated bit rate controller of the present invention.Frame level bit-rate control is carried out, includes scene detection and GOP length Adjustment function, the regulatory function of variable frame rate, the control function of variable bit rate.The concrete operating principle of this step is entered by Fig. 2 Row is described in detail.

Step 105：Encoder is encoded.Frame type, QP are included according to the coding parameter that integrated bit rate controller is given The key parameters such as value, frame per second, the original video data to obtaining in step 103 carries out the Video coding of a frame.

Step 106：NAL packs.The stream that acquisition is encoded in step 105 is carried out into NAL packaging operations, will NAL stream writes In presetting file format (container).Statistics NAL length is bit number, the objective visual quality distortion after statistical coding PSNR。

Step 107：Whether cycle criterion coding completes.The situation that end-of-encode judges may have information source video to terminate and in advance If coding frame number reaches is expected both.There is one for true time when above two may appoint, end loop otherwise jumps to step 103 proceed next frame coding.

The above-mentioned description to Fig. 1 is presented residing for the integrated bit rate controller that Rate Control is realized in video coding process Position.Fig. 2 gives the present embodiment integrated bit rate controller operation principle.As shown in Fig. 2 including unique entrance and unique Outlet, wherein crucial step has：

Step 201：Rate Control relevant parameter initialization/update operation.In the first frame that coding is carried out, need to code Rate controls the initialization of relevant parameter, includes target bit rate, information source resolution, information source frame per second, default GOP length, buffering Area is initialized as 0, reads default QP values.The on-off control of integrated bit rate controller is checked, if variable bit rate control is opened, Variable GOP length adjustments variable frame rate adjusts and is set to open or close according to configuration input；If variable bit rate control is closed, Then variable GOP length adjustments variable frame rate adjusts pressure and is set off.Frame per second changed factor is setIt is initialized as 1.Whole In individual cataloged procedure, initialization operation is only once configured.When coding proceeds to non-first frame, Rate Control is carried out related Parameter updating operation：The frame type of statistics previous frame coding, actual QP values Q_t-1, NAL output bit number R_t-1If, previous frame coding Frame is I- frames, then encoded I- frame numbers value adds 1 and encoded P- frame numbers value sets to 0, if previous frame coded frame is P- frames, currently The encoded P- frame numbers values of GOP add 1.In whole cataloged procedure, updating operation will be carried out in each frame in addition to the first frame.

Step 202：Coder parameters are rewritten.The coding parameter of each frame is stored and adjusted in this step.Adjust Process is exactly to be rewritten one by one by external step 203 splitter.

Step 203：Rate Control functional branch device.The splitter one by one separates the function of three parts, is walked first Rapid 204 calling and rewriting, then carries out calling and rewriting for step 205, finally carries out calling and rewriting for step 206.This In emphasize, step 204,205,206 are called and must sequentially be carried out.

Step 204：Variable GOP length adjustments.When scene detection and GOP length adjustments controlling switch are opened, field is performed Scape detects the adjustment with GOP length；Return is redirected during closing and does not make any operation.The idiographic flow of this step is given by Fig. 3.

Step 205：Variable frame rate is adjusted.When variable frame rate adjusts controlling switch to be opened, perform same to the change of frame per second When be related to the setting of reading manner thereafter to information source data；Return is redirected during closing and does not make any operation.This step it is concrete Flow process is given by Fig. 4.

Step 206：Variable bit rate is controlled.When variable bit rate controlling switch is opened, this step is performed, otherwise redirected Any operation is not made in return.The idiographic flow of this step is given by Fig. 5.

Step 207：Storage and record coded data.Include information source video data, the encoded output video stream of each frame NAL, current relief area height.The related data of these codings by for step 204,205,206 provide the foundation of calculating with it is anti- Feedback.

Topmost three partial function is launched by Fig. 3, Fig. 4 and Fig. 5 in above-mentioned Fig. 2.Fig. 3 gives scene switching inspection Survey and adjust flow chart with dynamic GOP, specifically include：

Step 301：Read and obtain current frame (t) to be encoded, information source video data.Regulation t=0,1,2 ....

Step 302：Temporary cache information source video data.Z^-1It is a hysteresis memories, is input into the information source for present frame (t) Video data, is output as the information source video data of adjacent upper moment coded frame (t-1).

Step 303：The grey level histogram H of statistics present frame and nearest neighbor frame_t-1And H_t。

Step 304：Similarity Measure splitter.By intensity histogram diagram data H_t-1And H_tStep 305 and step are sent to simultaneously Computing is carried out in rapid 306.

Step 305：For t >=1, karr Pearson linearly dependent coefficient r is calculated_t-1,t：

Wherein,

Step 306：For t >=1, high dimension vector included angle cosine value cos (θ is calculated_t-1,t)：

Step 307：Similarity Measure component collects.According to the r that acquisition is calculated in step 305 and step 306_t-1,tAnd cos (θ_t-1,t) carry out index of similarity Sim_tComputing Sim₀=1,

Step 308：Judge whether index of similarity Sim_t<ξ, if being very then considered that scene switching is produced and gone to step 309, if vacation then thinks that scene switching, without producing, does not deal with and terminates.The value of ξ is empirical value 0.85.

Step 309：The numerical value whether the P- number of frames encoded in current GOP reaches a frame per second cycle is counted, is Then going to step 310 carries out GOP length adjustments, otherwise terminates.

Step 310：Terminate a upper GOP, newly start a GOP, present frame type is set to into I- frames.

Step 311：New GOP is set, the encoded P- frame numbers of GOP are set to 0, it is preset value to arrange GOP length.

Fig. 4 gives the flow chart of variable frame rate adjustment, specifically includes：

Step 401：Calculate currently employed encoder QP values low (20%), in (80%), high (20%) three section of interval.For QP spans [0,31] encoder (as MPEG-1, MPEG-2, MPEG-4, H.261, H.263, WMV1, WMV2, RV10, RV20 for), it is segmented into low section [0,8], stage casing [9,24], high section [25,31].For QP spans are in the volume of [0,51] Code device (as H.264/AVC, H.264/SVC, HEVC) for, its be segmented into low section [0,10], stage casing [11,40], high section [41, 51]。

Step 402：Judge whether encoded frame number reaches frame per second numerical value (i.e. the frame number of 1 second video), be then to enter step 403, otherwise terminate.

Step 403：QP averages in nearest 1 second coding are counted,

Wherein, actual coding frame per second F_CInitialization is arranged to frame per second F of sampling_S。

Step 404：Judge to be calculated in step 403 the QP averages for gettingWhether fall interval in high section QP, be to turn step Rapid 407, otherwise go to step 405 and judged again.

Step 405：Judge to be calculated in step 403 the QP averages for gettingWhether fall in low section of QP interval, be to turn step Rapid 406, otherwise illustrateFall interval in stage casing QP, do not deal with and exit.

Step 406：The double process of frame per second.IfThenAndOtherwise F_CKeep not Become.

Step 407：Frame per second is processed by half.IfThenAndOtherwise F_CKeep not Become.

The frame per second changed factor occurred in above-mentioned Fig. 4 stepsInitialization operation complete in Fig. 2 steps 201.Change The factorNumerical value will be limited for the reading of information source video in follow-up Rate Control.

Fig. 5 gives the control flow chart of variable bit rate, specifically includes：

Step 501：Current frame pixel point target bit value is calculated,

Wherein, target bit rate TBR is initially set when coding starts, and can be written in an encoding process, here Using newest nearest numerical value；Actual coding frame per second is designated as information source frame per second i.e. F in initialization_C←F_S, here using newest Nearest numerical value.

Step 502：Updating the data in Fig. 2 steps 201, carries out buffer size renewal,

Buffer size is corrected using PD control device, is calculated asIts InRepresent B (t) be corrected after result, parameter alpha, β by empirical value choose α=0.5, β=0.55；Calculate 1 frame per second week In phase consecutive frame coding QP values absolute difference with,

Last bit increment Δ R is calculated as：

Wherein, μ is that empirical parameter value is 0.618.

Step 503：Carry out frame type and judge branch.If I- frames coding then goes to step 504, if P frames then go to step 506, if B- frames then go to step 508 and reporting an error and exiting.

Step 504：I- frame per second distortion model lnR=alnQ+blnC+c parameters update.Nearest continuous I- before collection The Q, C and R of the s frame data of frame constructs the matrix of the row of s rows 3,

The value of window size s is the reasonable maximum between minimum 5, maximum 30.Returned using multiple linear Return, be fitted using least square approximation, calculate and obtain model parameter a, b, c.

Step 505：Present frame Texture complication is calculated,

Incremental computations obtain present encoding I- frame QP values,

Finally, Q_tValue will be bound, Q_t←min{Q_t-1+2,max{Q_t,Q_t-1- 2 } }, and it is limited in legal span.

Step 506：P- frame per second distortion model lnR=alnQ+c parameters update.The s frames of nearest continuous P-frame before collection The Q and R of data constructs the matrix of the row of s rows 2：

The value of window size s is the reasonable maximum between minimum 5, maximum 30.Linearly returned using unitary Return, be fitted using least square approximation, calculate and obtain model parameter a, c.

Step 507：Incremental computations obtain present encoding P- frame QP values,

Finally, Q_tValue will be bound, Q_t←min{Q_t-1+2,max{Q_t,Q_t-1- 2 } }, and it is limited in legal span.

In foregoing description, Fig. 3 calculates the QP that the GOP and frame type of modification, Fig. 4 calculate the frame per second of modification, Fig. 5 calculating is obtained Value will all be used directly to next code device and carry out a frame coding.

The present invention has been realized integrated and is provided interface in the way of dynamic link.In order to illustrate the overall performance of the present invention, With the speedy coder x265 after encoder platform HEVC cutting of increasing income【Compatible to HM-10.0, Win32SDK, http://code.google.com/p/x265/】As a example by, interface of the present invention can be directly invoked, realize unified variable bit Rate, variable frame rate, variable GOP length Rate Control.

The present invention is with CIF (352 × 288,4:2:0) as a example by sequence, three groups of videos are constructed with 20 common YUV sequence assemblies Data are as shown in table 1.

Table 1CIF sequence of packets and totalframes

^*Note：Contain Table sequences in M-cif, Table sequences itself have 2 scenes.Therefore the total scene numbers of M-cif It is 7.

To construct three groups of video sequences carry out encoded test, with fix QP values (respectively 17,22,27,32,37,42) Output result under low latency coding (IPP..PP) structure is target, carries out test of the present invention.Target bit rate and initial Q P value All configure according to the output result of fixed QP values coding.As shown in table 2, RC (on/off), SC (on/off), AS (on/ Off Rate Control switch, scene detection switch, frame per second controlling switch) are represented respectively.With fixed bit rate (CBR) as control in table 2 Target processed is tested.BD-Rate represents the curve linear relationship between bit rate and distortion, and it is a percentages, its value Identical visual quality is reached for negative indication, the ratio that bit rate is saved is just to represent the ratio that bit rate is excessively consumed.From table As can be seen that the Rate Control of present invention enforcement, scene detection, frame per second control all to go out under three kinds of different switch combinations in 2 The situation that existing bit rate is saved.BD-Rate numerical value respectively reaches -32.89%, -31.74%, -42.71% in Y-component, this explanation The present invention implements that directly for Video coding obvious performance boost will be brought.The numerical value that last column of table 2 is listed is BD- The formula result of calculation of Rate-Old, to do control reference, does not make special discussion.

BD-Rate Performance comparisions after the present invention enforcement of table 2

H-cif as shown in Figure 6, test knot of the tri- groups of cycle testss of M-cif, L-cif under coding structure IPP..PP Really, X- axial coordinates represent the time scale that CIF (30Hz) video image is played, and Y- axles show respectively relief area height and interframe Similarity numerical value.During three width subgraphs of relief area are represented with Y- axles, it can be seen that relief area curve is near the line of theoretical level 0 Now closely surround and frequent moving.GOP is caused to initialize in scene switching, its first frame (I- frames) coding causes relief area substantially to rush Height, then Rate Control can quickly effectively cause relief area tend to 0 line again, show the present invention Rate Control ability it is strong. In the three width subgraphs with similarity as Y- axial coordinates, it can clearly be seen that effective due to index of similarity proposed by the present invention Property, when scene is unified, index of similarity all unusual convergence theoretially optimum values 1, when scene switching occurs, similarity There is different degrees of obvious tenesmus in index.In the enforcement test of the present invention, for above-mentioned three kinds of sequences, carried using the present invention , there is not false retrieval and missing inspection, accuracy rate 100% in the scene change detection method for going out.

I- frames continuous programming code CIF (30Hz) sequence D ealdline complete as shown in Figure 7 totally 1372 frame.X- coordinates tables in Fig. 7 Show coding frame number flowing water.It is 1mbps to arrange target bit rate increment, closes target pixel points bit 0.328809.It is with 196 frames Bit rate variation section, is divided into 7 sections：0 frame -195 frame TBR=2mbps, Tbpp=0.657618；The frame TBR=3mbps of 196 frame -391, Tbpp=0.986427；392 frame -587 frame TBR=4mbps, Tbpp=1.315236；588 frame -783 frame TBR=5mbps, Tbpp= 1.644045；784 frame -979 frame TBR=4mbps, Tbpp=1.315236；980 frame -1175 frame TBR=3mbps, Tbpp= 0.986427；1176 frame -1371 frame TBR=2mbps, Tbpp=0.657618.Can see from first subgraph of Fig. 7, frame is put down Pixel reality output and target it is extraordinary press close to, with the growth and reduction of target pixel points bit, reality output Frame mean pixel point bit has rapidly carried out following up and around fluctuation.Second width subgraph shows, in very narrow relief area In interval [- 1,1], the present invention can effectively control the Video coding under variable bit rate.Third and fourth width subgraph is distinguished Give PSNR curves and frame actual coding QP value.Can see, with the change of target bit rate, PSNR curves and QP values point Cloth is presented the characteristics of being segmented corresponding, has good fluctuation or distribution rule in each segmentation.

Claims (8)

1. variable frame rate bit rate control method in low latency Video coding, it is characterised in that include：

Present encoding frame per second F is adjusted by current quantization parameter Q_C, when the value of quantization parameter Q is in low section of interval, not high Increase present encoding frame per second F in frame per second upper range_C；When the value of quantization parameter Q is in high section interval, in the case where frame per second is not less than Present encoding frame per second F is reduced in the range of limit_C；When the value of quantization parameter Q is interval in stage casing, present encoding frame per second F is kept_CIt is constant；

Also adjust including present image group GOP, the GOP is adjusted and comprised the following steps：

1) grey level histogram of present frame is calculated；

2) index of similarity of two consecutive frames is calculated by the grey level histogram of present frame and the grey level histogram of previous frame；It is described Index of similarity is represented by cosine similarity：

$cos\left({\mathrm{\&theta;}}_{t-1,t}\right)=\frac{\underset{i=1}{\overset{n}{\&Sigma;}}{H}_{t-1}\&lsqb;i\&rsqb;\&CenterDot;H_t\&lsqb;i\&rsqb;}{\sqrt{\underset{i=1}{\overset{n}{\&Sigma;}}{(H_{t-1}\&lsqb;i\&rsqb;)}^2}\sqrt{\underset{i=1}{\overset{n}{\&Sigma;}}{(H_t\&lsqb;i\&rsqb;)}^2}};$

Wherein, cos (θ_t-1,t) it is the histogrammic cosine similarity of two consecutive frames, t represents present encoding moment, H_t[i] is represented The video frame image grey scale pixel value of coding moment t is total for the pixel of i, H_t-1[i] is represented upper one coding moment t-1's Video frame image grey scale pixel value is total for the pixel of i, and the scope of video frame image grey scale pixel value is 1 to n, and n is gray scale Total element number when rectangular histogram one-dimension array is represented；

3) when the index of similarity of two consecutive frames is less than threshold value, then it represents that occurrence scene switches, into step 4)；Otherwise arrange Present frame type is set to P- frames, extracts return to step 1 after next frame data)；

4) count in present image group GOP whether frame per second cycle value is reached with the P- frames for encoding, in this way, then into step 5), Present frame type is otherwise set and is set to P- frames, extract return to step 1 after next frame data)；

5) present frame type is set and is set to I- frames, start a new GOP, extract return to step 1 after next frame data).

2. variable frame rate bit rate control method in low latency Video coding as claimed in claim 1, it is characterised in that introduce linear Correlation coefficient, comes together to characterize index of similarity with cosine similarity；

$r_{t-1,t}=\frac{\underset{i=1}{\overset{n}{\&Sigma;}}(H_{t-1}\&lsqb;i\&rsqb;-{\stackrel{\&OverBar;}{H}}_{t-1})(H_t\&lsqb;i\&rsqb;-{\stackrel{\&OverBar;}{H}}_t)}{\sqrt{\underset{i=1}{\overset{n}{\&Sigma;}}{(H_{t-1}\&lsqb;i\&rsqb;-{\stackrel{\&OverBar;}{H}}_{t-1})}^2}\sqrt{\underset{i=1}{\overset{n}{\&Sigma;}}{(H_t\&lsqb;i\&rsqb;-{\stackrel{\&OverBar;}{H}}_t)}^2}}$

${\stackrel{\&OverBar;}{H}}_t=\frac{1}{n}\underset{i=1}{\overset{n}{\&Sigma;}}{H}_t\&lsqb;i\&rsqb;$

Wherein, r_t-1,tLinearly dependent coefficient is crossed for two consecutive frames are histogrammic,It is the video frame image pixel in coding moment t Gray value average,For in the video frame image grey scale pixel value average of coding moment t-1.

3. variable frame rate bit rate control method in low latency Video coding as claimed in claim 2, it is characterised in that similarity refers to It is designated as the product of cosine similarity and linearly dependent coefficient, or square taking advantage of with cosine similarity for linearly dependent coefficient Product.

4. variable frame rate bit rate control method in low latency Video coding as claimed in claim 1, it is characterised in that by frame per second Changed factorTo adjust present encoding frame per second F_C,Wherein ← represent parameter assignment to the direction of arrow, F_SBe from The sampling frame per second that video source is obtained；

Increase present encoding frame per second F_CForReduce present encoding frame per second F_CFor

5. variable frame rate bit rate control method in low latency Video coding as claimed in claim 1, it is characterised in that current amount Change the meansigma methodss of the quantization parameter that parameter Q is expressed as used in the nearest 1 second coding till present encoding moment t

6. variable frame rate bit rate control method in low latency Video coding as claimed in claim 1, it is characterised in that frame per second lower limit For 10Hz, frame per second upper limit 100Hz.

7. variable frame rate bit rate control method in low latency Video coding as claimed in claim 1, it is characterised in that described low section Interval is low section 20% in legal quantization parameter Q-value span, and high section is interval for height in legal quantization parameter Q-value span Section 20%, remaining 60% is that stage casing is interval in quantization parameter Q-value span.

8. variable frame rate bit rate control method in low latency Video coding as claimed in claim 1, it is characterised in that current Coding moment t, when such as current volume frame is I- frames, the quantization parameter Q for using is：

$Q_t=Q_{t-1}+\frac{Q_{t-1}}{a}\&CenterDot;\&lsqb;\frac{\mathrm{\&Delta;}R}{R_{t-1}}-\frac{b\&CenterDot;(C_t-C_{t-1})}{C_{t-1}}\&rsqb;;$

In current coding moment t, when such as current volume frame is P- frames, the quantization parameter Q for using is：

$Q_t=Q_{t-1}+\frac{Q_{t-1}}{a}\&CenterDot;\frac{\mathrm{\&Delta;}R}{R_{t-1}};$

Wherein, Q_tFor quantization parameter Q, Q that current coding moment t is used_t-1For the quantization ginseng that upper one coding moment t-1 is used Number Q, R_t-1Represent the frame coding output bit of a coding moment t-1, C_tFor the coded image stricture of vagina of current coding moment t Reason complexity, C_t-1For the coded image Texture complication of upper one coding moment t-1；△ R are to encode the needs amendment of moment t Bit increment,Ipt (t) represents 1 actual coding frame per second F_CThe quantization parameter Q of consecutive frame coding in cycle Absolute difference and,Revised t buffer pool sizeForB T () is the true buffer pool size of current coding moment t, B (t-1) is the true buffer pool size of upper coding moment t-1, α, β are controller parameter, μ be prevent and kill off 0 empirical parameter；

Distortion rate model is lnR=aln Q+bln C+c when current volume frame is I- frames, and Q represents quantization parameter, and R represents frame Coding output bit, C presentation code image texture complexities, a, b, c be distortion rate model parameter, distortion rate model parameter a, b Value be updated by multiple linear regression；Distortion rate model is lnR=aln Q+c when current volume frame is P- frames, is lost The value of sincere model parameter a is updated by one-variable linear regression；

Coding frame per second F in upper one coding moment_CAfter change, need according to new coding frame per second F_CRedefine pixel mesh Mark bit Tbpp,So as to buffer pool size B (t) at present encoding moment, B (t)=B (t-1)+R_t-1- Tbpp, R_t-1A coding moment t-1 time frame coding output bit is represented, TBR is target bit rate, W is image pixel width, H It is high for image pixel.