CN101394554A - Adaptive frequency hopping method and device - Google Patents

Abstract

The invention discloses a self-adaptive frame skipping method and a device. The method comprises the following steps: the mean value of the motion vector of a current frame is calculated; the weighted sum is used as a smoothing factor, and the weighted sum is composed of the ratio of the mean value of the motion vector of the current frame and the mean value of the average motion vector of at least one adjacent latest frame which is calculated as well as the ratio of the relative mean value of the motion vector of the current frame and the relative mean value of the average motion vector of at least one adjacent latest frame which is calculated; and if the value of the smoothing factor is within the range of the threshold value, the current frame is processed through frame skipping. The self-adaptive frame skipping device is also provided. The self-adaptive frame skipping method and the device accord with the HVS visual characteristics, ensure that a terminal user obtains optimal visual quality, and satisfy the image quality without affecting the visual quality when a user views the image.

Description

A kind of adaptive frequency hopping method and device

Technical field

The present invention relates to the multimedia processing technology field, be meant a kind of adaptive frequency hopping method and device in the video coding especially.

Background technology

In multi-medium data was handled, Rate Control was the key technology of real-time video coding, and it is determining the quality of coded image; Particularly in wireless low variable code rate real-time video communication, rate control algorithm is most important.

It is the frame layer control technology of Rate Control that frame-skipping is handled, and in the variable code rate channel temporal quality and the space selected topic is carried out balance, finally obtains optimum decoded video quality.Mainly in the finite element network transmission, bandwidth fluctuation is very little for video in the past, is the cbr (constant bit rate) coding substantially, seldom has frame-skipping to handle, so research is also few.Compare the frequency hopping method in the Rate Control that classical frequency hopping method is TMN8, it is judged according to the fullness degree of encoding buffer fully.Rate Control hypothesis encoder-side has individual virtual buffering region, and every volume is finished a frame, just the volume bit stream all is put into a little buffering areas, then carries out the next frame coding.Data in the buffering area then send to channel by the channel width requirement and transmit (as 64k/s).If overflowing so just can not appear in the buffer pool size infinity, but in real-time video was used, buffering area too conference brought corresponding time delay, also claims the encoding buffer time delay.In order to guarantee real-time video communication, delay requirement is less than 100ms, so buffer pool size must be less than the data volume (being that bandwidth multiply by time delay) of the Channel Transmission in the tolerance time delay, otherwise data are long in the buffering area holding time, will surpass the minimum-time lag requirement, cause picture lag.The capacity of buffering area can not be too little, if very little, data are often overflowed, though satisfied in real time, view data is often lost, and picture quality can't guarantee.Be kept in the buffering area data can not be excessive can not be too small.If buffer overflow then can be lost these data and be caused decoding error; On the contrary, if buffer occupancy cross low will waste bandwidth.So encoder to control buffering area can not overflow can not underflow, the general pattern data are bigger, are not easy to occur underflow, mainly are to prevent overflow, and overflow produces decoding error and can have a strong impact on picture quality.

To preserve data volume very big when buffering area, avoiding effective ways of overflow is exactly frame-skipping, the algorithm that carries out frame-skipping according to buffer state only, it is a kind of unexpected frame-skipping algorithm, have tangible problem: it does not consider institute's skip pictures self character, may skip the image of complicated image or motion violent (visual sensitivity), causes motion smoothing and continuity to descend, picture quality is inconsistent, causes that subjective visual quality do seriously descends.

Summary of the invention

In view of this, the invention reside in provides a kind of adaptive frequency hopping method and device, to solve the relatively poor problem of picture quality that above-mentioned frame-skipping scheme exists.

For addressing the above problem, the invention provides a kind of method of adaptive frame skipping, comprising: the mean value of the motion vector of computing present frame; The ratio of the mean value of the average motion vector of the mean value of the motion vector of use present frame and the adjacent at least one frame recently that calculates, with the ratio weighting sum of, the relative mean value of the motion vector of present frame and the relative mean value of the average motion vector of the adjacent nearest at least one frame that calculates as smoothing factor; When the value of judging described smoothing factor within threshold range, described present frame is done frame-skipping handles.

Preferably, the process of the mean value of the motion vector of described computing present frame comprises:

Calculate the value of all motion vectors, preserve and the motion vector of all macro blocks that sort, 20% to 30% motion vector value MVx, MVy that the value of selecting is maximum draw mean value MA by the motion vector value of selecting.

Preferably, the process that draws smoothing factor adopts following formula operation:

$\mathrm{MI}=0.5*\frac{{\mathrm{MA}}_{\mathrm{curr}}}{{\mathrm{MA}}_{\mathrm{avg}}}+0.5*\frac{\mathrm{\Δ}{\mathrm{MA}}_{\mathrm{curr}}}{\mathrm{\Δ}{\mathrm{MA}}_{\mathrm{avg}}}$

MA _CurrThe average motion vector information of expression present frame, i.e. MA (motion average) value; Δ MA _CurrThe absolute value of the difference of the preceding relatively N frame MA of the MA of table present frame, N is more than or equal to 1; MA _AvgWith Δ MA _AvgThe mean value of representing front multiframe MA, Δ MA respectively.

Preferably, the described value that ought judge described smoothing factor is within threshold range, and the process of described present frame being done the frame-skipping processing comprises:

If between [0.9～1.1], then doing frame-skipping, the value of judging smoothing factor handles; Or, if the value of judging smoothing factor is then judged buffer state between [0.7～0.9] or [1.1～1.3], if buffer area data volume between 50% to 70%, is then done frame-skipping to present frame and handled.

The present invention also provides a kind of device of adaptive frame skipping, comprising:

Receiving element is used for each Frame;

Connect the arithmetic element of described receiving element, comprising:

The mean value module is used for the mean value of the motion vector of each frame of computing;

The smoothing factor module, be used to use the ratio of mean value and the mean value of the average motion vector of the adjacent nearest at least one frame that calculates of the motion vector of present frame, with the ratio weighting sum of, the relative mean value of the motion vector of present frame and the relative mean value of the average motion vector of the adjacent nearest at least one frame that calculates as smoothing factor;

The judging unit that connects described arithmetic element, the value that is used for judging described smoothing factor trigger the frame-skipping unit within threshold range;

The frame-skipping unit that connects described judging unit is used for that described present frame is done frame-skipping and handles.

Preferably, the process of the mean value of the motion vector of described mean value module arithmetic present frame comprises:

Preserve and the motion vector of all macro blocks that sort, calculate the value of all motion vectors, 20% to 30% motion vector value MVx, MVy of the value of selecting maximum draw mean value MA by the motion vector value of selecting.

Preferably, the process of value within threshold range of the described smoothing factor of described judgment unit judges comprises:

The value of judging smoothing factor whether between [0.9～1.1], or, if the value of judging smoothing factor between [0.7～0.9] or [1.1～1.3], judges then that the data volume of buffering area is whether between 50% to 70%.

Method and apparatus of the present invention meets the HVS visual characteristic, and for the harmonic motion image sequence, HVS is more responsive to spatial detail comparison time details; For high motion sequence, HVS is more responsive to time detail.Therefore to the harmonic motion image sequence, can skip more multiframe and do not influence subjective picture quality, saving bit number improves space quality; Correspondingly, reduce frame-skipping as far as possible, obtain temporal quality preferably with the reduction space quality to high motion sequence.When channel width is low especially, in the time of can satisfying time continuity and space quality simultaneously, according to human vision characteristics, rationally reduce video time resolution and improve the sdi video quality, to guarantee that the terminal use has best visual quality, satisfy picture quality, and can not influence the user and watch visual quality for images.

Description of drawings

Fig. 1 is the flow chart of the inventive method embodiment;

Fig. 2 is the structure chart of apparatus of the present invention embodiment.

Embodiment

For clearly demonstrating the scheme among the present invention, provide preferred embodiment below and be described with reference to the accompanying drawings.

Referring to Fig. 1, the method in the embodiment of the invention comprises:

Step 11: the mean value of the motion vector of computing present frame;

During the arithmetic average value, calculate the value of all motion vectors earlier, preserve and the motion vector of all macro blocks that sort, promptly get MAx and MAy, inside 20% to 30% maximum motion vector value MVx, MVy of the value of selecting draws mean value MA by the motion vector value of selecting; For the motion vector value of INTRA piece because have only estimation coupling to make the INTRA model based coding less than suitable profound learning, so can think that its motion vector is the maximum of estimation scope, with maximum as motion vector value.

Step 12: the ratio of the mean value of the average motion vector of the mean value of the motion vector of use present frame and the adjacent at least one frame recently that calculates, with the ratio weighting sum of, the relative mean value of the motion vector of present frame and the relative mean value of the average motion vector of the adjacent nearest at least one frame that calculates as smoothing factor;

The computational process of smoothing factor can be referring to following formula:

The motion smoothing factor represents with MI, that is:

$\mathrm{MI}=0.5*\frac{{\mathrm{MA}}_{\mathrm{curr}}}{{\mathrm{MA}}_{\mathrm{avg}}}+0.5*\frac{\mathrm{\Δ}{\mathrm{MA}}_{\mathrm{curr}}}{\mathrm{\Δ}{\mathrm{MA}}_{\mathrm{avg}}}$

MA _CurrThe average motion vector information of expression present frame is MA (motion average) value; Δ MA _CurrThe absolute value of the difference of the relative former frame MA of MA of table present frame; MA _AvgWith Δ MA _AvgThe mean value of representing front multiframe MA, Δ MA respectively, all variablees all are positive numbers.Adopt the sliding window weighted mean method to try to achieve MA _AvgWith Δ MA _Avg, sliding window size decision systems can freely be selected the speed of motion reflection, is generally the 1-5 frame.

Step 13: when the value of judging described smoothing factor within threshold range, described present frame frame-skipping is handled.

In order to obtain the threshold value of MI, 6 sequences are tested, wherein sliding window size value is 2, and the result shows: the front and back two field picture is similar more, and its variations is also more little, changes relatively also hour, and then the value of MI is with regard to more near 1.As seen, the very mild image that moves also is the less image of MI value from experiment, and the result of this and our subjective observation also cunning is consistent.Therefore definition when MI 0.9 between 1 the time, be that gentle movement can the active frame-skipping; When MI 0.7 between 1.3 the time, the binding buffer district considers frame-skipping, like this no matter image sequence itself is to move greatly or little motion, its frame-skipping is based on relative motion, the frame-skipping ratio is reasonable.As shown in table 1, in each 103 frame of 6 sequences of test, the MA of X, the Y both direction ratio between 0.7 to 1.3 is about 10%-20%, and whole sequence is minimum for the ratio that the foreman and the whole sequence of big motion all is smooth motion Claire, so just can be to itself all not being that big moving image carries out unnecessary frame-skipping with own with regard to the smooth motion sequence.Test result is just meeting our anticipation: the smoothed image frame that does not influence subjective quality in the big motion image sequence is not encoded, doing frame-skipping handles, give the image of big motion to save bit number, and its in decoding end by former frame graphical representation, do not influence the subjective vision impression substantially.

Image sequence	The frame number of MA between 0.7-1.3	MA is in the ratio of 0.7-1.3
			tempete	17	16.5

claire	10	9.7％
			mobile	22	21.3％
paris	21	20.3％
			suize	14	13.4％
foreman	11	10.6％

Table 1

Judge whether the smooth motion image by the MI value size that calculates, handle the smooth motion image if the value of judging MI between [0.9～1.1], is then done frame-skipping; If the value of MI is then judged buffer state between [0.7～0.9] or [1.1～1.3], if buffer area data volume between 50% to 70%, smooth motion image is then done frame-skipping to present frame and is handled.

Describe method of the present invention above in detail, describe device embodiment of the present invention in detail below in conjunction with Fig. 2.Referring to Fig. 2, comprising:

Receiving element is used for each Frame;

Connect the arithmetic element of described receiving element, comprising:

The mean value module is used for the mean value of the motion vector of each frame of computing;

The smoothing factor module, be used to use the ratio of mean value and the mean value of the average motion vector of the adjacent nearest at least one frame that calculates of the motion vector of present frame, with the ratio weighting sum of, the relative mean value of the motion vector of present frame and the relative mean value of the average motion vector of the adjacent nearest at least one frame that calculates as smoothing factor;

The judging unit that connects described arithmetic element, the value that is used for judging described smoothing factor trigger the frame-skipping unit within threshold range;

The frame-skipping unit that connects described judging unit is used for that described present frame is done frame-skipping and handles.

Preferably, the process of the mean value of the motion vector of described mean value module arithmetic present frame comprises:

Preserve and the motion vector of all macro blocks that sort, calculate the value of all motion vectors, 20% to 30% motion vector value MVx, MVy of the value of selecting maximum draw mean value MA by the motion vector value of selecting.

Preferably, the process of value within threshold range of the described smoothing factor of described judgment unit judges comprises:

The value of judging smoothing factor whether between [0.9～1.1], or, if the value of judging smoothing factor between [0.7～0.9] or [1.1～1.3], judges then that the data volume of buffering area is whether between 50% to 70%.

Method and apparatus of the present invention meets the HVS visual characteristic, and for the harmonic motion image sequence, HVS is more responsive to spatial detail comparison time details; For high motion sequence, HVS is more responsive to time detail.Therefore to the harmonic motion image sequence, can skip more multiframe and do not influence subjective picture quality, saving bit number improves space quality; Correspondingly, reduce frame-skipping as far as possible, obtain temporal quality preferably with the reduction space quality to high motion sequence.When channel width is low especially, in the time of can not satisfying time continuity and space quality simultaneously, need between time continuity and space quality, weigh.According to HVS, the very poor image of continuous space quality is felt uncomfortable by human eye, obtain the definition of image for the mild especially image of motion not as sacrificing continuity, take frame-skipping in advance, substantially do not influence well as subjective video quality because reduce its frame per second, to stay more bits to give motion violent image, obtain better subjective visual quality do.Under the bandwidth condition condition of limited,, improve the sdi video quality like this, best visual quality is arranged to guarantee the terminal use by reasonable minimizing video time resolution according to human vision characteristics.The present invention can realize realizing frame-skipping based on the HVS characteristic, satisfies picture quality, and can not influence the user and watch visual quality for images.

For the method and apparatus of being set forth among each embodiment of the present invention, within the spirit and principles in the present invention all, any modification of being done, be equal to replacement, improvement etc., all should be included within protection scope of the present invention.

Claims (7)

1, a kind of method of adaptive frame skipping is characterized in that, comprising: the mean value of the motion vector of computing present frame; The ratio of the mean value of the average motion vector of the mean value of the motion vector of use present frame and the adjacent at least one frame recently that calculates, with the ratio weighting sum of, the relative mean value of the motion vector of present frame and the relative mean value of the average motion vector of the adjacent nearest at least one frame that calculates as smoothing factor; When the value of judging described smoothing factor within threshold range, described present frame is done frame-skipping handles.

2, method according to claim 1 is characterized in that, the process of the mean value of the motion vector of described computing present frame comprises:

Calculate the value of all motion vectors, preserve and the motion vector of all macro blocks that sort, 20% to 30% motion vector value MVx, MVy that the value of selecting is maximum draw mean value MA by the motion vector value of selecting.

3, method according to claim 1 is characterized in that, the process that draws smoothing factor adopts following formula operation:

$\mathrm{MI}=0.5*\frac{{\mathrm{MA}}_{\mathrm{curr}}}{{\mathrm{MA}}_{\mathrm{avg}}}+0.5*\frac{\mathrm{\Δ}{\mathrm{MA}}_{\mathrm{curr}}}{\mathrm{\Δ}{\mathrm{MA}}_{\mathrm{avg}}}$

MA _CurrThe average motion vector information of expression present frame, i.e. MA (motion average) value; Δ MA _CurrThe absolute value of the difference of the preceding relatively N frame MA of the MA of table present frame, N is more than or equal to 1; MA _AvgWith Δ MA _AvgThe mean value of representing front multiframe MA, Δ MA respectively.

4, method according to claim 1 is characterized in that, the described value that ought judge described smoothing factor is within threshold range, and the process of described present frame being done the frame-skipping processing comprises:

If between [0.9～1.1], then doing frame-skipping, the value of judging smoothing factor handles; Or, if the value of judging smoothing factor is then judged buffer state between [0.7～0.9] or [1.1～1.3], if buffer area data volume between 50% to 70%, is then done frame-skipping to present frame and handled.

5, a kind of device of adaptive frame skipping is characterized in that, comprising:

Receiving element is used for each Frame;

Connect the arithmetic element of described receiving element, comprising:

The mean value module is used for the mean value of the motion vector of each frame of computing;

The smoothing factor module, be used to use the ratio of mean value and the mean value of the average motion vector of the adjacent nearest at least one frame that calculates of the motion vector of present frame, with the ratio weighting sum of, the relative mean value of the motion vector of present frame and the relative mean value of the average motion vector of the adjacent nearest at least one frame that calculates as smoothing factor;

The judging unit that connects described arithmetic element, the value that is used for judging described smoothing factor trigger the frame-skipping unit within threshold range;

The frame-skipping unit that connects described judging unit is used for that described present frame is done frame-skipping and handles.

6, device according to claim 5 is characterized in that, the process of the mean value of the motion vector of described mean value module arithmetic present frame comprises:

Preserve and the motion vector of all macro blocks that sort, calculate the value of all motion vectors, 20% to 30% motion vector value MVx, MVy of the value of selecting maximum draw mean value MA by the motion vector value of selecting.

7, device according to claim 5 is characterized in that, the process of value within threshold range of the described smoothing factor of described judgment unit judges comprises:

The value of judging smoothing factor whether between [0.9～1.1], or, if the value of judging smoothing factor between [0.7～0.9] or [1.1～1.3], judges then that the data volume of buffering area is whether between 50% to 70%.

Images