CN114449281A - Intelligent code rate control optimization method based on scene change - Google Patents

Abstract

The invention discloses an intelligent code rate control optimization method based on scene change, which comprises the following steps: acquiring a video image; performing motion estimation on video coding; carrying out foreground change detection to obtain a detection result; evaluating a dynamic I frame set point according to the detection result; scene change judgment is carried out according to the coded internal motion vector, and the interval of the I frame is intelligently controlled, so that the code rate in a specific scene can be effectively reduced, and human eyes cannot feel obvious smear.

Description

Intelligent code rate control optimization method based on scene change

Technical Field

The invention relates to the technical field of video coding and decoding, in particular to an intelligent code rate control optimization method based on scene change.

Background

In practical use of video coding, due to the practical network bandwidth limitation, there is usually a fixed upper limit of the code rate. In particular, under the condition of a wireless mobile network, the limitation condition on the code rate is higher, so code rate coding is generally limited. In the field of video monitoring, the requirement on video real-time performance is high. The situation that the foreground of the picture is changed greatly, the change intensity is very high, and the duration is long often occurs, when the picture changes to be stable, the picture always appears smear, and even the picture cannot disappear for a long time.

The main treatment schemes at present are as follows:

a. increasing the upper limit of the bandwidth; the hardware cost is obviously increased, and the performance cost is lower;

b. setting coding code control as a variable code rate, and reducing a fixed I frame interval; the method can eliminate the ghost image quickly when the bandwidth is sufficient. However, there are two problems, the first is to fix the interval of I frame, or there is no I frame brushing in time when the picture change is over, the visual perception is affected by the ghost of the static picture; secondly, when the picture changes more, the code rate can not be further controlled and the ghost is more obvious because I frames are increased;

c. forcibly turning the upper limit of the coding QP down; code control may not be effectively controlled, resulting in frame loss in an actual network;

d. optimizing a code control algorithm; the general code control algorithm has more clamping conditions and is difficult to obviously optimize the effect of the specific scene.

Disclosure of Invention

In view of the technical problem that smear always appears after picture change is stable in the technical field of video coding and decoding at present, the invention provides an intelligent code rate control optimization method based on scene change, which judges the scene change according to the motion vector inside the code and intelligently controls the interval of I frames, so that the code rate in a specific scene can be effectively reduced, and human eyes can not feel obvious smear.

In order to achieve the above purpose, the embodiment of the invention adopts the following technical scheme:

an intelligent rate control optimization method based on scene change, the method comprising the following steps:

acquiring a video image;

performing motion estimation on video coding;

carrying out foreground change detection to obtain a detection result;

and evaluating the dynamic I frame set point according to the detection result.

According to an aspect of the invention, the performing foreground variation detection comprises: the foreground variation is determined using P-frame inter-prediction motion vectors.

According to an aspect of the invention, the performing foreground variation detection comprises: and obtaining the transverse motion intensity and the longitudinal motion intensity of the current image, judging that the foreground of the current image has stronger variation when the transverse motion intensity and the longitudinal motion intensity are both greater than a strong motion weight, and judging that the foreground of the current image has weak motion when the transverse motion intensity and the longitudinal motion intensity are both less than a weak motion weight.

According to one aspect of the present invention, the lateral motion strength is a ratio of a sum of lateral motion vectors of all blocks in a frame to a number of lateral motion vector blocks in a frame, and the vertical motion strength is a ratio of a sum of vertical motion vectors of all blocks in a frame to a number of vertical motion vector blocks in a frame.

In accordance with one aspect of the invention, the method further comprises: an I frame interval minimum is set.

In accordance with an aspect of the invention, the minimum value of the I frame interval is set to 25.

According to an aspect of the present invention, the performing foreground variation detection to obtain a detection result includes: and judging whether the current image is in a changing state or tends to be in a static state.

According to one aspect of the invention, the evaluating the dynamic I-frame set point based on the detection result comprises the steps of: p-frame coding is used when the image is in a changing state, and I-frame coding refresh is used when the image is in a state tending to be still.

According to one aspect of the invention, the evaluating the dynamic I-frame set point based on the detection result comprises the steps of: the default coding frame type of the first frame is an I frame; the frame type setting condition of each subsequent frame is determined according to the coded motion vector information of the previous frame.

According to one aspect of the invention, the method comprises the steps of:

judging whether to set the current frame as an I frame or not according to the initial value;

if the current frame is set as an I frame, counting an accumulated value of the coded frames after the I frame, and limiting the minimum range of the interval of the I frame;

setting a judgment value to judge whether the foreground stronger motion exists;

setting a judgment value to judge whether the foreground variation tends to be gentle or not;

setting a judgment value to judge whether the next frame is set with an I frame;

and if the current frame is not set as the I frame, setting the current frame as the P frame.

The implementation of the invention has the advantages that:

scene change judgment is carried out according to the coded internal motion vector, and the interval of the I frame is intelligently controlled, so that hardware resource consumption caused by an additional scene change detection algorithm can be avoided;

based on scene change, the interval of the I frame is adjusted, so that the code rate in a specific scene (for example, the foreground of a coded code stream picture changes for a long time and is strong, a code control vbr cannot be suppressed, the problem of overlarge coded frame loss is caused, and a code control cbr causes serious smear) can be effectively reduced, and human eyes cannot feel obvious smear;

by the method, a scene change starting point is detected, and then I frame refreshing is forcibly set when the scene change starting point is close to stop, so that afterimages are avoided. Meanwhile, P frame coding is continuously used in the subsequent coding, thereby controlling the code rate. The algorithm uses two limiting conditions to distinguish image change or stillness so as to accurately detect the image change or stillness degree, thereby accurately judging the position of the set I frame, eliminating ghost in time, not influencing human eye feeling, and simultaneously ensuring effective control of code rate.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a flowchart of an intelligent rate control optimization method based on scene changes according to the present invention;

fig. 2 is a schematic diagram of motion estimation according to the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The related scientific terms explanation and reference data referred to in this embodiment are as follows:

(1) code control (rate control);

in video coding, because the bandwidth of a communication channel is limited, the bit number of each frame of image coding output is controlled, and the image distortion is minimized under the limit of a certain total bit number. Commonly used methods include adjusting the frame rate, adjusting the image size, adjusting the quantization parameter, and the like.

(2) A code control mode;

typically CBR and VBR. CBR is a fixed rate control, aiming to fix the code rate at a constant value; VBR is a variable rate control, which allows the rate to fluctuate at a certain level, and can achieve smoother video quality.

(3) A frame type;

i frame: can be completely decoded without depending on other frames;

p frame: using inter prediction, relying on previous I and P frames;

b frame: the B frame is a bidirectional difference frame, namely the B frame records the difference between the current frame and the previous and next frames and depends on the I/P frame. Decoding is complex, delay is generated, and the scene with high real-time performance is not suitable for B frames.

(4) Changing scenes;

the scene contains a foreground and a background.

And (3) prospect: moving objects exist in the fixed lens;

background: the overall change of the picture or the overall change of other areas except the video main body due to the motion of the picture lens;

the embodiment is optimized based on the smear phenomenon caused by the large foreground variation in the fixed background.

(5) Inter-frame prediction;

inter-frame prediction has two important pointsThe concept is as follows: motion estimation and motion compensation. Motion estimation is to find the best corresponding block of the currently coded block in the coded picture and calculate the offset (i.e. motion vector) of the corresponding block. Assume that the current frame is P and the reference frame is P_rThe current coding block is B, at P_rThe best matching block with B is found, and the process is the motion estimation. As shown in FIG. 2, B^*The same as the demagnified position of B in the image. The Motion Vector (MV) is B_rCoordinates of upper left corner (x)_r，y_r) Minus B^*Coordinates (x, y) of the upper left corner are equal to (x)_r-x，y_r-y)。B_rThe block is the reference block of the B block, B_rThe pixel value of (a) is taken as the predicted value of the B block pixel. Because the temporal redundancy of the video is very large, the residual value to be coded will be very small as long as the reference block of the reference frame is properly selected. Then the motion vector is coded into the code stream, and the decoding end can decode the original image through the information. The motion compensation is a process of obtaining an estimated value of a current frame according to a decoded motion vector and an inter-frame prediction method. Motion estimation and motion compensation techniques are key methods to eliminate temporal redundancy of moving images.

As shown in fig. 1, an intelligent rate control optimization method based on scene change includes the following steps:

step S1: acquiring a video image;

video images are acquired by a video surveillance device, such as a surveillance camera.

Step S2: performing motion estimation on video coding;

video coding is motion estimated by inter prediction.

Step S3: carrying out foreground change detection to obtain a detection result;

the performing foreground variation detection includes: the foreground variation is determined using P-frame inter-prediction motion vectors.

And obtaining the transverse motion intensity and the longitudinal motion intensity of the current image, judging that the foreground of the current image has stronger variation when the transverse motion intensity and the longitudinal motion intensity are both greater than a strong motion weight, and judging that the foreground of the current image has weak motion when the transverse motion intensity and the longitudinal motion intensity are both less than a weak motion weight. The horizontal motion intensity is the ratio of the sum of horizontal motion vectors of all blocks in a frame to the number of horizontal motion vector blocks in a frame, and the vertical motion intensity is the ratio of the sum of vertical motion vectors of all blocks in a frame to the number of vertical motion vector blocks in a frame.

In the present embodiment, an I-frame interval minimum value is set to 25.

Step S4: and evaluating the dynamic I frame set point according to the detection result.

And judging whether the current image is in a changing state or tends to be in a static state.

P-frame coding is used when the image is in a changing state, and I-frame coding refresh is used when the image is in a state tending to be still. The default coding frame type of the first frame is an I frame; the frame type setting condition of each subsequent frame is determined according to the coded motion vector information of the previous frame.

In this embodiment, the method specifically includes the following steps:

judging whether to set the current frame as an I frame or not according to the initial value;

if the current frame is set as an I frame, counting an accumulated value of the coded frames after the I frame, and limiting the minimum range of the interval of the I frame;

setting a judgment value to judge whether the foreground stronger motion exists;

setting a judgment value to judge whether the foreground variation tends to be gentle or not;

setting a judgment value to judge whether the next frame is set with an I frame;

and if the current frame is not set as the I frame, setting the current frame as the P frame.

In practical application, the following specific examples are provided:

(1) the MV value of the area with high movement change in the video frame is larger than that of the area with slow movement or static movement, and the size of the motion vector in the predictive coding macro block has a certain direct proportional relation with the foreground movement change intensity in the video frame. Based on the rule, abstracting a video foreground average motion intensity calculation mode:

the transverse movement strength is as follows: averagemove_xThe sum of the horizontal MVs of all the blocks in one frame/the number of the horizontal MV blocks in one frame;

longitudinal movement strength: AverageMove_yThe sum of horizontal MVs of all blocks in a frame/the number of vertical MV blocks in a frame;

when AverageMove_x>Strong movethreshold and,

AverageMove_y>strong movethreshold, which considers the foreground motion to be strong;

when AverageMove_x<A WeakMoveThreshold and,

AverageMove_y<WeakMoveThreshold, believes that the foreground motion is weak and is close to no obvious motion;

the experiment shows that the strong exercise weight Strong move threshold is 2, and the weak exercise weight WeakMove threshold is 1, so that the effect is better.

(2) I frame interval minimum limit;

to avoid consuming too much bandwidth for a large number of I-frames in a short time, the I-frames need to have a minimum interval range. Meanwhile, in order to prevent mosaic ghost and the like caused by a large amount of image changes, tests show that the sensory visual perception of a large amount of sport mosaic with human eyes in the range of 1s is not obvious, and the interval time of the I frame is limited to 1s at least; the frame rate setting of the video monitoring field is generally 25 or 30 fps, so the minimum value of the I frame interval is set to be 25;

(3) evaluating a dynamic I frame set point according to a foreground change detection algorithm;

because the P frame can eliminate a large amount of redundancy in an interframe prediction mode, the P frame is used for coding when the motion is severe, and the code rate can be well controlled. Meanwhile, the visual perception of human eyes is improved as much as possible, I frames need to be inserted in time when the movement tends to be smooth, and smear and the like caused by insufficient instantaneous code rate are eliminated;

(4) the default coding frame type of a first frame is an I frame, the frame type setting condition of each subsequent frame is determined according to a foreground change detection algorithm and a dynamic I frame algorithm of coding MV information of a previous frame, and the specific steps are as follows:

1. setting an initial value:

a) the count FrameCount is 0; i/count the accumulated value of the encoded frame after the I frame to limit the minimum range 30 of the I frame interval

b) Setting scenecongeflag to 0; // whether there is strong foreground motion

c) Setting ChangeFinishFlag to 0; whether the foreground is stronger and the movement tends to be smooth

d) Setting SetKeyFrameFlag to 0; i frame is set for next frame

2. Each frame algorithm processing step:

a) before the coding is started:

i. if SetKeyFrameFlag is 1

1. Setting the current frame as an I frame;

2. the count FrameCount is 0; i/count the accumulated value of the encoded frame after the I frame to limit the minimum range 30 of the I frame interval

3. Setting scenecongeflag to 0; whether or not to move in the foreground

4. Setting ChangeFinishFlag to 0; whether the foreground is stronger and the movement tends to be smooth

5. Setting SetKeyFrameFlag to 0; i frame is set for next frame

Else set the current frame to be a P frame, FrameCount + ═ 1

b) After coded MV generation (I frame MV 0):

i. if the scene changeflag is 1, detecting whether the foreground motion of the image tends to be gentle according to a foreground change detection algorithm, and if the foreground change tends to be gentle, setting changefinishaflag to be 1;

if the sceneconngeflag is 0, detecting whether the image has strong motion according to a foreground change detection algorithm, and if so, setting the sceneconngeflag to 1;

set SetKeyFrameFlag 1 if scenecongeflag 1 and ChangeFinishFlag 1 and FrameCount > 25.

The algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. Moreover, the present invention is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.

In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore, may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that while some embodiments herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.

The above description is only an embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention disclosed herein are intended to be covered by the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.

Claims (10)

1. An intelligent rate control optimization method based on scene change is characterized by comprising the following steps:

acquiring a video image;

performing motion estimation on video coding;

carrying out foreground change detection to obtain a detection result;

and evaluating the dynamic I frame set point according to the detection result.

2. The intelligent rate control optimization method based on scene change according to claim 1, wherein the performing foreground change detection comprises: the foreground variation is determined using P-frame inter-prediction motion vectors.

3. The intelligent rate control optimization method based on scene change according to claim 2, wherein the performing foreground change detection comprises: and obtaining the transverse motion intensity and the longitudinal motion intensity of the current image, judging that the foreground of the current image has stronger variation when the transverse motion intensity and the longitudinal motion intensity are both greater than a strong motion weight, and judging that the foreground of the current image has weak motion when the transverse motion intensity and the longitudinal motion intensity are both less than a weak motion weight.

4. The intelligent rate control optimization method based on scene change according to claim 3, wherein the horizontal motion strength is a ratio of a sum of horizontal motion vectors of all blocks in a frame to a number of blocks of the horizontal motion vectors in a frame, and the vertical motion strength is a ratio of a sum of vertical motion vectors of all blocks in a frame to a number of blocks of the vertical motion vectors in a frame.

5. The method of claim 1, further comprising: an I frame interval minimum is set.

6. The intelligent rate control optimization method based on scene change according to claim 5, wherein the minimum value of I frame interval is set to 25.

7. The intelligent rate control optimization method based on scene change according to any one of claims 1 to 6, wherein the performing foreground change detection to obtain a detection result comprises: and judging whether the current image is in a changing state or tends to be in a static state.

8. The intelligent rate control optimization method based on scene change according to claim 7, wherein said estimating the dynamic I-frame set point according to the detection result comprises the following steps: p-frame coding is used when the image is in a changing state, and I-frame coding refresh is used when the image is in a state tending to be still.

9. The intelligent rate control optimization method based on scene change according to claim 8, wherein said estimating the dynamic I-frame set point according to the detection result comprises the following steps: the default coding frame type of the first frame is an I frame; the frame type setting condition of each subsequent frame is determined according to the coded motion vector information of the previous frame.

10. The method of claim 9, wherein the method comprises the steps of:

judging whether to set the current frame as an I frame or not according to the initial value;

if the current frame is set as an I frame, counting an accumulated value of the coded frames after the I frame, and limiting the minimum range of the interval of the I frame;

setting a judgment value to judge whether the foreground stronger motion exists;

setting a judgment value to judge whether the foreground variation tends to be gentle or not;

setting a judgment value to judge whether the next frame is set with an I frame;

and if the current frame is not set as the I frame, setting the current frame as the P frame.

Images