CN101436301A - Method for detecting characteristic movement region of video encode - Google Patents

Abstract

The invention relates to a method for detecting a character motion region in video coding. The method can rapidly detect the character motion region in a video sequence and in particular can effectively detect the character motion region which is interested by human eyes in a frame of an video image under the condition of the violent motion and complex texture of background. Firstly, a video frame is subjected to wave filtering pretreatment; secondly, a character motion equation is applied to calculate the character motion region of the frame; thirdly, an eight-direction motion model is utilized to obtain a local motion region; and finally, macroblocks of the character motion region and the local motion region are integrated to obtain the final character motion region. The method has good self-adaptability, simple algorithm and accurate detection of a target object.

Description

The detection method in characteristic kinematic zone in the video coding

Technical field

The present invention relates to the detection method in characteristic kinematic zone in a kind of video coding, the motion macro block of reflection visually-perceptible feature is effectively detected, particularly can still can detect the interested characteristic kinematic macro block of human eye under the complicated situation very much in background strenuous exercise and background texture.This method is at first carried out low-pass filtering treatment to frame of video, sets up the characteristic kinematic equation then, utilizes the eigenwert of motion vector and matrix to find out the characteristic kinematic macro block; Set up 8 direction motion models again: utilize macro block direction of motion to find out the local motion macro block; Utilize the correlativity in characteristic kinematic zone and local motion zone to obtain final characteristic kinematic zone at last.

Background technology

The motion of video sequence is very complicated, the most basic motion is meant that video camera fixes, moving object is moved with respect to background, because video camera is fixed, these backgrounds are static constant for the consecutive frame sequence, and the motion that these moving object showed is referred to as local motion.Yet in Flame Image Process, normally obtain original video sequence image by video camera, video camera is installed on the motion platform under a lot of situations, in order to catch image better, video camera itself also will be done scanning motion in full extent of space, as convergent-divergent, tangential movement, vertical motion, rotatablely move etc., so, fixing background objects is known from experience the two dimensional motion that demonstrates a kind of overall situation on the plane of delineation, is referred to as global motion.

It is the forward position direction that receives much concern in the computer vision field in recent years that moving object detects.He is in security monitoring, video conference, and human motion analysis and compression of images and content-based aspects such as image storage and retrieval have wide practical use and potential economic worth.In video image was handled, the top priority that moving object detects will split foreground target exactly from background image.Following three kinds of implementation methods are generally arranged: background subtraction method, interframe movement analytic approach and optical flow method.Background subtraction is present the most frequently used a kind of method.It utilizes the Differential Detection of present image and background image to go out the moving region.This method generally can provide characteristic the most completely, but in the utilization poor effect of dynamic scene; The interframe movement analytic approach, promptly the frame-to-frame differences point-score be to adopt between adjacent two or three consecutive frames to give the picture point time difference in the continuous images sequence, and thresholding extracts the moving region in the image.This method has stronger adaptivity for dynamic environment, but is easy to generate cavitation; Optical flow method adopts the time dependent light stream characteristic of moving target, extracts effectively and the pursuit movement target, detects target though this method is applicable under the prerequisite that motion cameras exists, yet, most of optical flow algorithm complexity, noiseproof feature is poor.

How from motion sequence, to search out our subjectivity or human eye interested moving region then be a bigger difficult problem.General way all is to utilize the time shielding effect to search out these moving regions now.The time shielding effect is variation discontinuous lifting or the decline that causes the vision threshold value in time owing to brightness.Just explanation is in frame of video for this, and when the target of scene sudden change or rapid movement, the vision threshold value will promote to some extent, and this moment, the perception of human eyes degree will reduce greatly.

Summary of the invention

The objective of the invention is to utilize the defective of prior art, the detection method in characteristic kinematic zone in a kind of video coding is provided, utilize the motion vector and the direction of macro block to seek the interested characteristic kinematic macro block of human eye, if these characteristic kinematic macro blocks are carried out perceptual coding then can significantly promote subjective visual quality do.

For achieving the above object, design of the present invention is:

As shown in Figure 1, at first frame of video is carried out low-pass filtering treatment, set up the characteristic kinematic equation then, utilize the eigenwert of motion vector and matrix to find out the characteristic kinematic macro block; Set up 8 direction motion models again: utilize the macro block travel direction to find out the local motion macro block; Utilize the correlativity in characteristic kinematic zone and local motion zone to obtain final characteristic kinematic zone at last.

According to above-mentioned design, technical scheme of the present invention is:

The detection method in characteristic kinematic zone in a kind of video coding, can be come out by fast detecting in characteristic kinematic zone in the video sequence, particularly can effectively the interested characteristic kinematic of the human eye in frame video image zone effectively be detected, it is characterized in that at first frame of video being carried out filter preprocessing, use the characteristic kinematic zone in the characteristic kinematic Equation for Calculating frame then, utilize 8 direction motion models to obtain the local motion zone again, obtain final characteristic kinematic zone behind the macro block of last comprehensive characteristics moving region and local moving region.Concrete step is as follows:

(1) frame of video is carried out pre-service: uncoded original series is carried out low-pass filtering treatment;

(2) set up the characteristic kinematic equation: utilize the eigenwert of motion vector and matrix to find out the characteristic kinematic macro block;

(3) set up 8 direction motion models: utilize macro block direction of motion to find out the local motion macro block;

(4) utilize the correlativity in characteristic kinematic zone and local motion zone to obtain final characteristic kinematic zone.

Above-mentioned four steps further specify as follows:

(1) utilize Gauss's template of 5 * 5 to carry out low-pass filtering to present frame, template is as follows:

$\left\{\begin{array}{ccccc}2& 4& 5& 4& 2\\ 4& 9& 12& 9& 4\\ 5& 12& 15& 12& 5\\ 4& 9& 12& 9& 4\\ 2& 4& 5& 4& 2\end{array}\right\}$

(2) set up the characteristic kinematic equation:

1. define one 2 * 2 matrix, formula is as follows:

$Z=\underset{y\∈\mathrm{\ω}}{\mathrm{\Σ}}\underset{x\∈\mathrm{\ω}}{\mathrm{\Σ}}\left[\begin{array}{cc}{I_x}^2& I_x{I}_y\\ I_x{I}_y& {I_y}^2\end{array}\right]---\left(1\right)$

ω represents to wish to obtain the size of characteristic area in the following formula, gets ω=16 in this method; I _x, I _yThe single order partial derivative of presentation video P on x and y direction assigns to replace the calculating of single order partial derivative with the equation of the ecentre when calculating, formula is as follows:

$I_x=\frac{\∂P(x,y)}{\∂x}=\frac{P_{\mathrm{cur}}(x+1,y)-P_{\mathrm{pre}}(x+\mathrm{mv}\_x-1,y+\mathrm{mv}\_y)}{2}---\left(2\right)$

$I_y=\frac{\∂P(x,y)}{\∂y}=\frac{P_{\mathrm{cur}}(x,y+1)-P_{\mathrm{pre}}(x+\mathrm{mv}\_x,y+\mathrm{mv}\_y-1)}{2}---\left(3\right)$

P in the following formula _Cur(x, y) coordinate (x, the y) brightness value on, the P of expression present image P _Pre(x, y) (x, the y) brightness value on, mv_x, mv_y are represented the motion vector of the level and the vertical direction of current macro to the coordinate of expression former frame image.

2. after matrix Z calculates, utilize mathematical method to calculate two eigenwerts of Z, wherein bigger value defined is λ ₁, less value is λ ₂, keeping characteristics value λ ₂

3. calculate the matrix Z of each macro block in the present frame and its minimal eigenvalue according to above-mentioned method, the minimal eigenvalue to each macro block sorts from big to small then, and the pairing macro block of preceding 20% eigenwert is the characteristic kinematic macro block.

(3) set up 8 direction motion models:

1. add up the motion vector of all macro blocks in the last coded frame, be divided into 8 directions according to the direction of motion vector, as shown in Figure 2, by counterclockwise 8 directions being numbered direction 1-direction 8 respectively.

Offset direction about 1 15 degree angles with all interior motion vectors all naturalization be the motion vector of direction 1, by that analogy, depart from all directions 15 degree with interior motion vector all naturalization be the motion vector of this direction.And add up the number of the motion macro block of this direction, the mean value that calculates all macroblock level of this direction and vertical movement size then is ave_mv_x, ave_mv_y.If the motion vector of macro block is 0, promptly this macro block is static macro block, and then this macro block of mark is direction 0 macro block.

2. add up the number of 8 direction motion macro blocks and static macro block, this moment, the motion macro block of which direction was maximum, and then we suppose that this direction is the global motion direction, and the average motion vector of this direction is a global motion vector.

3. make mv_x _i, mv_y _iBe the level of i macro block and the motion vector of vertical direction, utilize formula (4), (5) to recomputate the motion vector of macro block, ave_mv_x wherein, ave_mv_y is level and vertical global motion vector.

mv_x _i＝mv_x _i-ave_mv_x (4)

mv_y _i＝mv_y _i-ave_mv_y (5)

4. obtain recomputating the global motion direction after the new macroblock motion vector according to step 3, promptly jump to step 1 and step 2, obtain final global motion direction.

5. suppose that direction i is the global motion direction, for each macro block, if the direction of motion of this macro block is direction i or the both direction adjacent with i, then this macro block of mark is the global motion macro block, otherwise mark its be the local motion macro block.(4) method for rapidly positioning in characteristic kinematic zone in the H.264 video according to claim 1 coding is characterized in that the step that the described correlativity of utilizing characteristic kinematic zone and local motion zone obtains final characteristic kinematic zone is as follows:

1. be the center with current local motion macro block, if there is the characteristic kinematic macro block of some in 3 * 3 macro blocks around this macro block, then current local motion macro block is final characteristic kinematic macro block;

2. be the center with current non-feature macro block, if there is the local motion macro block of some in 3 * 3 macro blocks around this macro block, then current local motion macro block is final characteristic kinematic macro block.

The present invention has following conspicuous outstanding substantive distinguishing features and a bit remarkable compared with prior art:

The present invention utilizes motion feature proof equation and 8 direction motion models to unite and detects the interested object of human eye in the motion sequence, and adaptivity is relatively good, and algorithm is fairly simple, and it is more accurate that target object detects.

Description of drawings

Fig. 1 is the method for controlling self-adaption code rate FB(flow block) based on space-time shielding effect of the present invention.

Fig. 2 is from all directions to the synoptic diagram of motion model among Fig. 1.

Fig. 3 is the testing result figure in characteristic kinematic zone that adopts the football sequence of CIF form.

Fig. 4 is the testing result figure in characteristic kinematic zone that adopts the foreman sequence of CIF form.

Fig. 5 is the testing result figure in characteristic kinematic zone that adopts the coastguard sequence of CIF form.

Fig. 6 is the testing result figure in characteristic kinematic zone that adopts the deadline sequence of CIF form.

Fig. 7 is the testing result figure in characteristic kinematic zone that adopts the hall sequence of CIF form.

Fig. 8 is the testing result figure in characteristic kinematic zone that adopts the children sequence of CIF form.

Embodiment

Details are as follows in conjunction with the accompanying drawings for one embodiment of the present of invention: the detection method in characteristic kinematic zone is by flow chart shown in Figure 1 in this video coding, is that programming realizes on the PC test platform of Athlon x2 2.0GHz, internal memory 1024M at CPU.Referring to Fig. 1, the detection method in characteristic kinematic zone is at first carried out low-pass filtering treatment to frame of video in the video coding of the present invention, sets up the characteristic kinematic equation then, utilizes the eigenwert of motion vector and matrix to find out the characteristic kinematic macro block; Set up 8 direction motion models again: utilize macro block direction of motion to find out the local motion macro block; Utilize the correlativity in characteristic kinematic zone and local motion zone to obtain final characteristic kinematic zone at last.The steps include:

(1) frame of video is carried out pre-service: uncoded original series is carried out low-pass filtering treatment;

(2) set up the characteristic kinematic equation: utilize the eigenwert of motion vector and matrix to find out the characteristic kinematic macro block;

(3) set up 8 direction motion models: utilize macro block direction of motion to find out the local motion macro block;

(4) utilize the correlativity in characteristic kinematic zone and local motion zone to obtain final characteristic kinematic zone.

The pretreated process of frame of video of above-mentioned steps (1) is as follows:

1. utilize Gauss's template of 5 * 5 to carry out gaussian filtering to present frame, template is as follows.

$\left\{\begin{array}{ccccc}2& 4& 5& 4& 2\\ 4& 9& 12& 9& 4\\ 5& 12& 15& 12& 5\\ 4& 9& 12& 9& 4\\ 2& 4& 5& 4& 2\end{array}\right\}$

The process of the sensor model of above-mentioned steps (2) is as follows:

1. define one 2 * 2 matrix, formula is as follows:

$Z=\underset{y\∈\mathrm{\ω}}{\mathrm{\Σ}}\underset{x\∈\mathrm{\ω}}{\mathrm{\Σ}}\left[\begin{array}{cc}{I_x}^2& I_x{I}_y\\ I_x{I}_y& {I_y}^2\end{array}\right]---\left(1\right)$

ω represents to wish to obtain the size of characteristic area in the following formula, gets ω=16 in this method; I _x, I _yThe single order partial derivative of presentation video P on x and y direction assigns to replace the calculating of single order partial derivative with the equation of the ecentre when calculating, formula is as follows:

$I_x=\frac{\∂P(x,y)}{\∂x}=\frac{P_{\mathrm{cur}}(x+1,y)-P_{\mathrm{pre}}(x+\mathrm{mv}\_x-1,y+\mathrm{mv}\_y)}{2}---\left(2\right)$

$I_y=\frac{\∂P(x,y)}{\∂y}=\frac{P_{\mathrm{cur}}(x,y+1)-P_{\mathrm{pre}}(x+\mathrm{mv}\_x,y+\mathrm{mv}\_y-1)}{2}---\left(3\right)$

P in the following formula _Cur(x, y) coordinate (x, the y) brightness value on, the P of expression present image P _Pre(x, y) (x, the y) brightness value on, mv_x, mv_y are represented the motion vector of the level and the vertical direction of current macro to the coordinate of expression former frame image.

2. after matrix Z calculates, utilize mathematical method to calculate two eigenwerts of Z, wherein bigger value defined is λ ₁, less value is λ ₂, keeping characteristics value λ ₂

3. calculate the matrix Z of each macro block in the present frame and its minimal eigenvalue according to above-mentioned method, the minimal eigenvalue to each macro block sorts from big to small then, and the pairing macro block of preceding 20% eigenwert is the characteristic kinematic macro block.The process of the method for controlling self-adaption code rate of above-mentioned steps (3) is as follows:

1. add up the motion vector of all macro blocks in the last coded frame, be divided into 8 directions according to the direction of motion vector, as shown in Figure 2, by counterclockwise 8 directions being numbered direction 1-direction 8 respectively.

Offset direction about 1 15 degree angles with all interior motion vectors all naturalization be the motion vector of direction 1, by that analogy, depart from all directions 15 degree with interior motion vector all naturalization be the motion vector of this direction.And add up the number of the motion macro block of this direction, the mean value that calculates all macroblock level of this direction and vertical movement size then is ave_mv_x, ave_mv_y.If the motion vector of macro block is 0, promptly this macro block is static macro block, and then this macro block of mark is direction 0 macro block.

2. add up the number of 8 direction motion macro blocks and static macro block, this moment, the motion macro block of which direction was maximum, and then we suppose that this direction is the global motion direction, and the average motion vector of this direction is a global motion vector.

3. make mv_x _i, mv_y _iBe the level of i macro block and the motion vector of vertical direction, utilize formula (4), (5) to recomputate the motion vector of macro block, ave_mv_x wherein, ave_mv_y is level and vertical global motion vector.

mv_x _i＝mv_x _i-ave_mv_x (4)

mv_y _i＝mv_y _i-ave_mv_y (5)

4. obtain recomputating the global motion direction after the new macroblock motion vector according to step 3, promptly jump to step 1 and step 2, obtain final global motion direction.

5. suppose that direction i is the global motion direction, for each macro block, if the direction of motion of this macro block is direction i or the both direction adjacent with i, then this macro block of mark is the global motion macro block, otherwise mark its be the local motion macro block.

The process of the method for controlling self-adaption code rate of above-mentioned steps (4) is as follows:

1. be the center with current local motion macro block, if there is the characteristic kinematic macro block of some in 3 * 3 macro blocks around this macro block, then current local motion macro block is final characteristic kinematic macro block;

2. be the center with current non-characteristic area, if there is the local motion macro block of some in 3 * 3 macro blocks around this macro block, then current local motion macro block is final characteristic kinematic macro block.

Example when below providing the input video form and be 352 * 288 CIF adopts the H.264 scrambler of JM10.2 version that standard test sequences is encoded.H.264 the configuration of scrambler is as follows: Baseline Profile, and IPPP, per 15 frames insert 1 I frame, 1 reference frame, bandwidth is set to 256k bps, and frame per second is set to 30fps, and the initial quantization parameter is set to 32.

Adopt the CIF of typical standard test sequences 352 * 288 to test as input video, these sequences had both been had powerful connections complicated, and it is static also to have powerful connections.By these testing results of Fig. 3 to Fig. 8 as can be seen, this method all has the quite good detecting result to stationary background and the sequence with big global motion.

Claims (5)

1. the detection method in characteristic kinematic zone during a video is encoded, can be come out by fast detecting in characteristic kinematic zone in the video sequence, particularly can effectively the interested characteristic kinematic of the human eye in frame video image zone effectively be detected, it is characterized in that at first frame of video being carried out filter preprocessing, use the characteristic kinematic zone in the characteristic kinematic Equation for Calculating frame then, utilize 8 direction motion models to obtain the local motion zone again, obtain final characteristic kinematic zone behind the macro block of last comprehensive characteristics moving region and local moving region.Concrete step is as follows:

(1) frame of video is carried out pre-service: uncoded original series is carried out low-pass filtering treatment;

(2) set up the characteristic kinematic equation: utilize the eigenwert of motion vector and matrix to find out the characteristic kinematic macro block;

(3) set up 8 direction motion models: utilize macro block direction of motion to find out the local motion macro block;

(4) utilize the correlativity in characteristic kinematic zone and local motion zone to obtain final characteristic kinematic zone.

2. the detection method in characteristic kinematic zone in the video according to claim 1 coding, it is characterized in that described step (1) frame of video is carried out pretreated method is to utilize Gauss's template of 5 * 5 to carry out low-pass filtering to present frame, template is as follows:

$\left\{\begin{array}{ccccc}2& 4& 5& 4& 2\\ 4& 9& 12& 9& 4\\ 5& 12& 15& 12& 5\\ 4& 9& 12& 9& 4\\ 2& 4& 5& 4& 2\end{array}\right\}.$

3. the detection method in characteristic kinematic zone in the video coding according to claim 1 is characterized in that the step of setting up the characteristic kinematic equation of described step (2) is:

1. define one 2 * 2 matrix, formula is as follows:

$Z=\underset{y\∈\mathrm{\ω}}{\mathrm{\Σ}}\underset{x\∈\mathrm{\ω}}{\mathrm{\Σ}}\left[\begin{array}{cc}{I_x}^2& I_x{I}_y\\ I_x{I}_y& {I_y}^2\end{array}\right]---\left(1\right)$

ω represents to wish to obtain the size of characteristic area in the following formula, gets ω=16; I _x, I _yThe single order partial derivative of presentation video P on x and y direction assigns to replace the calculating of single order partial derivative with the equation of the ecentre when calculating, formula is as follows:

$I_x=\frac{\∂P(x,y)}{\∂x}=\frac{P_{\mathrm{cur}}(x+1,y)-P_{\mathrm{pre}}(x+\mathrm{mv}\_x-1,y+\mathrm{mv}\_y)}{2}---\left(2\right)$

$I_y=\frac{\∂P(x,y)}{\∂y}=\frac{P_{\mathrm{cur}}(x,y+1)-P_{\mathrm{pre}}(x+\mathrm{mv}\_x,y+\mathrm{mv}\_y-1)}{2}---\left(3\right)$

P in the following formula _Cur(x, y) coordinate (x, the y) brightness value on, the P of expression present image P _Pre(x, y) coordinate of expression former frame image (x, the y) brightness value on, mv_x, mv_y are represented the motion vector of the level and the vertical direction of current macro;

2. after matrix Z calculates, utilize mathematical method to calculate two eigenwerts of Z, wherein bigger value defined is λ ₁, less value is λ ₂, keeping characteristics value λ ₂

3. calculate the matrix Z of each macro block in the present frame and its minimal eigenvalue according to above-mentioned method, the minimal eigenvalue to each macro block sorts from big to small then, and the pairing macro block of preceding 20% eigenwert is the characteristic kinematic macro block.

4. the detection method in characteristic kinematic zone in the video coding according to claim 1 is characterized in that the step of setting up 8 direction motion models of described step (3) is as follows:

1. add up the motion vector of all macro blocks in the last coded frame, be divided into 8 directions, by counterclockwise 8 directions being numbered direction 1-direction 8 respectively according to the direction of motion vector;

Offset direction about 1 15 degree angles with all interior motion vectors all naturalization be the motion vector of direction 1, by that analogy, depart from all directions 15 degree with interior motion vector all naturalization be the motion vector of this direction; And add up the number of the motion macro block of this direction, the mean value that calculates all macroblock level of this direction and vertical movement size then is ave_mv_x, ave_mv_y; If the motion vector of macro block is 0, promptly this macro block is static macro block, and then this macro block of mark is direction 0 macro block;

2. add up the number of 8 direction motion macro blocks and static macro block, this moment, the motion macro block of which direction was maximum, and then we suppose that this direction is the global motion direction, and the average motion vector of this direction is a global motion vector;

3. make mv_x _i, mv_y _iBe the level of i macro block and the motion vector of vertical direction, utilize formula (4), (5) to recomputate the motion vector of macro block, ave_mv_x wherein, ave_mv_y is level and vertical global motion vector.

mv_x _i＝mv_x _i-ave_mv_x (4)

mv_y _i＝mv_y _i-αve_mv_y (5)

4. 3. obtain recomputating the global motion direction after the new macroblock motion vector according to step, promptly jump to step 1. with step 2., obtain final global motion direction;

5. suppose that direction i is the global motion direction, for each macro block, if the direction of motion of this macro block is direction i or the both direction adjacent with i, then this macro block of mark is the global motion macro block, otherwise mark its be the local motion macro block.

5. the detection method in characteristic kinematic zone in the video according to claim 1 coding, the step that the correlativity of utilizing characteristic kinematic zone and local motion zone that it is characterized in that described step (3) obtains final characteristic kinematic zone is as follows:

1. be the center with current local motion macro block, if there is the characteristic kinematic macro block of some in 3 * 3 macro blocks around this macro block, then current local motion macro block is final characteristic kinematic macro block;

2. be the center with current non-feature macro block, if there is the local motion macro block of some in 3 * 3 macro blocks around this macro block, then current local motion macro block is final characteristic kinematic macro block.

Images