CN102665033A - Real time digital video image-stabilizing method based on hierarchical block matching - Google Patents

Abstract

The invention discloses a real time digital video image-stabilizing method based on hierarchical block matching. The method comprises the following steps: at first, a down-sampling processing on an input image is conducted; then full search of M number of search blocks in a down-sampling image is conducted in the setting range; the optimal motion vector is selected from the search blocks of the down-sampling image and output; further perfect and full search is further conducted in the original image according to the optimal motion vector of the down-sampling image, and a final motion vector of the original image is output; a motion trajectory of the original image is subjected to a low-pass smoothing filtering, and compensation for the image is calculated; a current image is compensated according to the compensation for the image. The real time digital video image-stabilizing method based on hierarchical block matching has the advantages that the method has high real-time property; overall motion and local motion can be distinguished effectively; and the method can adapt to various vibration frequencies and amplitudes, and is strong in robustness.

Description

Real-time digital video image stabilization method based on the hierarchical block coupling

Technical field

The present invention relates generally to a kind of image processing method, specifically refers to a kind of real-time digital video image stabilization method based on the hierarchical block coupling.

Background technology

Along with picture pick-up device use increasingly extensive, it is more and more diversified that platform for video camera becomes.The vision signal that obtains through vehicle-mounted pick-up system, hand-held photographic equipment, aircraft or naval vessel photography platform etc. tends to because the motion of video camera is shaken.Video jitter not only can influence visual effect, makes the beholder produce dizzy easily and then causes erroneous judgement and fail to judge, and bring difficulty for further image processing.How these vision signals are converted into high-quality stable video and have important significance for theories and practical value.

Surely divide as principle by video, commonly used surely can be divided three classes as device: mechanical type is picture surely, and optics surely looks like and electronic steady image.Electronic steady image is as a kind of novel steady picture technology, than the steady picture of traditional gyroscope, damping device surely as and optics surely have simple in structurely as waiting, equipment volume is little, price is low, the little grade of power consumption clear superiority.

The algorithm of electronic steady image system comprises block matching algorithm, representative point matching algorithm, Gray Projection algorithm, bit plane matching algorithm, optical flow field method, frequency domain estimation technique or the like.Wherein block matching algorithm is realized simple with algorithm; Complexity is low; The commercial Application degree extensively is celebrated; But there are following 2 subject matters in traditional block matching algorithm: at first being the contradiction between search precision and algorithm time complexity, is the differentiation problem of video image global motion and local motion secondly.

Summary of the invention

The present invention is directed to the problem of the steady picture of conventional block coupling algorithm, a kind of real-time digital video image stabilization method based on the hierarchical block coupling is provided.

It comprises the steps: that at first input luminance component image being carried out down-sampling handles; Then M search block in the luminance component down-sampled images carried out the full search of setting range, export each search block motion vector and SAD statistical information; Judge its motion vector validity according to each search block SAD statistical information of luminance component down-sampled images; To effective search block movable information statistics " direction of motion " and " motion size " similarity, choose the similarity soprano again as optimum movement vector and output; Luminance component down-sampled images optimum movement vector after luminance component original image equal proportion is amplified, is further done the setting range refinement and searched for entirely, and output luminance component original image final motion vector; To the at interval interior luminance component original image movement locus low pass smothing filtering of certain hour, and calculate output final image compensation rate; According to the final image compensation rate present image is compensated;

With the existing steady picture of coupling compared with techniques, the invention has the advantages that: 1, computation complexity is low.Adopt layering to mate soon, extensive search is operated in the down-sampled images carries out, and original image only carries out fine search among a small circle, significantly reduces computation complexity; 2, search precision is high.In down-sampled images and original image, all adopt full search, guarantee the motion vector accuracy; 3, effectively distinguish global motion and local motion, improve the subjective and objective performance of steady picture.Choose suitable search block size, and choose optimum down-sampled images motion vector through SAD statistical information and motion vector similarity degree.

Description of drawings

Fig. 1 is the real-time digital video image stabilization method overall flow figure based on the hierarchical block coupling,

Fig. 2 chooses flow chart for the down-sampled images optimum movement vector;

Fig. 3 is a FIR LPF sketch map;

Embodiment

Below will combine accompanying drawing and specific embodiment that the present invention is explained further details.As shown in Figure 1, the real-time digital video image stabilization method based on the hierarchical block coupling of the present invention may further comprise the steps:

The first step, the input picture preliminary treatment.

In level and vertical direction the input image lightness component is carried out N times of down-sampling processing respectively.

Second step, the down-sampled images motion search.

In down-sampled images, respectively M search block carried out the full search of setting range, in search procedure, add up the SAD relevant information, and export with motion vector.Concrete steps are following:

Step 1 is searched for each search block in reference to down-sampled images entirely.

If search window is wide is SW, and height is SH, and search block is wide to be BW, and height is BH, and total number of positions to be searched is SCAND, and the search block interior pixel is counted out and is PNUM, then

SCAND＝SW×SH

PNUM＝BW×BH。

Full search block matching criterior is SAD:

$\mathrm{SAD}=\underset{i=0}{\overset{N}{\mathrm{\Σ}}}\underset{j=0}{\overset{M}{\mathrm{\Σ}}}({\mathrm{cur}}_{i,j}-{\mathrm{ref}}_{i,j})$

N wherein, M is the wide height of search block, and cur is the present frame search block, and ref is the respective reference frame search block.

Step 2, statistics SAD information A VAIL in full search procedure _i, i=0,1,2,3 ... M, i are the search block index.If T0 is a natural number, CURSAD is a search block current location sad value, and the minimum sad value of search block is MINSAD, then

MINSAD＝PNUM×PNUM。

Before the full search of beginning, AVAIL _iBe initialized as 0.Each search block when traversal reference picture SCAND searching position, as if current location CURSAD smaller or equal to MINSAD, then corresponding AVAIL _iAdd fixed increment, otherwise, corresponding AVAIL _iRemain unchanged.

The 3rd step, the down-sampled images extraction of motion information.

As shown in Figure 2; At first judge its motion vector validity according to M search block SAD of luminance component down-sampled images statistical information; To effective search block statistics motion vector similarity, choose the similarity soprano again as luminance component down-sampled images optimum movement vector and output.Concrete steps are following:

Step 1 is judged each search block movable information validity according to the SAD statistical information of luminance component down-sampled images motion search output.

If T1 is a natural number, and 0＜=T1＜=SCAND, if the AVAIL of search block i _iThen this search block motion vector is effective less than T1, otherwise this search block motion vector is invalid.Search block with efficient motion-vector is called active block, otherwise, be called invalid block.

Step 2 is carried out motion vector correlation statistics to active block.

If the horizontal motion vector of active block i is Xi; The vertical motion vector of active block i is Yi; SIMXi is that the motion vector horizontal component is the active block number of Xi in M search block of luminance component down-sampled images; SIMYi is that the motion vector vertical component is the active block number of Yi in M search block of luminance component down-sampled images, and the motion vector similarity of active block i is SIMi, then

SIMi＝SIMXi+SIMYi。

Ask for the motion vector similarity of all active blocks, getting the maximum active block block motion vector of similarity is luminance component down-sampled images optimal motion vector and output.

The 4th step, the original image fine search.

Luminance component down-sampled images optimum movement vector after luminance component original image equal proportion is amplified, is carried out the full search of refinement and the output original image final motion vector of setting range to the corresponding original image search block of luminance component image optimum motion vector active block.

The 5th step, movement locus filtering.

The at interval interior luminance component original image movement locus of certain hour is done the low pass smothing filtering, and calculate output final image compensation rate, as shown in Figure 3.Concrete steps are following:

Step 1: movement locus statistics.

If i; N is a natural number, expression input picture frame number, and the n frame original image terminal level motion vector of above-mentioned the 4th step output is Xn; The final vertical motion vector of n frame original image of above-mentioned the 4th step output is Yn; The i frame is SUMXi with respect to the horizontal-shift position of the 0th frame, and the i frame is SUMYi with respect to the vertical shift position of the 0th frame, then

$\mathrm{SUMXi}=\underset{n=0}{\overset{i}{\mathrm{\Σ}}}\mathrm{Xn}$

$\mathrm{SUMYi}=\underset{n=0}{\overset{i}{\mathrm{\Σ}}}\mathrm{Yn}.$

Step 2: movement locus filtering.

If T2 is a natural number, FIR _T2For the rank umber of beats is the low pass filter of T2, i two field picture terminal level compensation rate is MCxi, and the final VCP amount of i two field picture is MCyi, then

MCxi＝SUMXi-FIR _T2(SUMX _i-T2，SUMX _i-T2+1，SUMX _i-T2+2，......，SUMX _i)

MCyi＝SUMYi-FIR _T2(SUMY _i-T2，SUMY _i-T2+1，SUMY _i-T2+2，......，SUMY _i)。

The 6th step, motion compensation.

According to horizontal compensation rate MCxi of final image and final image VCP amount MCyi current image frame is carried out the translation compensation.

Claims (4)

1. based on the real-time digital video image stabilization method of hierarchical block coupling, it may further comprise the steps:

Step (1): input picture preliminary treatment;

In level and vertical direction input luminance component image is done N times of down-sampling processing respectively;

Step (2): down-sampled images motion search;

Respectively M search block of luminance component down-sampled images carried out the full search of setting range, and output movement vector and SAD statistical information;

Step (3): down-sampled images extraction of motion information;

Judge its motion vector validity according to each search block SAD statistical information of luminance component down-sampled images, to effective search block statistics motion vector similarity, choose the similarity soprano and export again as luminance component down-sampled images optimum movement vector;

Step (4): original image fine search;

Luminance component down-sampled images optimum movement vector after luminance component original image equal proportion is amplified, is done the setting range refinement at the luminance component original image and searched for entirely, and output luminance component original image final motion vector;

Step (5): movement locus filtering;

The at interval interior luminance component original image movement locus of certain hour is done the low pass smothing filtering, and calculate output final image compensation rate;

Step (6): motion compensation;

According to the final image compensation rate present image is compensated.

2. according to the said real-time digital video image stabilization method based on the hierarchical block coupling of claim 1, it is characterized in that: said step (2) down-sampled images motion search specifically may further comprise the steps:

Step 1: each search block is searched for entirely;

If search window is wide is SW, and height is SH, and search block is wide to be BW, and height is BH, position to be searched add up to SACND, the search block interior pixel is counted out and is PNUM, then

SCAND＝SW×SH

PNUM＝BW×BH；

Step 2: statistics SAD information A VAIL in full search procedure _i, i=0,1,2,3 ..., M, i is the search block index, and establishing T0 is natural number, and CURSAD is a search block current location sad value, and the minimum SAD threshold value of search block is MINSAD, then

MINSAD＝PNUM×PNUM；

Each search block is in traversal during SCAND searching position, as if current location CURSAD smaller or equal to MINSAD, then corresponding AVAIL _iAdd fixed increment, otherwise, corresponding AVAIL _iRemain unchanged.

3. according to the said real-time digital video image stabilization method based on the hierarchical block coupling of claim 1, it is characterized in that: said step (3) down-sampled images extraction of motion information specifically may further comprise the steps:

Step 1: the SAD statistical information according to the output of luminance component down-sampled images motion search is judged each search block motion vector validity;

If T1 is a natural number, and 0＜=T1＜=SCAND, if the AVAIL of search block i _iThen this search block motion vector is effective less than T1, otherwise this search block motion vector is invalid.Search block with efficient motion-vector is called active block, otherwise, be called invalid block;

Step 2: active block is carried out motion vector correlation statistics:

If the horizontal motion vector of active block i is Xi; The vertical motion vector of active block i is Yi; SIMXi is that the motion vector horizontal component is the active block number of Xi in M search block of luminance component down-sampled images; SIMYi is for being that the motion vector vertical component is the active block number of Yi in M search block of luminance component down-sampled images, and the motion vector similarity of active block i is SIMi, then

SIMi＝SIMXi+SIMYi；

Ask for the motion vector similarity of whole active blocks, getting the maximum active block motion vector of motion vector similarity is luminance component down-sampled images optimal motion vector.

4. according to the said real-time digital video image stabilization method based on the hierarchical block coupling of claim 1, it is characterized in that: said step (5) movement locus filtering specifically may further comprise the steps:

Step 1: movement locus statistics;

If i; N is a natural number, expression input picture frame index, and the n frame original image terminal level motion vector of claim 1 step (4) output is Xn; The final vertical motion vector of n frame original image of claim 1 step (4) output is Yn; The i frame is SUMXi with respect to the horizontal-shift position of the 0th frame, and the i frame is SUMYi with respect to the vertical shift position of the 0th frame, then

Step 2: movement locus filtering;

If T2 is a natural number, FIR _T2For the rank umber of beats is the low pass filter of T2, i two field picture terminal level compensation rate is MCxi, and the final VCP amount of i two field picture is MCyi, then

MCxi＝SUMXi-FIR _T2(SUMX _i-T2，SUMX _i-T2+1，SUMX _i-T2+2，......，SUMX _i)

MCyi＝SUMYi-FIR _T2(SUMY _i-T2，SUMY _i-T2+1，SUMY _i-T2+2，......，SUMY _i)。