CN104506869A - Method for motion estimation of video sequences based on block matching under different resolutions - Google Patents

Abstract

The invention relates to a method for motion estimation of video sequences based on block matching under different resolutions. According to the comparison under a single resolution, the method is utilized to perform the motion estimation performance comparison of an international standard video sequence under two resolutions at a quick speed, a medium speed and a slow speed on the basis of three classical algorithms including full search, three-step search and new three-step search and the traditional matching law SAD, namely the search block number, search time, peak signal to noise ratio (PSNR) and reconstruction images are compared. The important conclusion of the method is that video sequences at a medium speed and a slow speed have a good performance under low resolutions; on the contrary, quick video sequences have good motion estimation effect under the high resolution. The conclusion obtained through the method overcomes the shortcomings of motion estimation under the different resolutions, more suitable searching methods for video sequences of different characteristics under the different resolutions are achieved, and the result of the method can further be applied to the aspects of object identification, motion tracking and video compression.

Description

Based on the method for estimating of Block-matching video sequence under different resolution

Technical field

The present invention relates to technical field of image processing, is a kind of method for estimating based on Block-matching video sequence under different resolution.

Background technology

Along with the fast development of society, multimedia technology has enriched the life of people.The development that video image is very fast in people's life, such as: network visual telephone, Digital Television etc.But the bulk information comprised in video image causes transmitting the very high bandwidth of needs.In order to address this problem, " video compression technology " can improve propagation efficiency under limited bandwidth, and therefore, the redundancy reduced on video time is very necessary.In video compression, block-based motion estimation is a kind of method be commonly used to for reducing Video Redundancy.In fact, the amount of calculation of estimation will exceed 50% of whole compression amount of calculation, this means that the quality of estimation directly determines quality and the Video coding of motion compensation.Common method for estimating has " optical flow method ", " block matching algorithm (BMA) ", " PRA " and " bayesian algorithm ".BMA is usually used in different video compression standards.In addition, block matching algorithm accepted by MPEG, H261, H263 and other different types of videos widely.Although block matching algorithm is widely used, but, existing method for estimating is normally for realizing to save search time, improve for the purpose of search precision, in addition, existing method for estimating only compares single resolution, lacks the comparison of motion-estimation precision under different resolution, therefore, different resolution lower difference showing estimation characteristic is not cognitive.

Up to now, have no about pin: make supplementary to the experiment of classic algorithm and conclusion thereof, " three kinds of different video sequences " Performance comparision under " different resolution " is proposed, by bibliographical information and the practical application of high, medium and low three kinds of international standard video sequences method for estimating under two kinds of conventional resolution (640 × 480,176 × 144).

Summary of the invention

The present invention seeks to, substantial improvements is carried out to method for estimating under existing single resolution, a kind of method for estimating based on Block-matching video sequence under different resolution is proposed, the method can for different resolution and video sequence characteristics, more clearly determine searching algorithm accurately, for block search principle is made supplementary.

Realizing the technical scheme that the object of the invention adopts is: a kind of method for estimating based on Block-matching video sequence under different resolution, and it has in comprising:

A () block motion compensation: be that picture is divided into some pieces, uses coupling rule, finds the block of previous frame in this frame position;

(b) search window: be the scope of determining to search for, each frame has the macro block of M × N, each macro block needs will find best matching blocks in region of search, and moving target determines the size of search window, and the size of search window should slightly larger than the moving object having maximum likelihood;

C () searches for entirely: be FS, obtains best Searching point by searching for all candidate blocks in consecutive frame, and the search factor of FS is P=7, and therefore, each block needs search 225 points;

D () three-wave mixing: be TSS, uses " three steps " to find optimal match point exactly, in addition, this is a searching method from coarse to meticulous, is easy to be absorbed in Local Minimum, is applicable to the video sequence larger to motion amplitude and estimates;

(e) new three-wave mixing: be NTSS, be propose on the basis of TSS, and introduce the method for center-biased, therefore, higher than TSS efficiency, be applicable to the search in the little or static video sequence of motion amplitude;

(f) matching criterior: find best matching blocks with " cost function " exactly, this process can carry out computation of match errors by given price value function, wherein minimal error match block is exactly best matching blocks, matching criterior not only has impact to precision, and also have impact to the speed of estimation, the amount of calculation of estimation is determined by the complexity of search point and matching criterior, and in order to weigh both precision and speed, matching criterior is a key factor that must consider;

G () evaluates the precision of estimation with PSNR: Y-PSNR is used for the objective estimation of picture quality, be used for especially estimating the quality of reconstructed image, be usually used in video compression, usually, after video compression, output video and original video have some differences, and in order to the video quality after measurement processing, the value of PSNR can reflect the visual effect whether this process meets the mankind; It is characterized in that, also comprise:

H () selects three kinds of international standard video sequences: the video sequence selecting the different resolution of three kinds of features, be respectively, Girl video sequence, traditional Talk-heading video sequence, background is motionless, prospect only has a small amount of motion, and Caltrain is the medium video sequence of image quality exquisiteness, and Football is then video sequence violent fast;

I () preliminary treatment: be that video sequence is changed framing, becomes gray scale picture the colour picture of each frame by Matlab program batch process;

The comparison of motion estimation performance under (j) different resolution: select two kinds of conventional resolution, to the search time of three kinds of velocity standard video sequences under different resolution classic algorithm different from three kinds, search block number, PSNR, as follows from the past different item compareds:

1) more same video sequence mean P SNR under different resolution obtains a result;

2) the poorest frame under high-low resolution is observed, i.e. PSNR minimum point;

3) comparison of the lower three kinds of classic algorithm mean P SNR of QCIF video sequence;

4) reconstructed image of the poorest frame under the reconstructed image of the poorest frame under Caltrain high-resolution and low resolution is observed;

K () motion estimation performance index is averaged: each group estimation all can produce last set block and PSNR, in order to weigh the estimation effect of whole video, needs to get average to search block and PSNR.

Method for estimating based on Block-matching video sequence under different resolution of the present invention, its motion estimation quality is by PSNR, and CPU time and search block number determine; Block motion estimation selects the block of 16 × 16 to search in the search window of P=7, the block at the minimum value place that SAD calculates is then best matching blocks, vector shift is produced between match block and reference frame block, reconstructed image obtains according to reference picture and vector shift, can have difference with original image, and the similarity degree PSNR of reconstructed image and original image calculates, PSNR is larger, and proof is more similar, namely estimation effect is better, and time shorter proof coupling is estimated faster; Comparison under two groups of resolution, its experimental result is for have circumscribed classic algorithm now, and namely TSS, NTSS and FS algorithm has been made and having been supplemented, and its supplementary conclusion is image recognition, and the aspects such as compression bring facility.

Accompanying drawing explanation

Fig. 1 is block motion compensation figure;

Fig. 2 is region of search figure;

Fig. 3 is full search helical scanning schematic diagram;

Fig. 4 is three-wave mixing schematic diagram;

Fig. 5 is new three-wave mixing schematic diagram;

Fig. 6 is average search block number schematic diagram under Football high-resolution;

Fig. 7 is PSNR schematic diagram under Football high-resolution;

Fig. 8 is the search block number schematic diagram under Football low resolution;

Fig. 9 is the PSNR schematic diagram under Football low resolution;

Figure 10 is average search block number under Caltrain high-resolution;

Figure 11 is PSNR schematic diagram under Caltrain high-resolution;

Figure 12 is the search block number schematic diagram under Caltrain low resolution;

Figure 13 is the PSNR schematic diagram under Caltrain low resolution;

Figure 14 is average search block number schematic diagram under Girl high-resolution;

Figure 15 is PSNR schematic diagram under Girl high-resolution;

Figure 16 is the search block number schematic diagram under Girl low resolution;

Figure 17 is the PSNR schematic diagram under Girl low resolution;

Figure 18 is the reconstruct schematic diagram at 25 frames under Caltrain sequence high-resolution;

Figure 19 is the reconstruct schematic diagram at 25 frames under Caltrain sequence low resolution.

Embodiment

Method for estimating based on Block-matching video sequence under different resolution of the present invention, comprises following content:

A () block motion compensation: each frame first will be divided into some pieces by block motion compensation, then to take exercises compensation to each block.

With reference to Fig. 1, the coupling that estimation need search for current block in reference frame is fast.Current block need in the region of search of reference frame (usual current block is as search center), and then, all pieces in comparison domain, find least residual block (best matching blocks), block size is 16 × 16.

(b) search window: search window) size should slightly larger than the moving object having maximum likelihood.The step-length of search window can be given in reference frame.In order to find optimum movement vector, best search window step-length should be the size of 1 pixel.

With reference to Fig. 2, each frame is divided into the macroblock size of M × N, and each macro block needs will find best matching blocks in the S of region of search.Wherein, region of search is S, and search window coefficient is S=(2p+16+1) ².

C () searches for entirely: full search has two kinds of scan methods, and one is " helical scanning ", and another kind is " raster scan ", and this experiment adopts the helical scanning of full search.

With reference to Fig. 3, be helical scanning, this be one typically from as far as near scan method.It starts from the center (0,0) of search window, and according to the candidate blocks that the order removal search of helical scanning is all, given price value function, finds optimal match point, but this method is better for the object performance of low frequency movement.Motion vector is distributed in moving object and surrounding thereof usually, and therefore, the feature of misalignment is that this method performs to the best, therefore accelerates the operation of program.

D () three-wave mixing: be TSS, with reference to Fig. 4, is the step of three-wave mixing.

1) given price value function, finds optimal match point (select an applicable cost function to do matching primitives to current block and reference block, the minimum value calculated is exactly best matching blocks, namely " point 1 " in figure).

2) use optimal match point 1 as center, the step before repetition, reduce step size to 2, and search for 8 candidate blocks, obtain minimum match error point, find best matching blocks, as shown in " point 2 ".

3) in this process, centered by " point 2 ", step-length is reduced to 1, search 8 points around, finds impact point-" point 3 " after calculating minimum match error point.Evaluate: the search point of TSS is 9+8+8, namely 25 steps.Compare with 255 points of FS, this method is highly effective to acceleration, and on the other hand, TSS first step model unalterable, makes it show under little estimation poor, be easily absorbed in Local Minimum.

(e) new three-wave mixing: being NTSS, with reference to Fig. 5, is the step of new three-wave mixing.

The first step: NTSS search starts from the center (0,0) of search window.The first step searches for 17 points, respectively: central point, and outer 8 points (step-length is 4) and middle 8 points (step-length is 1).

If 1. minimum match error is central point, then stop search.

If 2. minimal error point is a bit around search center in interior 8, centered by this point, search for the point around it.

As shown in the figure, if the first step around the corner, then (5 triangles on figure are exactly the point that will search for search for 5 points of surrounding.Here 8 points need not be searched for, because wherein three complete search in the first step).If this point is a bit in transverse and longitudinal, then search for three points around it, as shown in three little squares.Meanwhile, minimal error match point is exactly the optimum point that will search for.

If 3. minimal error coupling is one of them of outer 8 points, then according to the method removal search of TSS, then according to carrying out second step.

Second step: using minimal error match point as center, step-length is reduced to half, with step length searching 8 points upgraded, calculates minimal error with cost function.

3rd step: repeat second step, step-length is reduced to 1, calculates the value of 8 points, afterwards, finds smallest match point.This point is exactly the optimal match point of whole search.

Evaluate: usually, TSS needs 25 Searching point, NTSS only needs 17 Searching point at its best, but needs 33 Searching point (probability is low) in the worst case.The NTSS of center-biased not just improves the speed of block search, and reduces the possibility being absorbed in Local Minimum.In addition, the employing of termination techniques greatly reduces search complexity, and improves search efficiency.Large quantity research proves that NTSS can reduce more error of compensation, and robustness is better than TSS.

(f) matching criterior: matching criterior is the standard of measurement two sub-block similarity degrees.SAD is conventional matching criterior, and optimal match point is the minimum value of SAD, as shown in formula (1).By calculating the minimum value of present frame and reference frame, best matching blocks can be found.

$\mathrm{SAD}(i,j)=\underset{m=0}{\overset{M}{\mathrm{\Σ}}}\underset{n=0}{\overset{N}{\mathrm{\Σ}}}|f_k(m,n)-f_{k-1}(m+i,n+j)|---\left(1\right)$

G () evaluates the precision of estimation with PSNR: Y-PSNR is used for evaluating the objective standard of reconstructed image and original image gap.But it can not meet human vision effect in a way.Y-PSNR PSNR is as formula (2), and MSE is as shown in formula (3).

$\mathrm{PSNR}=10\log \frac{{255}^2}{\mathrm{MSE}}---\left(2\right)$

$\mathrm{MSE}(i,j)=\frac{1}{\mathrm{MN}}\underset{m=0}{\overset{M}{\mathrm{\Σ}}}\underset{n=0}{\overset{N}{\mathrm{\Σ}}}{[f_k(m,n)-f_{k-1}(m+i,n+j)]}^2---\left(3\right)$

As formula (1), the value of MSE determines the value of PSNR.Square very little when two frame mean differences, the value of PSNR will be very high, if two images very similar, even can overlap, in theory, PSNR is an infinitely-great number.In fact, not only there is the object of movement in video, also there is interference, this means that PSNR can not reach " infinity ".

H () selects three kinds of international standard video sequences: the International video sequence selecting three kinds of friction speeds and feature, is divided into according to speed, high, normal, basic, is divided into background to move according to content, and background is fine and smooth, stationary background.Therefore Football, Caltrain and Girl video sequence has been selected.

(i) preliminary treatment: preliminary treatment to look, a frequency sequence point framing is extracted, and gray scale picture that colour picture is programmed.

Resolutions form is adopted to test.Be a picture for high-resolution, specification is 640 × 480, and another kind is low resolution, and specification is 176 × 144, is color video, therefore, needs preliminary treatment.Preprocessing process is as follows.

1) standard video sequence is selected to test.

2) framing is divided after whole video being passed through MATLAB routine processes.

3) each video sequence extracts 30 frames.

4) colour picture often organized is converted to gray scale picture.

Often organize video sequence and extract 30 frames, the same video sequence under Resolutions should extract same number of frames.Color image sequence is processed into gray scale picture in Matlab.Reduce amount of calculation, speed is accelerated.

The comparison of motion estimation performance under (j) different resolution; The search time of three kinds of standard video sequence under different resolution classic algorithm different from three kinds, average search block number, mean P SNR are compared in the comparison of motion estimation performance.Reacted the speed degree of search search time, on average search plain block number more few better, mean P SNR higher proof motion estimation quality is higher.By above four item compareds, draw simulation result by analogous diagram and comprehensive form.

K () motion estimation performance index is averaged: motion estimation performance is averaged, and is the estimation of conveniently comparing video sequence entirety.Each frame PSNR is more difficult, and therefore, use that average search block and average PSNR's is fast more convenient, sample more, representativeness is better.

The embodiment of motion estimation performance is mainly through the CPU time, and search block number and PSNR embody.This experiment is to the Football of Resolutions, and Caltrain, Girl tri-kinds of international standard video sequences compare.The macro block of experiment employing 16 × 16 is searched for as a unit, and each macro block is searched in the region of search S of reference frame, and finds out the block the most similar to current block according to matching criterior SAD.Searching method adopts classic algorithm entirely to search for, three-wave mixing, new three-wave mixing.The picture that emulation experiment adopts is the gray scale picture that color image sequence preliminary treatment obtains.Under Resolutions, simulation result such as the accompanying drawing of three groups of video sequences illustrates with shown in form.

Table 1.CPU time (s)

Table 2. average search block (block)

Table 3. mean P SNR (dB)

1. more same video sequence PSNR under different resolution obtains a result.

Simulation result: the PSNR not under high-resolution is higher, and the PSNR under low resolution is higher than fast video sequence for middle low speed video sequence, and rapid movement sequence is then contrary.

Be not that PSNR under high-resolution is higher, under Caltrain and Girl video sequence, the PSNR under low resolution is than the height under high-resolution, and the high-resolution of Football sequence is higher than low resolution PSNR.

As can be seen from the mean P SNR of form 3, under QCIF, NTSS performance than TSS better effects if, even in the fast video sequence of Football also higher than TSS (fast video sequence TSS performance better).Therefore, three kinds of video sequences are that speed and PSNR have exceeded TSS under QCIF.The PSNR of TSS Football sequence at high resolutions, just apparently higher than NTSS, shows advantage.

Although in the performance of Large Amplitude Motion sequence better, under low resolution, QCIF is greater than TSS at the mean P SNR of NTSS to TSS.Find out thus, resolution is higher, TSS Large Amplitude Motion sequence show better.Therefore the PSNR that QCIF estimates at NTSS is greater than TSS, supplements for TSS has well made in the performance of significantly video sequence.

2. observe the poorest frame under high-low resolution, i.e. PSNR minimum point.

Simulation result: the not necessarily the poorest frame under low resolution of the poorest frame under high-resolution.

Search for block number and PSNR under being respectively Football high-resolution with reference to Fig. 6, Fig. 7, be respectively search block number under low resolution and PSNR with reference to Fig. 8, Fig. 9.

Search for block number and PSNR under being respectively Caltrian high-resolution with reference to Figure 10, Figure 11, be respectively search block number under low resolution and PSNR with reference to Figure 12, Figure 13.

Search for block number and PSNR under being respectively Girl high-resolution with reference to Figure 14, Figure 15, be respectively search block number under low resolution and PSNR with reference to Figure 16, Figure 17.

As can be seen from PSNR curve chart, under high-resolution, the minimum point of PSNR is not necessarily minimum under low resolution.

Comparison diagram 7 Fig. 9, Football PSNR, under high-resolution, Fig. 7 PSNR minimum point is at 30 frames, and low resolution Fig. 9 is at 21 frames.Relatively Figure 11 Figure 13, Caltrain PSNR, high resolution graphics 7PSNR minimum point is at 25 frames, and low resolution is at 19 frames.Relatively Figure 15 Figure 17 Girl PSNR, high resolution graphics 7PSNR minimum point is at 1 frame, and low resolution is at 30 frames.

The comparison of the lower three kinds of classic algorithm PSNR of 3.QCIF video sequence.

Simulation result: QCIF PSNR under middle low speed video sequence is of slight difference, and under high-resolution, PSNR differs greatly.

Low speed video sequence precision under three kinds of methods close (Girl be accurate to decimal point after 3) under QCIF, even equal (Caltrain sequence).On the contrary, the PSNR difference that resolution is high is larger.

4. observe the reconstructed image of the poorest frame under the reconstructed image of the poorest frame under Caltrain high-resolution and low resolution.

Simulation result: under high-resolution, the middling speed video sequence of image quality exquisiteness has obvious information dropout, and does not have under low resolution.

Reconstructed image under Caltrain high-resolution has obvious information dropout, and does not have under low resolution, and under low resolution own, mean P SNR is higher than the mean P SNR under high-resolution.

Figure 18 and Figure 19 is the reconstructed image of Caltrain sequence at 25 frames.TSS the poorest frame 25 frame at high resolutions will lower than FS close to 1.8dB, and other both frames can not more than 1dB, and because digital lines are comparatively thin, the motion estimation result of TSS is not good, causes loss of learning.

To sum up, simulation results show TSS in the performance of Large Amplitude Motion sequence better in the past, and experimental results demonstrate the advantage that TSS shows under Football video sequence.Owing to adopting the limitation of resolution in the past, do not observe the performance situation of Football under Resolutions.In fact, Football video sequence is under QCIF, and the mean P SNR of NTSS is greater than TSS.To sum up, under the prerequisite of balanced PSNR and speed two indices, NTSS estimation effect is better, even in Football sequence.

Embodiment 1: video monitoring, video conference etc. are due under the restriction of bandwidth needs to be operated in low resolution.Particularly this little motion amplitude video sequence of video conference, when resolution is QCIF (176 × 144), application TSS or NTSS method, can obtain being close to equal precision with FS.

Embodiment 2: current algorithm is commonly used TSS and applied in fast motion estimation, in fact, the operations such as the compression of the rapid movement sequence of QCIF form can apply NTSS algorithm (because can select TSS in theory), obtain the PSNR higher than TSS and speed.

Embodiment 3: when to high-resolution and the video sequence of image quality exquisiteness take exercises estimate time, without the need to adopting TSS (low resolution can use).Because in high-resolution and in the video sequence of image quality exquisiteness, the easy drop-out of reconstructed image, and low resolution can not drop-out.

Embodiment 4: three kinds of Classical Motions are less than the PSNR under low resolution under estimating due to the mean P SNR under middle low speed video sequence high-resolution, therefore, under low resolution, the reconstructed picture of estimation is higher with former figure degree of approximation.Its experimental result can be applicable to the target identification under low resolution, the aspects such as motion tracking.

Three kinds of searching algorithms used and matching criterior are classical conventional searching algorithm, and its special character is, the comparative approach of experiment, and therefore, the data drawn and result are supplemented having made block motion estimation now.The Matlab signal to noise ratio emulation that software program uses is the technology known by image processing field.

English lexical or textual analysis involved by method for estimating based on Block-matching video sequence under different resolution of the present invention is as follows:

TSS: three-wave mixing; NTSS: new three-wave mixing; FS: entirely search for; BMA: block matching algorithm; MPEG: dynamic image expert group; H261:H.261 is the video encoding standard that nineteen ninety ITU-T formulates; H263:H.263 is the Low Bit-Rate Video Coding standard of the video conference of being formulated by ITU-T; SAD: absolute error and; PSNR: Y-PSNR; Football: football international standard sequence; Caltrain: toy train calendar international standard video sequence; Girl: girl talks international standard video sequence; QCIF: i.e. 1/4th public intermediate forms; Talk-heading: background is motionless, the video sequence of head movement.

The present invention is not limited to this embodiment, and to those skilled in the art, simple copy and improvement without creative work all belong to the scope that the claims in the present invention are protected.

Claims (1)

1., based on a method for estimating for Block-matching video sequence under different resolution, it has in comprising:

A () block motion compensation: be that picture is divided into some pieces, uses coupling rule, finds the block of previous frame in this frame position;

(b) search window: be the scope of determining to search for, each frame has the macro block of M × N, each macro block needs will find best matching blocks in region of search, and moving target determines the size of search window, and the size of search window should slightly larger than the moving object having maximum likelihood;

C () searches for entirely: be FS, obtains best Searching point by searching for all candidate blocks in consecutive frame, and the search factor of FS is P=7, and therefore, each block needs search 225 points;

D () three-wave mixing: be TSS, uses " three steps " to find optimal match point exactly, in addition, this is a searching method from coarse to meticulous, is easy to be absorbed in Local Minimum, is applicable to the video sequence larger to motion amplitude and estimates;

(e) new three-wave mixing: be NTSS, be propose on the basis of TSS, and introduce the method for center-biased, therefore, higher than TSS efficiency, be applicable to the search in the little or static video sequence of motion amplitude;

(f) matching criterior: find best matching blocks with " cost function " exactly, this process can carry out computation of match errors by given price value function, wherein minimal error match block is exactly best matching blocks, matching criterior not only has impact to precision, and also have impact to the speed of estimation, the amount of calculation of estimation is determined by the complexity of search point and matching criterior, and in order to weigh both precision and speed, matching criterior is a key factor that must consider;

G () evaluates the precision of estimation with PSNR: Y-PSNR is used for the objective estimation of picture quality, be used for especially estimating the quality of reconstructed image, be usually used in video compression, usually, after video compression, output video and original video have some differences, and in order to the video quality after measurement processing, the value of PSNR can reflect the visual effect whether this process meets the mankind; It is characterized in that, also comprise:

H () selects three kinds of international standard video sequences: the video sequence selecting the different resolution of three kinds of features, be respectively, Girl video sequence, traditional Talk-heading video sequence, background is motionless, prospect only has a small amount of motion, and Caltrain is the medium video sequence of image quality exquisiteness, and Football is then video sequence violent fast;

I () preliminary treatment: be that video sequence is changed framing, becomes gray scale picture the colour picture of each frame by Matlab program batch process;

The comparison of motion estimation performance under (j) different resolution: select two kinds of conventional resolution, to the search time of three kinds of velocity standard video sequences under different resolution classic algorithm different from three kinds, search block number, PSNR, as follows from the past different item compareds:

1) more same video sequence mean P SNR under different resolution obtains a result;

2) the poorest frame under high-low resolution is observed, i.e. PSNR minimum point;

3) comparison of the lower three kinds of classic algorithm mean P SNR of QCIF video sequence;

4) reconstructed image of the poorest frame under the reconstructed image of the poorest frame under Caltrain high-resolution and low resolution is observed;

K () motion estimation performance index is averaged: each group estimation all can produce last set block and PSNR, in order to weigh the estimation effect of whole video, needs to get average to search block and PSNR.