CN1723477A - Video encoding method and corresponding computer programme - Google Patents

Abstract

The invention relates to a method of encoding a sequence of frames, composed of picture elements (pixels), by means of a three-dimensional (3D) subband decomposition involving a filtering step applied, in the sequence considered as a 3D volume, to the spatial-temporal data which correspond in said sequence to each one of successive groups of frames (GOFs), and to implementations of said method. The GOFs are themselves subdivided into successive pairs of frames (POFs) including a so-called previous frame and a so-called current frame, and the decomposition is applied to said GOFs together with motion estimation and compensation steps performed in each GOF on saids POFs and on corresponding pairs of low-frequency temporal subbands (POSs) obtained at each temporal decomposition level. The process of motion compensated temporal filtering leading in the previous frames on the one hand to connected pixels, that are filtered along a motion trajectory corresponding to motion vectors defined by means of said motion estimation steps, and on the other hand to a residual number of so-called unconnected pixels, that are not filtered at all, each motion estimation step comprises a motion search provided for returning a motion vector that minimizes a cost function depending at least on a distorsion criterion, said criterion taking into account the unconnected pixels phenomenon for the minimizing operation, itself based on specific rules allowing to obtain, either by a non-recursive or a recursive implementation, the optimal set of motion vectors.

Description

Method for video coding and corresponding computer programs

Present invention relates in general to field of data compression, and more specifically relate to a kind of method of decomposing the frame sequence formed by picture element (pixel) of encoding by means of three-dimensional (3D) sub-band (subband), be included in the sequence that is considered to the 3D volume filtration step is applied to the space-time data, described space-time data in described sequence corresponding to each continuous frame group (GOF), these GOF itself are subdivided into continuous frame to (POF), described frame is to comprising so-called previous frame and so-called present frame, together with in each GOF described POF and corresponding low frequency chronon wave band being applied to described GOF to estimation and the compensation process that (POS) carries out, described POS obtains on each time decomposition level described decomposition.

The invention still further relates to a kind of computer program that comprises instruction set, when by being included in processor in the code device when carrying out described program, described instruction set is used to realize described coding method.

In recent years, based on 3D or (2D+t), three-dimensional (3D) subband analysis of wavelet decomposition that is considered to the frame sequence of three-D volumes is studied at video compress more and more.Wavelet transformation produces the coefficient that constitutes hierarchical pyramid (pyramid), wherein because the 3D direction tree of the father and mother-children subordinate relation of proof between described coefficient comes definition space-time relation.So, in classification tree, carry out the depth scan of the coefficient that produced and Bit-Plane Encoding technology line by line, produce the quality scalability of wanting.

Be used to realize that the actual solution of this method is, use simple two taps wavelet filter to produce the motion-compensated time sub-band, illustrate as GOF in Fig. 1 to eight frames.In the illustrated embodiment, divide framing group (GOF) input video sequence, and itself be subdivided into continuous frame each GOF to (it has and so-called motion compensated temporal filtering (Montion-Compensated Temporal Filtering) or the as many input end of MCTF module), at first passive movement compensation (MC) is then by temporal filtering (TF).Also filter (TF) low frequency very first time decomposition level, that produced (L) chronon wave band, and after the decomposition that produces one or more lower frequency wavelet sections-be called root chronon wave band arbitrary number (in diagram, provided nonrestrictive example with two decomposition level that produce two root wave band LL), this process just can stop.In the example of Fig. 1, illustrational group frame be reference number F1 to F8, and dotted arrow is corresponding to the high pass temporal filtering, other then corresponding to the low pass temporal filtering.Show two stages (L and the H=phase one of decomposition; LL and LH=subordinate phase).Each time decomposition level of illustrational 8 frame groups, produce one group of motion vector field (field) (in this example, MV4 is in the first order, and MV3 is in the second level).

When the Haar multiresolution analysis is used for the time when decomposing, because in each time decomposition level, in the frame group of considering, produce a motion vector field between per two frames, so the number of motion vector field equals in chronon wave band frame number half, promptly four in the first order of motion vector field and two in the second level.Because two simple wavelet filter have been carried out the time to down-sampling, so per two frames of list entries are just carried out an estimation (ME) and motion compensation (MC) (adopting mode forward generally).Use these very simple filtering devices, the right time average of each low frequency chronon wave band (L) expression incoming frame, and high frequency time sub-band (H) is included in MCTF step residual error afterwards.

Unfortunately, motion compensated temporal filtering may increase the weight of not connect the problem of pixel, and these do not connect that pixel is basic and just be not filtered (or also increase the weight of the problem of dual connection pixel, these dual connection pixels have been filtered twice).The number that does not connect pixel is represented the weakness of 3D sub-band coding method, and this is because it very seriously influences the image quality that is produced, and is especially true in the closed region.This is all the more so for high motion sequence or for final time decomposition level, and wherein temporal correlation is not so good.These numbers that do not connect pixel depend on the intensive motion vector field that has been produced by estimation.

Being used for not considering in the current standard of the employed optimal motion vector of estimation search will be as the number of the pixel result, that do not connect of motion compensation.Most of advanced algorithm using tendencies are in the rate/distortion criterion that minimizes cost function, described cost function depend on be used to send motion vector (ratio) the bit number that consumes and shift difference energy (distortion).For example, described motion search returns the motion vector that minimizes following expression (1):

J(m)＝SAD(s，c(m))+λ _MOTION·R(m-p) (1)

In this expression formula (1), m=(m _x, m _y) ^TBe described motion vector, p=(p _x, p _y) ^TBe prediction to described motion vector, and λ _MOTIONIt is Lagrange's multiplier.Rate term R (m-p) only represents movable information, and the SAD as distortion measurement is calculated as:

$\mathrm{SAD}(s,c\left(m\right))=\underset{x=1,y=1}{\overset{B,B}{\mathrm{\Σ}}}\left|s\right[x,y]-c[x-m_x,y-m_y\left]\right|---\left(2\right)$

Wherein s is original vision signal, and c is the vision signal of coding, and B is block size (noticing that B can be 1).Unfortunately, these algorithms are not considered between the counter motion amortization period distortion introduced by the pixel that does not connect, and this is because usually these optimizations are applied to it not carried out the hybrid coding of counter motion compensation.

Therefore the objective of the invention is to avoid this shortcoming, and propose a kind of method for video coding, wherein in distortion measurement, consider the described set of pixels that does not connect.

For this reason, the present invention relates to a kind of such as defined method in introducing section, and described method is characterized in that, the process of described motion compensated temporal filtering is formerly introduced the connection pixel in the frame on the one hand, these connect pixel and are filtered along the movement locus corresponding to the motion vector that defines by means of motion-estimation step, and introduce the number that residual what is called does not connect pixel on the other hand, these do not connect pixel and are not filtered at all, each motion-estimation step is included as return movement vector and the motion search that provides, described motion vector also is applied to the described set of pixels that does not connect to described measurement distortion at least according to comprising that the distortion criterion of distortion measurement minimizes cost function.

To describe the present invention with reference to the accompanying drawings with the form of giving an example now, wherein Fig. 1 shows the time multiresolution analysis with motion compensation.

Because the pixels tall that connects does not participate in the quality degradation of counter motion compensating images, so, in distortion measurement, consider the set of pixels that does not connect according to the present invention.For this reason, introduce new rate/distortion criterion in this suggestion, it has expanded the equation of considering not connect the pixel phenomenon.This is in equation (3) and (4) illustrated, and described equation (3) and (4) are of equal value:

K(m)＝J(m)+λ _UNCONNECTED·D(S _UNCONNECTED(m)) (3)

K (m)=SAD (s, C (m))+λ _UNCONNECTEDD (S _UNCONNECTED(m))+λ _MOTIONR (m-p) (4) is D (S wherein _UNCONNECTED(m)) be to the set of pixels S that does not connect by motion vector m generation _UNCONNECTEDDistortion measurement.Can be applied to the described set of pixels that do not connect to several distortion measurements.Preferably, once very simple measurement is exactly the counting that does not connect pixel to the motion vector in the research.

But should be noted that have only when motion vector information for the entire frame time spent, just can calculate that produce by motion search, actual not connection set of pixels.Therefore, optimum solution almost is (in fact should find the solution the complex set of the minimization standard of entire frame) that impossible draw, and has therefore proposed a kind of suboptimal embodiment.Can think that this non-recursive embodiment be used to consider because the plain mode of the distortion that pixel caused that does not connect.For the image that will carry out motion compensation give certain portions (part of image can be the macroblock of pixel, block of pixels, pixel or supposition segment set covered entire image and without any overlapping any zone), and for given candidate motion vector m, use the time reversal motion compensation, the set of pixels that identification does not connect, and can be in the hope of D (S _UNCONNECTED(m)) value.Can calculate current K (m) value then, and with itself and current minimum value K _Min(m) compare, so that check this candidate's motion vector whether to produce lower K (m) value (for first candidate's motion vector, K (m) obviously equals the value K (m) that calculates).When after tested during all candidate item, (final) counter motion compensation is applied to optimal candidate item (identification connects and the pixel that is not connected).Can handle the next part of described image then, like that up to the processing of finishing entire image.

Yet, in this non-recursive embodiment, always the space is not uniform on entire image in the decision that is produced: for the first that will carry out the image of motion compensation, the described set of pixels that does not connect may be empty, and for the decline that will carry out the described image of motion compensation, the probability of the pixel of Lian Jieing is very not high.This situation may cause uneven spatial distortion.In order to overcome this problem and to produce a kind of embodiment of one way, the embodiment of multipass can be proposed, this minimizes the embodiment that global criteria ∑ K (m) can improve described one way really by all parts for entire image, and this can also finish with the embodiment of the multipass that comprises the following steps.

At first, for all parts of image, calculate optimal motion vector m _Opt, and the N that minimum value is provided to the J of equation (1) (m) _Sub-optSuboptimal motion vector set { m _Sub-opt, do not use number (the suboptimum vector N that does not connect pixel in this stage _Sub-optNumber be that embodiment is relevant).For all these vectors, store the respective value of described standard J (m), so that produce J (m _Opt) and { J (m _Sub-opt).Then, for optimal motion vector m _OptApplication counter motion compensation is so that calculate

(note, Be not

Optimal value, this is because m _OptOptimize J (m) rather than K (m)).Then, from the tabulation of suboptimum vector, select to minimize | { J (m _Opt)-{ J (m _Candidate) | candidate motion vector m _Candidate(note m _CandidateCan be the vector of any part of present image).For optimal motion vector and candidate vector (optimal vector of alternative image counterpart) collection, use counter motion compensation and calculating once more

If it is worth ratio

Low, use m so _CandidateReplace optimal value m _Opt(for the counterpart of described image).At last, from the tabulation of suboptimum vector, abandon m _CandidateThen, selecting new candidate item, and in order to obtain the optimum collection of motion vector, use identical mechanism, is empty up to the tabulation of described suboptimum vector.

Claims (5)

1. one kind is used for by means of the encode method of the frame sequence be made up of picture element (pixel) of three-dimensional (3D) sub-band decomposition, be included in the sequence that is considered to the 3D volume filtration step is applied to the space-time data, described space-time data in described sequence corresponding to each continuous frame group (GOF), these GOF itself are divided into continuous frame again to (POF), described frame is to comprising so-called previous frame and so-called present frame, described decomposition is applied to described GOF together with the low frequency chronon wave band to described POF and correspondence in each GOF to estimation and the compensation process that (POS) carries out, described POS obtains on each time decomposition level, the process of this motion compensated temporal filtering is formerly introduced the connection pixel in the frame on the one hand, these connect the pixel quilt along filtering corresponding to the movement locus by means of the defined motion vector of motion-estimation step, introduce residual on the other hand, the what is called that at all is not filtered does not connect the number of pixel, each motion-estimation step is included as to return at least according to the distortion criterion that comprises distortion measurement and minimizes the motion search that the motion vector of cost function provides, and described measurement distortion also is applied to the described set of pixels that does not connect.

2. coding method as claimed in claim 1 wherein provides described motion search so that return the motion vector that minimizes following expression (1):

J(m)＝SAD(s，c(m))+λ _MOTION·R(m-p) (1)

M=(m wherein _x, m _y) ^TBe motion vector, p=(p _x, p _y) ^TBe prediction to described motion vector, λ _MOTIONBe Lagrange's multiplier, rate term R (m-p) only represents movable information, and the SAD as distortion measurement is calculated as:

$\mathrm{SAD}(s,c\left(m\right))=\underset{x=1,y=1}{\overset{B_{1}B}{\mathrm{\Σ}}}\left|s\right[x,y]-c[x-m_x,y-m_y\left]\right|---\left(2\right)$

S is original vision signal, c is the vision signal of coding, and B is a block size, it is characterized in that: described distortion criterion extended equation (1), consider the pixel phenomenon that does not connect that is used to minimize operation, wherein the described operational applications that minimizes in following expression formula (3):

K(m)＝J(m)+λ _UNCONNECTED·D(S _UNCONNECTED(m)) (3)

D (S wherein _UNCONNECTED(m)) be the set of pixels S that does not connect that produces by motion vector m _UNCONNECTEDDistortion measurement.

3. coding method as claimed in claim 2 is characterized in that in order to consider owing to do not connect the distortion that pixel produces, and it comprises the following steps, successively the following step is applied to carry out each part of the entire image of motion compensation:

(a) for the consideration part of image with for given candidate motion vector m, use the time reversal motion compensation;

(b) discern the set of pixels that does not connect;

(c) obtain D (S _UNCONNECTED(m)) value;

(d) calculate current K (m) value and with itself and current minimum value K _Min(m) compare, so that check described candidate motion vector whether to produce lower K (m) value;

(e) when all after tested candidate item, final counter motion compensation is applied to the optimal candidate item;

(f) then step (a) to (e) is applied to the next part of image that can similar processing, the part of described image is that pixel, block of pixels, pixel macroblock or supposition segment set have covered entire image and without any overlapping any zone.

4. coding method as claimed in claim 2 is characterized in that in order to consider because the distortion that pixel produced that does not connect and minimize global criteria ∑ [all parts] K (m) of the entire image that will compensate, described method comprises the following steps:

(a) calculate optimal motion vector m _Opt, and the N that minimum value is provided for J (m) _Sub-optSuboptimum motion vector set { m _Sub-opt;

(b) for all these vectors, the respective value of storage standards J (m) is so that produce J (m _Opt) and { J (m _Sub-opt);

(c) for optimal motion vector m _Opt, use the counter motion compensation and answer, so that calculate ∑ [all parts] K (mopt);

(d) from the tabulation of suboptimum vector, select to minimize | { J (m _Opt)-{ J (m _Candidate) | candidate motion vector m _Candiate

(e), use the counter motion compensation, so that calculate ∑ [all parts] K (m) once more for optimal motion vector and candidate vector collection;

(f) if the value of ∑ [all parts] K (m) is lower than the value of ∑ [all parts] K (mopt), for the counterpart of described image, use m so _CandidateReplace optimal value m _Opt

(g) last, from the tabulation of suboptimum vector, abandon m _Candidate

(h) selecting new candidate item, in order to obtain the optimum collection of motion vector, uses identical mechanism then, is sky up to the tabulation of described suboptimum vector.

5. computer program that comprises instruction set, described instruction set is used for when carrying out described program by the processor that is included in code device, realizes according to any one method in claim 3 and 4.