CN101980537B - Object and fractal-based binocular three-dimensional video compression coding and decoding method - Google Patents

Abstract

The invention provides an object and fractal-based binocular three-dimensional video compression and decompression method. In binocular three-dimensional video coding, a left channel is used as a basic layer, a right channel is used as an enhancement layer, and the left channel is encoded by an independent motion compensation prediction (MCP) mode. The object and fractal-based binocular three-dimensional video compression coding method comprises the following steps of: firstly, acquiring a video object partition plane, namely an Alpha plane by a video partition method, encoding the initial frame of a left eye through block discrete cosine transformation (DCT), and performing block motion estimation/compensation coding on a non-I frame of the left eye; secondly, determining the area attribute of an image block by utilizing the Alpha plane, and if the block is not within a video object area of the current code, not processing an external block, and if the block is within the video object area of the current code completely, searching the most similar matching block by a full-searching method in a previous frame of an internal block, namely a reference frame searching window of a left eye video; and finally, compressing coefficients of an iterated function system by a Huffman coding method, and if part of pixels of the block are within the video object area of the current code, and the other part of pixels are not within the video object area of the current code, processing a boundary block independently. The right channel is encoded by a MCP mode and a disparity compensation prediction (DCP) mode, the MCP is similar to the processing of the left eye, and the block with the minimum error is used as a prediction result. When the DCP coding mode is performed, the polarization and directionality in a three-dimensional parallel camera structure are utilized fully.

Description

A kind of based on object and fractal binocular tri-dimensional video compression coding and decoding method

Technical field:

The invention belongs to the video compression coding field, relate to the compressed encoding of binocular tri-dimensional video, particularly a kind of based on object and fractal video compressing and encoding method.Under the prerequisite that guarantees picture quality, increase compression ratio to a certain extent, and accelerated fractal binocular tri-dimensional video speed of coding greatly, shortened the scramble time.

Background technology:

Based on object (Object-Based; Abbreviation OB) notion of coding is proposed by the MPEG-4 standard the earliest; Use object-based video compressing and encoding method to make the foreground object of each frame video and background object be able to independently encode; Can further improve compression ratio; Simultaneously can realize the function that some are new in decoding end, such as to each object video independently transmit and the replacement of decoding, object and background, object-based video frequency searching, particularly with respect to the coding method of standard, on the border of object, can obtain the better pictures quality; Because the border of object usually with image in the edge coincide, it is the difficult part of coding.MPEG-4 has proposed object-based video coding technique standard; In video analysis; With the object video is base unit, and each Video Events and scene can be regarded as being combined by static attribute of all semantic objects in the video (like shape, color, texture) and dynamic attribute (exercise intensity, direction, rhythm) thereof.It with the object methods of video analyses of fundamental analysis unit; Meet the human mode of thinking and visual characteristic; Got rid of unessential part in the video to the interference of video analysis (referring to Liang Zhang.Object-basedMethod of important video clips Extraction [J] .Journal of Beijing Information Science andTechnology University, 2009,24 (4): 83-86); (referring to Bo Huang; Yujian Huang.A ScalableObject-based Wavelet Coder [J] .Radio Communications Technology, 2009,35 (1): 35-38); (referring to Kassim Ashraf a; Zhao L F.Rate-scalable object-based wavelet codec with implicitshape coding [J] .IEEE Transactions on Circuits and Systems for Video Technology, 2000,10 (7): 1068-1079).Object-based method for video coding can improve the performance of video compression coding and make it have more flexibility.

Fractal theory proposes (referring to

B.Mandelbrot.TheFractal Geometry ofNature [M] .New York:W.H.Freeman and Company, 1982.) by Mandelbrot at first the seventies in last century.The Fundamentals of Mathematics of fractal image are iterated function system (IFS) theories.Barnsley at first with fractal image be used for interactive image compression (referring to Michael F.Barnsley, Alan D.Sloan.A better way to compress image [J] .ByteMagazine, 1988,13 (1): 215-233.).Jacqain has proposed full automatic Fractal Image Compression Approach (referring to Arnaud E.Jacquin.A novel fractal blocking-coding technique for digital image [C] .IEEEInternational Conference on Acoustics; Speech and Signal Processing; 1990; 4:2225-2228.), (referring to Arnaud E.Jacquin.Fractal image coding:a review [J] .Proceeding of the IEEE, 1993; 81 (10): 1451-1465.), this method adopts the affine transformation that replaces the overall situation based on the mode of image block with the affine transformation of part.Afterwards, Fisher utilizes quaternary tree to improve this method (referring to Y.Fisher.Fractal ImageCompression [J] .Fractals, 1994; 2 (3): 347-361.), (referring to Y.Fisher, E.W.Jacobs.Imagecompression:A study the iterated transform method [J] .Signal Processing; 1992; 29 (3), 251-263.), (referring to Y.Fisher.Fractal Image Compression:Theory and application to digitalimages [M] .New York:Spring-Verlag; 1995; 55-77.), improve code efficiency greatly, and become the main stream approach in the present fractal image coding.

On this basis, some scholars and researcher are applied to the method for fractal pattern compression in the compression of video sequence.Meiqing Wang etc. proposed comprehensively based on data cube and based on the fractal video-frequency compression method of frame (referring to Meiqing Wang; Choi-Hong Lai.A hybrid fractal video compression method [J] .Computers &Mathematics with Applications, 2005,50 (3-4): 611-621.); (referring to Meiqing Wang; ZhehuangHuang, Choi-Hong Lai.Matching search in fractal video compression and its parallelimplementation in distributed computing environments [J] .Applied Mathematical Modeling, 2006; 30 (8): 677-687.); (referring to Meiqing Wang, Rong Liu, Choi-Hong Lai.Adaptive partitionand hybrid method in fractal video compression [J] .Computers & Mathematics withApplications; 2006,51 (11): 1715-1726.).Wherein classical and influence bigger referring to (C.S.Kim; R.C.Kim; S.U.Lee.Fractal coding of video sequence using circular prediction mapping andnoncontractive interframe mapping [J] .IEEE Transactions on Image Processing; 1998,7 (4): 601-605.).This method adopts and is similar to the motion estimation/compensation technology that the normal video coding method is adopted, and this method has been utilized the time strong correlation between the consecutive frame, and compression has obtained effect preferably to video sequence.In CPM and NCIM, each image block in the sub-piece territory is all obtained through motion compensation by the father's piece territory from the identical size of consecutive frame.CPM need possess convergence with the maximum different CPM of being of NCIM between the two in the process of decoding, and NCIM does not need.But in circular prediction coding (CPM) method, can approximate convergence arrive original image in order to guarantee start frame through the circulation decoding of self, compression process need be passed through complex transformations, search and iteration etc., and compression time and picture quality are difficult to reach requirement.The operand of at present typical fractal pattern and video-frequency compression method is very big, and coding rate is slower, and the quality of decoding has much room for improvement, and makes fractal pattern and video-frequency compression method also need further improvement and raising.

Along with the continuous development of multimedia technology, multi-view point video has the focus that the incomparable superiority of monocular video becomes research gradually because of it.And binocular tri-dimensional video is the most a kind of form of using in the multi-view point video; It has increased the depth information of scene, makes the image of enjoying that strong presence and sense true to nature arranged, and can be applied to stereoscopic TV; Long-distance education; Long-range Industry Control, and various fields such as telemedicine diagnosis and virtual reality (referring to A Schertz.Sourcecoding of stereoscopic television pictures [C] .Proceedings of the IEEE InternationalConference on Image Processing and its Applications.Maastricht, The Netherlands; 1992,462-464.).Binocular tri-dimensional video is a binocular parallax principle of utilizing eye; Binocular receives the left and right sides image from the specific camera point of Same Scene independently of one another, and left eye is seen the image of taking back, and right eye is seen the image that takes over; Form binocular parallax; Big brain-capacity obtains the depth information of image, makes the image of enjoying that strong depth perception, sense true to nature arranged, and spectators can enjoy superpower stereoscopic visual effect.But with respect to monocular video, the binocular tri-dimensional video system must transmit and store the data volume of being doubled, so must compress effectively it.(referring to Shigang Wang; Xuejun Wang; HexinChen.Stereoscopic video compression coding based on is [J] .Chinese Journal ofStereology and Image Analysis H.264, and 2008,13 (1): 11-16)

In the binocular tri-dimensional video cataloged procedure, not only to consider in each passage before and after relativity of time domain and the spatial correlation of I picture between the two field picture, the spatial correlation that also will make full use of between the passage is encoded.It is redundant that the former can utilize motion compensated prediction (MCP) to remove, and it is redundant that the latter can adopt parallax compensation prediction (DCP) to remove interchannel.DCP compares with MCP and wants the many of complicacy, key technology wherein is the parallax coupling.Michael E.Lukaces is the early stage researcher of binocular tri-dimensional video coding; He has explored parallax compensation (DC-based) (refer to use binocular parallax to close and tie up to foundation correspondence between two width of cloth images) has been used for another video sequence of video sequence prediction from the binocular tri-dimensional frequency sequence; And multiple method based on parallax compensation has been proposed (referring to Lukacs M E; Predictive coding ofmulti-viewpoint image sets [C] .IEEE International Conference on Acoustics; Speech andSignal Processing, 1986,521-524).Perkins will reduce a kind of conditional compilation method based on the method for parallax compensation; For lossless coding is optimum; For lossy coding then be suboptimum (referring to Perkins M G.Datacompression of stereo pairs [J] .IEEE Transactions on Communications; 1992,40 (4): 684-696).People such as Tzovaras have proposed the hierarchical block coupling and the motion parallax bidirectional compensating method of disparity estimation; Being referred to as polymerization estimates (referring to Tzovaras D; Grammalidis N; Strintzis M G.Object-based codingof stereoscopic image sequences using joint 3D motion/disparity segmentation [C] .Proceedings of SPIE:Visual Communication and Image Processing; 1995,1678-1689).Franich has also proposed the parallax estimation method based on the generic block matching algorithm in addition; And introduce a kind of level and smooth detection means and estimate parallax coupling quality (referring to Franch R E H; Lagendijk R L; Biemond J.Stereo-enhanceddisplacement estimation by genetic block matching [C] .Proceedings of SPIE:Volume2094-Visual Communications and Image Processing.Cambridge, MA, USA; 1993,362-371).People such as Siram Sethuramn and M.Siegel have proposed the variable-block quaternary tree multiresolution dividing method based on parallax and motion; To coding based on the binocular tri-dimensional video of MPEG video encoding standard stream; By the needs whether the synthetic mesophase view is arranged in decoding end; Proposed two binocular tri-dimensional video streams the mixed-resolution coding method (referring to Sethuraman S, SiegelM W, Jordan A G.Segmentation based coding of stereoscopicimage sequences [C] .Proceedings of SPIE:Volume 2668-Digital Video Compression:Algorithms and Technologies.San Jose; CA; USA, 1996,420-429).Binocular tri-dimensional video encoding scheme commonly used has: independent MCP (Motion Compensated Prediction) coding, interlayer DCP (DisparityCompensated Prediction) coding and the MCP coding that combines with DCP.Wherein in MCP combined coding with DCP, basic layer adopted independent MCP coding, and enhancement layer adopts MCP and DCP dual mode to predict that therefrom the less a kind of conduct of Select Error predicts the outcome.

Summary of the invention:

The present invention proposes a kind of based on object and fractal binocular tri-dimensional video compression and decompression method; In binocular tri-dimensional video coding, it serve as basic layer with left passage; Right passage is an enhancement layer, and left passage adopts independent motion compensated prediction mode (MCP) to encode.At first utilizing methods of video segmentation to obtain the video object segmentation plane is the Alpha plane, and left purpose start frame is adopted piece dct transform coding, and the non-I frame of left purpose is carried out piece motion estimation/compensation coding.Utilize the Alpha plane to differentiate the area attribute of this image block then, if this piece does not then process this external block not in the object video zone of present encoding; If this piece is all in the object video zone of present encoding; In the former frame of internal block is the reference frame search window of left visual frequency, utilize all direction search method to seek the most similar match block, utilize Huffman coding method compression iterated function system coefficient at last; If the partial pixel of this piece is in the object video zone of present encoding, partial pixel in the object video zone of present encoding, does not then need individual processing to this boundary block.Right passage adopts MCP to add parallax compensation prediction mode (DCP) and encodes, and MCP similarly handles with left order, and the minimum conduct of Select Error predicts the outcome.When carrying out the DCP coded system, make full use of polarizability and directivity in the three-dimensional parallel shooting structure again.

A kind of based on object and fractal binocular tri-dimensional video compaction coding method, may further comprise the steps:

Step 1: utilize automatic video frequency Object Segmentation method that video sequence is cut apart automatically, the video object segmentation plane that obtains each frame is the Alpha plane, confirms the object video zone that will encode according to the video object segmentation plane that obtains; Call present frame to the frame that is compressing, call reference frame to the former frame of encoding and rebuilding of present frame;

Step 2: for left order; At first judge whether frame, if the I frame then carries out the piece division of the fixed size of non-overlapping copies to this frame into I; Each image block is adopted the I picture compression method based on the piece dct transform respectively; This two field picture is carried out independent Code And Decode, and the pixel after the decoding then keeps this pixel value as if belonging to object video to be encoded in the Alpha plane, otherwise with forwarding step 14 to after the pixel value zero setting; Said I frame is the picture frame that only carries out intraframe coding in video sequence start frame or the video sequence; The said image block that present frame is divided into the non-overlapping copies of fixed size is called macro block; Saidly current macro is carried out the piece that tree-shaped division obtains be called fritter; The set of said all pieces of present frame is called sub-piece territory; The set of all pieces of said former frame is called father's piece territory; Piece in said dct transform adopts the fixed size pattern; For right order, forward step 6 to;

Step 3: if left order is non-I frame; Motion compensated prediction (MCP) coding with conventional monocular; Present frame is carried out the macroblock partitions of non-overlapping copies; The pixel of the fritter that calculates these macro blocks then and obtain through tree-shaped division and, pixel quadratic sum, and left order former frame reconstructed image is in the reference frame are according to the pixel of the fritter of setting all macro blocks that step-length divides and obtaining through tree-shaped division and, pixel quadratic sum;

Step 4: to the current image block of handling is current block, utilizes the Alpha plane to differentiate the area attribute of this image block; If this piece in the object video zone of present encoding, does not process this piece; If this piece in the object video zone of present encoding, changes step 5 over to all; If the partial pixel of this piece is in the object video zone of present encoding, partial pixel in the object video zone of present encoding, does not need individual processing, changes step 13 over to; If all macro blocks all dispose, then forward step 14 to; The said not piece in the object video zone of present encoding is called external block, and said piece in the object video zone of present encoding all is called internal block, and the said partial pixel not piece in the object video zone of present encoding is a boundary block;

Step 5: successively all macro blocks of present frame are encoded, at first this macro block is carried out the piece coupling in the search window in father's piece territory; In the matching process that carries out sub-piece and father's piece, the position of sub-piece is as the initial search point of father's piece, and the size of father's piece is big or small identical with sub-piece; If matching error RMS is less than beginning preset threshold γ, preserving current iterated function system coefficient is the IFS coefficient, changes next macro block of step 5 coding over to; Otherwise, according to tree this piece is divided successively, and,, is changed over to next macro block of step 5 coding if RMS less than setting threshold γ, stops to divide and writing down this fritter IFS coefficient to the fritter difference computation of match errors RMS that each division obtains; Otherwise continue to divide, up to current block being divided into predefined smallest blocks, record IFS coefficient; Change next macro block of step 5 coding over to; Said search window is the rectangular search zone in reference frame; Said IFS coefficient comprises that (x is y) with scale factor s, displacement factor o in father's piece position; If all macro blocks of present frame have all been encoded finish, and be left order, then forward step 14 to, if right order then continues execution in step seven;

Step 6: to right order image; At first carry out the macroblock partitions of non-overlapping copies; The pixel of the fritter that calculates these macro blocks then and obtain through tree-shaped division and, pixel quadratic sum, and right order former frame reconstructed image reference frame are according to the pixel of the fritter of setting all macro blocks that step-length divides and obtaining through tree-shaped division and, pixel quadratic sum; To reduce the double counting in the piece matching process, forward step 5 to;

Step 7: calculate corresponding two field picture reference frame in the left order,, forward step 8 to according to the pixel of the fritter of setting all macro blocks that step-length divides and obtaining through tree-shaped division and, pixel quadratic sum;

Step 8: to the current image block of handling is current block, utilizes the Alpha plane to differentiate the area attribute of this image block; If this piece in the object video zone of present encoding, does not process this piece; If this piece in the object video zone of present encoding, changes step 9 over to all; If the partial pixel of this piece is in the object video zone of present encoding, partial pixel in the object video zone of present encoding, does not need individual processing, changes step 13 over to; If all macro blocks all dispose, then forward step 14 to;

Step 9: at first the father piece identical with the current sub-block position carried out piece coupling, obtains RMS, and preserve the iterated function system coefficient, this coefficient comprise father's piece and sub-piece the relative displacement vector (x, y), scale factor s and displacement factor o; Successively all macro blocks of present frame are encoded, at first this macro block is carried out the piece coupling in the search window in father's piece territory; In the matching process that carries out sub-piece and father's piece, the position of sub-piece is as the initial search point of father's piece, and the size of father's piece is with the big or small identical of sub-piece and change step 10 over to; If the matching error RMS of gained is less than beginning preset threshold γ, preserving current iterated function system coefficient is the IFS coefficient, changes next macro block of step 9 coding over to; Otherwise; According to tree this piece is divided successively, and the fritter that each division obtains is changed over to step 10 computation of match errors RMS respectively, if RMS is less than setting threshold γ; Stop to divide and writing down this fritter IFS coefficient, change next macro block of step 9 coding over to; Otherwise continue to divide, up to current block being divided into predefined smallest blocks, changing step 10 over to and calculate RMS, record IFS coefficient changes next macro block of step 9 coding over to; Last and step 5 gained result compares, and the minimum conduct of Select Error predicts the outcome; If all macro blocks of present frame have all been encoded finish, then forward step 14 to;

Step 10: two points are searched in interlacing about the elder generation in vertical direction; If the RMS of these two points all forwards step 11 to greater than the RMS that puts on the polarized line, otherwise forward step 12 to;

Step 11: only search for, in search procedure, carry out the great-jump-forward search along the right side of initial point; Search for to two points of right septum, calculate RMS, change step 10 over to and carry out the vertical direction search; If the RMS of current point less than more preceding RMS, as new initial search point, rotates back into the step 11 starting position with current point, continue under the horizon scan a bit; Otherwise the RMS that current point is described is greater than more preceding RMS, and promptly optimal match point selects the proper cell territory to search for entirely between these 2, so far selects the minimum point of RMS as final matching results, finishes the DCP search procedure, continues execution in step nine;

Step 12: select 2 points that middle RMS is minimum; Search for the point between adjacent 2 in vertical direction again; Calculate its RMS, the point of Select Error minimum changes step 11 over to and continues to search for along the polarized line direction as the optimal match point on this vertical direction;

Step 13: this step individual processing boundary block; The boundary block of present frame only belongs to search matched in boundary block and the internal block in father's piece; The pixel of current block and father's piece is chosen criterion: in current block, only calculate the pixel value in the object video zone that is positioned at present encoding, only in the same video subject area of reference frame, carry out the piece coupling; In father's piece, if also belong to this object video zone, then adopt original pixel value with a certain pixel in father's piece of the opposite position of current block, otherwise, replace with other mean value that belongs to this object video area pixel; Forward step 5 to;

Step 14: all IFS coefficients are carried out the Huffman coding, reduce the statistical redundancy of IFS coefficient data; Judge whether present frame is last frame, if last frame finishes coding; Otherwise, return step 2 and continue to handle the next frame image.

Said a kind of the video sequence of processing is a yuv format based on object and fractal binocular tri-dimensional video compaction coding method, adopts above-mentioned 14 steps to handle in 3 components each respectively.

In the said step 5 macro block is adopted tree-shaped division, the piece coupling adopts the matching error criterion; The matching error RMS of sub-piece and father's piece is:

$\mathrm{RMS}=\frac{1}{N}[\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{r}_i^2+s(s\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{d}_i^2-2\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{r}_i{d}_i+2o\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{d}_i^2)+o(N\·o-2\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{r}_i)]---\left(1\right)$

Wherein parameter s and o are respectively:

$s=\frac{[N\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{r}_i{d}_i-\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{r}_i\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{d}_i]}{[N\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{d}_i^2-{\left(\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{d}_i\right)}^2]}---\left(2\right)$

$o=\frac{1}{N}[\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{r}_i-s\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{d}_i]---\left(3\right)$

Wherein, N is the number of sub-piece and father's piece pixel, r _iBe the pixel value of sub-piece, d _iPixel value for father's piece;

Calculate the block-matching error RMS of current macro in reference frame, wherein r _iBe the pixel value of sub-piece, d _iIt is the pixel value of father's piece; If RMS is less than pre-set threshold γ, record IFS coefficient, IFS coefficient comprise match block displacement vector (x, y) with formula 2,3 in s and o, handle next macro block; Otherwise, current macro is carried out tree-shaped division, calculate the RMS that divides the back fritter, if less than threshold gamma, then stop to divide, otherwise continue to divide, till sub-piece reaches predefined smallest blocks.

The searching method of vertical direction has used the polarizability of binocular tri-dimensional video in the said step 10, and it is on the horizontal line that optimal match point is arranged on the polarized line: in the parallel camera system of solid, along continuous straight runs carries out the DCP search;

Two video cameras are watched the same characteristic point P (x in space at synchronization _c, y _c, z _c), be respectively P at the coordinate of left image and right image _Left=(X _Left, Y _Left), p _Right=(X _Right, Y _Right); Its mid point P (x _c, y _c, z _c) being called the plane of polarization with two determined planes of photocentre, the intersection of plane of polarization and left and right sides image is called polarized line; Obtain by geometrical relationship:

$\left\{\begin{array}{c}{X}_{\mathrm{left}}=f\frac{x_c}{z_c}\\ X_{\mathrm{right}}=f\frac{(x_c-B)}{z_c}\\ Y=f\frac{y_c}{z_c}\end{array}\right.---\left(4\right)$

The searching method of horizontal direction has used the directivity of binocular tri-dimensional video in the said step 11; The projected position difference vector of the same space o'clock in two planes of delineation be parallax

hence one can see that, and parallax D is the function of depth z, the expression relative depth; For same characteristic point, its perspective projection left side image is the local translation of right image along horizontal negative sense, says that the image coordinate of the same space point on right image takes over than the image coordinate on left image, when search, only needs to carry out along a direction.

A kind of based on object and fractal binocular tri-dimensional video decompression method, it is characterized in that comprising following steps:

Step I: at first read in left order compressed information, comprise the compression frame number, every two field picture wide and high, I frame compression quality and the interval of inserting the I frame;

Step II: judge that whether decoded frame is the I frame, if the I frame changes Step II I over to, otherwise changes step IV over to;

Step II I: for the I frame, from compressed file, read in code stream, and read in the Alpha plane of this frame; The decompression mode that is similar to JPEG is decoded, and the file after the decoding comprises based on the video file of different objects and complete video file, in object-based video file; According to the Alpha plane, judge corresponding object properties, belong to this object pixels and keep; Do not belong to this object pixels zero setting, frame number adds one and changes step VII over to;

Step IV:, calculate in the left order former frame according to the pixel of the fritter of setting all macro blocks that step-length divides and obtaining through tree-shaped division and, pixel quadratic sum for non-I frame;

Step V: read in right order compressed information, comprise the compression frame number, every two field picture wide and high, I frame compression quality and the interval of inserting the I frame; Calculate in the corresponding frame of right order former frame and left order according to the pixel of the fritter of setting all macro blocks that step-length divides and obtaining through tree-shaped division and, pixel quadratic sum, change step VI over to;

Step VI: from the compressed file of correspondence, read in the division information of piece and the Alpha plane of Huffman code stream and this frame; And obtain the dividing mode of these all macro blocks of frame and the iterated function system coefficient of each fritter according to above information; Utilize the Alpha plane to differentiate the area attribute of this image block, current block is divided into internal block, external block and boundary block; For internal block, carry out fractal decoding according to each macro block; For internal block, carry out fractal decoding according to each macro block; External block need not be handled; Boundary block replaces with average, carries out fractal decoding;

Step VII: judge this moment, whether all frames all decoded, finished if all decode, and finished decode procedure, otherwise changed Step II over to.

When decompressing, at first judge the dividing mode of this macro block when coding,, at first find and this corresponding zone of sub-piece that the formula below utilizing then obtains the pixel value of this sub-piece in father's piece territory for each sub-block for each macro block:

r _i＝s·d _i+o (5)

R wherein _iBe the pixel value of sub-piece to be decoded, d _iBe the pixel value in father's piece territory, s is a scale factor, and o is a displacement factor.

In object-based decode procedure, utilize the Alpha plane to differentiate the area attribute of this image block, current block is divided into internal block, external block and boundary block; Only belong to the pixel of this subject area in the current block, promptly internal block is just decoded; In father's piece territory, only utilize the pixel that belongs to the same target zone to decode; If not belong to this object video be boundary block to partial pixel in certain sub-block in father's piece territory, the value of this part pixel replaces with the mean value of the pixel that belongs to this subject area in this sub-piece.

In based on object and fractal binocular tri-dimensional video decode procedure; Left side order is with motion compensated prediction (MCP) coding of conventional monocular; Right each image block of purpose predicts that through motion compensated prediction (MCP) and parallax compensation prediction (DCP) dual mode therefrom the less a kind of conduct of Select Error predicts the outcome.

The video sequence of handling is a yuv format, adopts above-mentioned seven steps to handle in 3 components each respectively.

A kind of advantage based on object and fractal binocular tri-dimensional video compression method proposed by the invention is:

(1) in this method start frame is adopted piece dct transform coding in the frame,, not only reduced compression time and blocking effect significantly, and improved Y-PSNR with respect to the CPM/NCIM encoding scheme.

(2) this method earlier is divided into lineament non-overlapping copies, that be made up of the macro block of plurality of fixed size dimension with a two field picture; Then according to the relation of matching criterior and setting threshold; Macro block is carried out tree to be divided; With respect to the quad-tree partition criterion, greatly reduce computation complexity, improved coding rate.

(3) this method is before carrying out the piece coupling of sub-piece, calculate the macro block of present frame non-overlapping copies and the pixel of the fritter that after tree-shaped division, obtains and with the pixel quadratic sum; In reference frame, according to the coupling step-length calculate each macro block respectively and the pixel of the fritter that after tree-shaped division, obtains and with the pixel quadratic sum; Avoid in the piece matching process, duplicating the drawback of calculating, practiced thrift the match time of sub-piece greatly.

(4) in this method, left purpose coded system is identical with the monocular fractal image; In the right order,, added the corresponding frame of left order frame as a reference, chosen the minimum piece of error as father's piece except adopting former frame as a reference the frame, make towards wider, realize better effect.

(5) this method has been introduced object-based method for video coding, utilizes the Alpha plane to differentiate the area attribute of this image block, and current block is divided into internal block, external block and boundary block; Object-based coding method has not only increased compression ratio and Y-PSNR in the non-performance that has further improved fractal compression on based on the basis of object method again, and makes compression have greater flexibility.

(6) in this method, when carrying out the DCP coded system, make full use of polarizability and directivity in the three-dimensional parallel shooting structure, the DCP way of search has been carried out the simplification computing, proposed a kind of novel fast search algorithm.

Description of drawings:

Fig. 1 a is a kind of based on object and the compression of the fractal binocular tri-dimensional video left order MCP compression process figure with decompression method for the present invention;

Fig. 1 b is a kind of based on object and the compression of the fractal binocular tri-dimensional video right order DCP compression process figure with decompression method for the present invention;

Fig. 1 c is a kind of based on object and the compression of the fractal binocular tri-dimensional video left order MCP decompression flow process figure with decompression method for the present invention;

Fig. 1 d is a kind of based on object and the compression of the fractal binocular tri-dimensional video right order DCP decompression flow process figure with decompression method for the present invention;

Fig. 1 e is a kind of based on object and the compression of the fractal binocular tri-dimensional video DCP fast algorithm flow chart with decompression method for the present invention;

Fig. 2 a is the left order image of the 2nd frame of standard testing video sequence " ion.cif ";

Fig. 2 b is the right order image of the 2nd frame of standard testing video sequence " ion.cif ";

Fig. 2 c is the 2nd frame left side purpose Alpha segmentation plane of a kind of standard testing video sequence " ion.cif " that obtains based on object and the compression of fractal binocular tri-dimensional video and decompression method of the present invention;

Fig. 2 d is the right purpose Alpha of the 2nd frame segmentation plane of a kind of standard testing video sequence " ion.cif " that obtains based on object and the compression of fractal binocular tri-dimensional video and decompression method of the present invention;

Fig. 2 e is a kind of based on object and the compression of fractal binocular tri-dimensional video and the decompression method result images of the 2nd frame left side purpose prospect object video of the standard testing video sequence " ion.cif " of decoding after the inventive method compressed encoding separately for the present invention;

Fig. 2 f is a kind of based on object and the compression of fractal binocular tri-dimensional video and the decompression method result images of the right purpose prospect object video of the 2nd frame of the standard testing video sequence " ion.ciff " of decoding after the inventive method compressed encoding separately for the present invention;

Fig. 3 is a kind of based on object and the compression of fractal binocular tri-dimensional video MCP and the DCP structural representation with decompression method for the present invention;

Fig. 4 a is a kind of based on object and fractal binocular tri-dimensional video compression and the four kind partition mode figures of decompression method to macro block for the present invention;

Fig. 4 b compresses four kinds of partition mode figure that the partition mode four of macro block further divided with decompression method for the present invention is a kind of based on object and fractal binocular tri-dimensional video;

Fig. 5 is the parallel camera system imaging schematic diagram of binocular solid;

Fig. 6 is a kind of based on object and the fractal binocular tri-dimensional video video compression and the quick search pattern figure of DCP of decompression method for the present invention;

Fig. 7 a is a kind of based on object and fractal binocular tri-dimensional video compression and the signature of decompression method to object-based three kinds of image blocks for the present invention;

Fig. 7 b compresses the signature that boundary block is belonged to the pixel of different video subject area with decompression method for the present invention is a kind of based on object and fractal binocular tri-dimensional video;

Fig. 8 a is the comparison diagram of a kind of Y-PSNR that preceding 5 frames of the right purpose of standard testing video sequence " ion.cif " is compressed based on object and fractal binocular tri-dimensional video compression and decompression method and traditional CPM/NCIM method of the present invention;

Fig. 8 b is the comparison diagram of a kind of compression ratio that preceding 5 frames of the right purpose of standard testing video sequence " ion.cif " is compressed based on object and fractal binocular tri-dimensional video compression and decompression method and traditional CPM/NCIM method of the present invention;

Fig. 8 c is a kind of time comparison diagram that preceding 5 frames of the right purpose of standard testing video sequence " ion.cif " is compressed based on object and fractal binocular tri-dimensional video compression and decompression method and traditional CPM/NCIM method of the present invention.

Embodiment:

To combine accompanying drawing that the inventive method is done further to specify below, be example with luminance component Y only, and the compression step of color difference components U and V is identical with luminance component.

The present invention proposes a kind of based on object and fractal binocular tri-dimensional video compression and decompression method; In binocular tri-dimensional video coding, it serve as basic layer with left passage; Right passage is an enhancement layer, and left passage adopts independent motion compensated prediction mode (MCP) to encode.At first utilizing methods of video segmentation to obtain the video object segmentation plane is the Alpha plane, and left purpose start frame is adopted piece dct transform coding, and the non-I frame of left purpose is carried out piece motion estimation/compensation coding.Utilize the Alpha plane to differentiate the area attribute of this image block then, if this piece does not then process this external block not in the object video zone of present encoding; If this piece is all in the object video zone of present encoding; In the former frame of internal block is the reference frame search window of left visual frequency, utilize all direction search method to seek the most similar match block, utilize Huffman coding method compression iterated function system coefficient at last; If the partial pixel of this piece is in the object video zone of present encoding, partial pixel in the object video zone of present encoding, does not then need individual processing to this boundary block.Right passage adopts MCP to add parallax compensation prediction mode (DCP) and encodes, and MCP similarly handles with left order, and the minimum conduct of Select Error predicts the outcome.When carrying out the DCP coded system, make full use of polarizability and directivity in the three-dimensional parallel shooting structure again.

Shown in accompanying drawing 1a and Fig. 1 b, a kind of based on object and fractal binocular tri-dimensional video compaction coding method, may further comprise the steps:

Step 1: preceding 5 frames with standard testing video sequence " ion.cif " are example.Utilize automatic video frequency Object Segmentation method that video sequence is cut apart automatically, obtain the Alpha plane.Object in the video can define through the Alpha plane, and can independently carry out compressed encoding.Accompanying drawing 2b is right purpose the 2nd frame of standard testing video sequence " ion.cif ", and accompanying drawing 2d is the Alpha plane of this frame, and prospect representes that with black background is represented with white.Shown in accompanying drawing 2b, two object videos are arranged in the image, the Alpha plane just can be a bianry image so, represents prospect with black, white is represented background.Each object can compress independently, makes each object form a code stream.So when decompressing, need not obtain entire image, but can recover and control each object separately.The result of decompression prospect object video is accompanying drawing 2f separately.Introduce object-based coding and not only improved compression performance, and increased the flexibility of method for video coding.

Step 2: for left order; At first judge whether frame, if the I frame then carries out the piece division of the fixed size of non-overlapping copies to this frame into I; Each image block is adopted the I frame I picture compression method based on the piece dct transform respectively; This two field picture is carried out independent Code And Decode, and the pixel after the decoding then keeps this pixel value as if belonging to object video to be encoded in the Alpha plane, otherwise with forwarding step 14 to after the pixel value zero setting; Said I frame is the picture frame that only carries out intraframe coding in video sequence start frame or the video sequence; The said image block that present frame is divided into the non-overlapping copies of fixed size is called macro block; Saidly current macro is carried out the piece that tree-shaped division obtains be called fritter; The frame of said present frame for compressing, said reference frame is the former frame of encoding and rebuilding of present frame; The set of said all pieces of present frame is called sub-piece territory; The set of all pieces of said former frame is called father's piece territory; Piece in said dct transform adopts the fixed size pattern.For right order, forward step 6 to;

To video sequence " ion.cif " start frame, each image block is adopted the I frame I picture compression method based on the piece dct transform respectively, start frame is divided into the sub-piece of 8 * 8 non-overlapping copies, each sub-piece is carried out dct transform respectively.Discrete cosine transform is transformed into 8 * 8 coefficient matrix Y with 8 * 8 image pattern X.Conversion process (comprising inverse transformation) can be represented with transformation matrix A.

Forward DCT (FDCT) conversion of 8 * 8 sample block is following:

Y＝AXA ^T (6)

Oppositely DCT (IDCT) is as follows:

X＝A ^TYA (7)

Wherein A is 8 * 8 transformation matrix.Each element among the A is following:

$A_{\mathrm{ij}}=C_i\cos \frac{(2j+1)\mathrm{i\π}}{16}---\left(8\right)$

Wherein

$C_i=\sqrt{\frac{1}{8}}$ (i＝0) $C_i=\frac{1}{2}$ (i＞0) (9)

I, j are respectively the row and column of matrix A.

Conversion coefficient is quantized and encodes, change step 14 over to.Accompanying drawing 3 is the MCP and the DCP structural representation of binocular tri-dimensional video coding.

Step 3:,, current non-I two field picture is carried out 16 * 16 macroblock partitions of non-overlapping copies with motion compensated prediction (MCP) coding of conventional monocular if left order is non-I frame; Calculate 16 * 16,16 * 8,8 * 16 of present frame non-overlapping copies respectively; 8 * 8; 8 * 4,4 * 8, the pixel of 4 * 4 fritter and with the quadratic sum of pixel.Calculating reference frame then respectively is that the size of step-length is respectively 16 * 16,16 * 8,8 * 16 with 1 in the decompressing image of former frame just; 8 * 8,8 * 4,4 * 8; The pixel of 4 * 4 fritter and with the quadratic sum of pixel, to reduce the double counting in the piece matching process.Begin to handle from first macro block, successively all 16 * 16 macro blocks of present frame are encoded, change step 4 over to;

Step 4: 16 * 16 macro block according to non-overlapping copies in present frame carries out the piece coupling, utilizes the Alpha plane to differentiate the area attribute of this image block; According to the type of this macro block, make following selection, if this macro block does not process this piece, continue to handle next macro block not in the Alpha plane; If this macro block all is positioned at the Alpha plane, change step 5 over to; If this macro block partly is positioned at the Alpha plane, change step 13 over to.If after the whole couplings of present frame macro block finish, change step 14 over to.The said not piece in the object video zone of present encoding is called external block, and said piece in the object video zone of present encoding all is called internal block, and the said partial pixel not piece in the object video zone of present encoding is a boundary block.

Step 5: successively all 16 * 16 macro blocks of present frame are encoded, in the search window in father's piece territory whole macro block is carried out the piece motion estimation/compensation.The foundation of coupling is fractal iterated function system principle, briefly introduces Fundamentals of Mathematics-iterated function system (IFS:Iterative Function System) theory of fractal pattern compression.If D is R ⁿThe subclass of Euclidean space, ω is the mapping of D → D, if there is a real number C, 0≤C＜1 makes for R ⁿOn tolerance d, satisfy any x, y ∈ D, d (ω (x), ω (y))≤C is arranged, and (d (x, y)) claims that then ω is a compressing mapping, and real number C is called the compressibility factor of ω.Complete metric space (X, d) and n compressing mapping ω _i: (its compressibility factor is respectively C to X → X ₁, C ₂... C _n) together, just form an iterated function system (Iterated Function System), be called for short IFS, note is made { X: ω ₁, ω ₂..., ω _n.C=max (C ₁, C ₂..., C _n) be called the compressibility factor of IFS.Therefore { R ²: ω ₁, ω ₂, ω ₃Be exactly an IFS.

In the fractal pattern compression, general matching criterior is RMS, that is:

$\mathrm{RMS}=\frac{1}{N}[\underset{i=1}{\overset{\mathrm{<figure-callout\; id="2"\; label="N\; r\; i"\; filenames="HSA00000322123400063.png,HSA00000322123400083.png"\; state="\{\{state\}\}">N</figure-callout>}}{\mathrm{\&Sigma;}}}{r}_i^{}{}_2^{}+s(s\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_i^2-2\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{r}_i{d}_i+2o\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_i^2)+o(N\&CenterDot;o-2\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{r}_i)]$

S wherein, o is respectively:

$s=\frac{[N\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{r}_i{d}_i-\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{r}_i\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_i]}{[N\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_i^2-{\left(\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_i\right)}^2]}$

$o=\frac{1}{N}[\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{r}_i-s\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_i]$

Wherein, N is the number of sub-piece and father's piece pixel, r _iBe the pixel value of sub-piece, d _iPixel value for father's piece.

At first establish the match error threshold γ=tol * tol * no of stator block, wherein tol changes according to different sub-block sizes, and big sub-piece tol is just big more, and little sub-piece tol is just little.In this example, the tol that we get 16 * 16 macro blocks is that the tol of 10.0,8 * 8 sub-pieces is that the tol of 8.0,4 * 4 sub-pieces is 6.0, and no is the number of pixels that current sub-block belongs to this object video zone.

At first set the match error threshold γ of 16 * 16 macro blocks ₁₆=10.0 * 10.0 * no, the position with current sub-block in father's piece territory of reference frame begins in the search window 15 * 15 whole macro block to be carried out the piece coupling, if matching error RMS is less than beginning preset threshold γ ₁₆, preserve current IFS coefficient and comprise scale factor s, skew o, father's piece is with respect to the coordinate offset x of current sub-block, and y returns step 5, continues the coupling of next macro block.

Otherwise, according to tree this macro block is divided, the division of macro block there are four kinds of patterns; Like accompanying drawing 4a, pattern one is one 16 * 16 fritter, and pattern two is two 8 * 16 a fritter; Pattern three is two 16 * 8 a fritter, and pattern four is four 8 * 8 a fritter.

1, at first calculates, if two fritters all satisfy RMS＜γ in the pattern two by the division of pattern two ₁₆, preserve current IFS coefficient and comprise scale factor s, skew o, and father's piece is with respect to the coordinate offset x of current sub-block, y, and the division of stop piece forward 5 to;

2 otherwise divide by pattern three, if two fritters all satisfy RMS＜γ in the pattern three ₁₆, preserve current IFS coefficient and comprise scale factor s, skew o, and father's piece is with respect to the coordinate offset x of current sub-block, y, and the division of stop piece forward 5 to;

3 otherwise divide according to four pairs of current macro of pattern, this moment, match error threshold was set to γ ₈=8.0 * 8.0 * no is if 4 fritters in the pattern four all satisfy RMS＜γ ₈, preserve current IFS coefficient and comprise scale factor s, skew o, and father's piece is with respect to the coordinate offset x of current sub-block, y, and the division of stop piece forward 5 to;

4 otherwise each fritter in the pattern four is divided according to the mode division among the accompanying drawing 4b in proper order, can be divided into 18 * 8 fritter successively, 24 * 8 fritter, 28 * 4 fritter, 44 * 4 fritter.Here only the matching process of first 8 * 8 fritter is set forth, the matching process of other 38 * 8 fritters is identical with first, repeats no more.At first divide, carry out the piece coupling, if the matching error RMS of two sub-block is all less than γ according to 24 * 8 fritter ₈The time, preserve current IFS coefficient and comprise scale factor s, skew o, and father's piece is with respect to the coordinate offset x of current sub-block, y, and the division of stop piece.Otherwise, carry out the division of piece according to 28 * 4 dividing mode, this two sub-block is carried out the piece coupling, if the matching error RMS of two sub-block is all less than γ ₈The time, preserve current IFS coefficient and comprise scale factor s, skew o, and father's piece is with respect to the coordinate offset x of current sub-block, y, and the division of stop piece.Otherwise, this sub-piece is divided into 44 * 4 fritter, match error threshold is made as γ simultaneously ₄=6.0 * 6.0 * no carries out piece coupling respectively to four fritters, and the IFS coefficient that writes down every sub-block respectively comprises scale factor s, skew o, and father's piece is with respect to the coordinate offset x of current sub-block, y, and the division of stop piece forward 5 to;

5, return step 5, continue next macroblock encoding.

If all macro blocks of present frame have all been encoded finish, and be left order, then forward step 14 to, if right order then continues execution in step seven;

Step 6: to right order image, at first carry out 16 * 16 macroblock partitions of non-overlapping copies, calculate 16 * 16,16 * 8,8 * 16,8 * 8,8 * 4,4 * 8 of present frame non-overlapping copies respectively, the pixel of 4 * 4 fritter and with the quadratic sum of pixel.Calculating respectively then in the decompressing image of corresponding frame in decompressing image and the left order of the just right order former frame of reference frame with 1 is that the size of step-length is respectively 16 * 16; 16 * 8,8 * 16,8 * 8; 8 * 4; The pixel of 4 * 8,4 * 4 fritter and with the quadratic sum of pixel, to reduce the double counting in the piece matching process.Encoder is at first searched in right purpose former frame, selects the D piece that matees most.Begin to handle from first macro block, successively all 16 * 16 macro blocks of present frame are encoded, change step 5 over to;

Step 7: encoder is selected the D piece that matees most at the corresponding frame search of left order.Begin to handle from first macro block, successively all 16 * 16 macro blocks of present frame are encoded, change step 8 over to;

Step 8: 16 * 16 macro block according to non-overlapping copies in present frame carries out the piece coupling, according to the type of this macro block, makes following selection, if this macro block does not process this piece not in the Alpha plane, continues to handle next macro block; If this macro block all is positioned at the Alpha plane, change step 9 over to; If this macro block partly is positioned at the Alpha plane, change step 13 over to.If after the whole couplings of present frame macro block finish, change step 14 over to.

Step 9: at first the father piece identical with the current sub-block position carried out piece coupling, obtains RMS, and preserve the iterated function system coefficient, this coefficient comprise father's piece and sub-piece the relative displacement vector (x, y), scale factor s and displacement factor o; Successively all macro blocks of present frame are encoded, at first this macro block is carried out the piece coupling in the search window in father's piece territory; In the matching process that carries out sub-piece and father's piece, the position of sub-piece is as the initial search point of father's piece, and the size of father's piece is with the big or small identical of sub-piece and change step 10 over to; If the matching error RMS of gained is less than beginning preset threshold γ, preserving current iterated function system coefficient is the IFS coefficient, changes next macro block of step 9 coding over to; Otherwise; According to tree this piece is divided successively, and the fritter that each division obtains is changed over to step 10 computation of match errors RMS respectively, if RMS is less than setting threshold γ; Stop to divide and writing down this fritter IFS coefficient, change next macro block of step 9 coding over to; Otherwise continue to divide, up to current block being divided into predefined smallest blocks, changing step 10 over to and calculate RMS, record IFS coefficient changes next macro block of step 9 coding over to; Last and step 5 gained result compares, and the minimum conduct of Select Error predicts the outcome; If all macro blocks of present frame have all been encoded finish, then forward step 14 to;

Step 10: Fig. 1 e is a DCP fast algorithm flow chart, the concrete realization as follows: according to the constraint of solid geometry polar curve, Fig. 5 mid point P (x _c, y _c, z _c) must be positioned on the corresponding polarized line at the point on the image of the left and right sides; Therefore can know that by polarizability it is on the horizontal line that optimal match point is positioned on the polarized line, but in reality, possibly have certain error, adopt in vertical direction earlier two points of interlacing search, for example point among Fig. 62 and point 3 up and down; If the RMS of these two points all forwards step 11 to greater than the RMS that puts on the polarized line, otherwise forward step 12 to;

Two video cameras are watched the same characteristic point P (x in space at synchronization _c, y _c, z _c), it is respectively p at the coordinate of left image and right image _Left=(X _Left, Y _Left), p _Right=(X _Right, Y _Right).Its mid point P (x _c, y _c, z _c) being called the plane of polarization with two determined planes of photocentre, the intersection of plane of polarization and left and right sides image is called polarized line.Obtain by geometrical relationship:

$\left\{\begin{array}{c}{X}_{\mathrm{left}}=f\frac{x_c}{z_c}\\ X_{\mathrm{right}}=f\frac{(x_c-B)}{z_c}\\ Y=f\frac{y_c}{z_c}\end{array}\right.---\left(10\right)$

Step 11:

can know by difference vector; For same characteristic point, its perspective projection left side image can be thought the local translation of right image along horizontal negative sense.That is to say that the image coordinate of the same space point on right image takes over than the image coordinate on left image, therefore when search, only need to carry out along a direction.In the parallel camera system of solid, only need along continuous straight runs search to get final product when carrying out the DCP search.Optimal match point is positioned at the right side of initial point, only searches for along the right side of initial point, in search procedure, carries out the great-jump-forward search; As shown in Figure 6, for example put 1 mated after, search for to two points of right septum, promptly search point 4, calculate RMS, and change step 10 over to and carry out the vertical direction search; Like the RMS of fruit dot 4 RMS value less than point 1, will put 4 as initial search point, rotate back into the step 11 starting position, under the continuation horizon scan a bit;

Otherwise the RMS that current point is described is greater than more preceding RMS, for example puts 13 the RMS RMS greater than point 7, explains that so optimal match point is between these 2; Less than point 9, this optimal match point is positioned at frame of broken lines so like the RMS of fruit dot 8, and search for point range and dwindle greatly this moment; And in order to maximize matching precision, this paper in this frame of broken lines not the point of search search for entirely, if instead put 8 RMS RMS greater than point 9; Optimal match point is positioned at solid box so, and solid box is searched for entirely, so far selects the minimum point of RMS as final matching results; Finish the DCP search procedure, continue execution in step nine;

Step 12: select 2 points that middle RMS is minimum; For example the RMS of this time point 2 is minimum; Search for point 1 so in vertical direction again and put the point between 2; Calculate its RMS, the point of selecting error minimum in these three points changes step 11 over to and continues to search for along the polarized line direction as the optimal match point on this vertical direction;

Step 13: for when matched sub-block and the father's piece; Avoiding belonging to different object pixels obscures mutually; Each pixel is done a mark in the Alpha plane; Indicate which object is the pixel that is labeled belong to, shown in accompanying drawing 7b, the pixel of this boundary block is marked as S1 and two parts of S2.

The concrete compression method of boundary block: what suppose current compression is object 1 (during compressed object 2, method is identical), i.e. the object at S1 place.For the amount relevant, only calculate the pixel value in the S1 zone, and the pixel in the S2 zone is not considered with sub-piece; For the amount relevant with father's piece, if with father's piece of the corresponding position of sub-piece in a certain pixel d _iAlso belong to the S1 zone, then use d _iOriginal pixel value, otherwise, replace d according to specific calculated value _i, what use among the present invention is that the pixel average that belongs to the S1 zone in father's piece replaces d _iNeed to prove that the coupling mapping is only carried out between homogeneous blocks, promptly sub-piece and father's piece must be boundary block simultaneously or be internal block (external block) simultaneously.After choosing criterion and confirm the pixel value of current block and father's piece according to the pixel of said current block and father's piece; Computation of match errors RMS is if RMS, then preserves current gained coefficient less than threshold gamma; Otherwise current boundary block is divided into fritter, each fritter is returned step 5 respectively handle;

Step 14: all IFS coefficients are carried out the huffman coding, and the huffman coding is according in the set (VLC) of probability with each sign map to one avriable length codes that occurs, and reduces the statistical redundancy of IFS coefficient data.Judge whether present frame is last frame, if last frame finishes coding, otherwise, change step 2 over to and continue to handle the next frame image.

Shown in accompanying drawing 1c and Fig. 1 d, a kind of based on object and fractal binocular tri-dimensional video decompression method, may further comprise the steps:

Step I: at first read in left order compressed information, comprise the compression frame number, every frame wide and high, I frame reconstruction quality, the interval of inserting the I frame;

Step II: judge that whether decoded frame is the I frame, if the I frame changes Step II I over to, otherwise changes step IV over to;

Step II I: for the I frame, from compressed file, read in code stream, and read in the Alpha plane of this frame, the decompression mode that is similar to JPEG is decoded, and carries out anti-dct transform, obtains the pixel value of each piece of 8 * 8; File after the decoding comprises based on the video file of different objects and complete video file, in object-based video file, according to the Alpha plane; Judge corresponding object properties; Belong to this object pixels and keep, do not belong to this object pixels zero setting, frame number adds one and changes step VII over to;

Step IV:, at first calculate left order former frame and be in the reference frame according to the pixel of the fritter of setting all macro blocks that step-length divides and obtaining through tree-shaped division and, pixel quadratic sum for non-I frame;

Step V: read in right order compressed information, comprise the compression frame number, every two field picture wide and high, I frame compression quality and the interval of inserting the I frame; Calculate the corresponding frame of right order former frame and left order and be in the reference frame according to the pixel of the fritter of setting all macro blocks that step-length divides and obtaining through tree-shaped division and, pixel quadratic sum, change step VI over to;

Step VI: from compressed file, read in the division information of piece and the Alpha plane of Huffman code stream and this frame; And obtain the dividing mode of these all macro blocks of frame and the iterated function system coefficient of each fritter according to above information; Decode according to each macro block, when decompressing for each macro block; At first judge the dividing mode of this macro block when coding,, find and this corresponding zone of sub-piece that the formula below utilizing then obtains the pixel value of this sub-piece in father's piece territory for each sub-block,

r _i＝s·d _i+o

R wherein _iBe the pixel value of sub-piece to be decoded, d _iBe the pixel value in father's piece territory, s is a scale factor, and o is a displacement factor.

In object-based decode procedure, utilize the Alpha plane to differentiate the area attribute of this image block, current block is divided into internal block, external block and boundary block; The pixel that only belongs to this subject area in the current block is just decoded; In father's piece territory, only utilize the pixel that belongs to the same target zone to decode; If partial pixel does not belong to this object video in certain sub-block in father's piece territory; The value of this part pixel replaces with the mean value of the pixel that belongs to this subject area in this sub-piece, carries out fractal decoding.

Step VII: judge this moment, whether all frames all decoded, finished if all decode, and finished decode procedure, otherwise changed Step II over to.

The binocular tri-dimensional frequency sequence of handling is a yuv format, adopts above-mentioned seven steps to handle in 3 components each respectively.

This method is selected the implementation language of Visual C++6.0 as said method, and CPU does

Core ^TM2DuoT8300, the 2.4GHz dominant frequency, memory size is 2G, and standard testing video sequence " ion.cif " has been carried out based on fractal binocular tri-dimensional video coding experiment.

Table 1, table 2 and table 3 are respectively the comparison based on the average peak signal to noise ratio of the preceding 5 frames compression of object 1 and object 2, average compression ratio, average compression time.First frame time is to handle for three purpose total times in the table 3, and other frame is similar.Can find out that this method improves on compression ratio.The restored image quality does not obviously reduce simultaneously, can guarantee watching of human eye.But on compression time, this method is greatly improved, and has approximately reduced by 20 times, makes encoding and decoding speed be significantly improved, and has improved the performance of fractal binocular tri-dimensional video encoding and decoding.Utilize this method to preceding 5 frames of " ion.cif " respectively based on object 1 and based on the compression ratio of object 2 to such as shown in the accompanying drawing 8a; Utilize this method to preceding 5 frames of " ion.cif " based on the Y-PSNR of object 1 to such as shown in the accompanying drawing 8b; Utilize this method to the compression time of preceding 5 frames of " ion.cif " to such as shown in the accompanying drawing 8c.

The compression performance average contrast of table 1CPM/NCIM method and the inventive method

The Y-PSNR average contrast of table 2CPM/NCIM method and the inventive method

The preceding 5 frame times contrast of table 3CPM/NCIM method and the inventive method

In the fractal image, coding rate and compression time always are the problem of being concerned about the most, also are the links that is difficult to solution.This method can have been carried out the simplification computing to the DCP way of search according to polarizability and the directivity in the parallel shooting structure of solid, has strengthened the practicality of binocular tri-dimensional video coding greatly.From result data, can see that this method having guaranteed certain image restoration quality prerequisite, has improved compression ratio to a certain extent, and coding rate and compression time there has been the improvement of significant proportion, improved the practicality that fractal binocular tri-dimensional video is encoded.

Claims (5)

1. one kind based on object and fractal binocular tri-dimensional video compaction coding method, it is characterized in that concrete steps are following:

Step 1: utilize automatic video frequency Object Segmentation method that video sequence is cut apart automatically, the video object segmentation plane that obtains each frame is the Alpha plane, confirms the object video zone that will encode according to the video object segmentation plane that obtains; Call present frame to the frame that is compressing, call reference frame to the former frame of encoding and rebuilding of present frame;

Step 2: for left order; At first judge whether frame, if the I frame then carries out the piece division of the fixed size of non-overlapping copies to this frame into I; Each image block is adopted the I picture compression method based on the piece dct transform respectively; This two field picture is carried out independent Code And Decode, and the pixel after the decoding then keeps this pixel value as if belonging to object video to be encoded in the Alpha plane, otherwise with forwarding step 14 to after the pixel value zero setting; Said I frame is the picture frame that only carries out intraframe coding in video sequence start frame or the video sequence; The image block that present frame is divided into the non-overlapping copies of fixed size is called macro block; Current macro is carried out the piece that tree-shaped division obtains be called fritter; The set of said all pieces of present frame is called sub-piece territory; The set of all pieces of said former frame is called father's piece territory; Piece in said dct transform adopts the fixed size pattern; For right order, forward step 6 to;

Step 3: if left order is non-I frame; Motion compensated prediction (MCP) coding with conventional monocular; Present frame is carried out the macroblock partitions of non-overlapping copies; The pixel of the fritter that calculates these macro blocks then and obtain through tree-shaped division and, pixel quadratic sum, and left order former frame reconstructed image is in the reference frame are according to the pixel of the fritter of setting all macro blocks that step-length divides and obtaining through tree-shaped division and, pixel quadratic sum;

Step 4: to the current image block of handling is current block, utilizes the Alpha plane to differentiate the area attribute of this image block; If this piece in the object video zone of present encoding, does not process this piece; If this piece in the object video zone of present encoding, changes step 5 over to all; If the partial pixel of this piece is in the object video zone of present encoding, partial pixel in the object video zone of present encoding, does not need individual processing, changes step 13 over to; If all macro blocks all dispose, then forward step 14 to; The said not piece in the object video zone of present encoding is called external block, and said piece in the object video zone of present encoding all is called internal block, and the said partial pixel not piece in the object video zone of present encoding is a boundary block;

Step 5: successively all macro blocks of present frame are encoded, at first this macro block is carried out the piece coupling in the search window in father's piece territory; In the matching process that carries out sub-piece and father's piece, the position of sub-piece is as the initial search point of father's piece, and the size of father's piece is big or small identical with sub-piece; If matching error RMS is less than beginning preset threshold γ, preserving current iterated function system coefficient is the IFS coefficient, changes next macro block of step 5 coding over to; Otherwise, according to tree this piece is divided successively, and,, is changed over to next macro block of step 5 coding if RMS less than setting threshold γ, stops to divide and writing down this fritter IFS coefficient to the fritter difference computation of match errors RMS that each division obtains; Otherwise continue to divide, up to current block being divided into predefined smallest blocks, record IFS coefficient; Change next macro block of step 5 coding over to; Said search window is the rectangular search zone in reference frame; Said IFS coefficient comprises that (x is y) with scale factor s, displacement factor o in father's piece position; If all macro blocks of present frame have all been encoded finish, and be left order, then forward step 14 to, if right order then continues execution in step seven;

Step 6: to right order image; At first carry out the macroblock partitions of non-overlapping copies; The pixel of the fritter that calculates these macro blocks then and obtain through tree-shaped division and, pixel quadratic sum, and right order former frame reconstructed image reference frame are according to the pixel of the fritter of setting all macro blocks that step-length divides and obtaining through tree-shaped division and, pixel quadratic sum; To reduce the double counting in the piece matching process, forward step 5 to;

Step 7: calculate corresponding two field picture reference frame in the left order,, forward step 8 to according to the pixel of the fritter of setting all macro blocks that step-length divides and obtaining through tree-shaped division and, pixel quadratic sum;

Step 8: to the current image block of handling is current block, utilizes the Alpha plane to differentiate the area attribute of this image block; If this piece in the object video zone of present encoding, does not process this piece; If this piece in the object video zone of present encoding, changes step 9 over to all; If the partial pixel of this piece is in the object video zone of present encoding, partial pixel in the object video zone of present encoding, does not need individual processing, changes step 13 over to; If all macro blocks all dispose, then forward step 14 to;

Step 9: at first the father piece identical with the current sub-block position carried out piece coupling, obtains RMS, and preserve the iterated function system coefficient, this coefficient comprise father's piece and sub-piece the relative displacement vector (x, y), scale factor s and displacement factor o; Successively all macro blocks of present frame are encoded, at first this macro block is carried out the piece coupling in the search window in father's piece territory; In the matching process that carries out sub-piece and father's piece, the position of sub-piece is as the initial search point of father's piece, and the size of father's piece is with the big or small identical of sub-piece and change step 10 over to; If the matching error RMS of gained is less than beginning preset threshold γ, preserving current iterated function system coefficient is the IFS coefficient, changes next macro block of step 9 coding over to; Otherwise; According to tree this piece is divided successively, and the fritter that each division obtains is changed over to step 10 computation of match errors RMS respectively, if RMS is less than setting threshold γ; Stop to divide and writing down this fritter IFS coefficient, change next macro block of step 9 coding over to; Otherwise continue to divide, up to current block being divided into predefined smallest blocks, changing step 10 over to and calculate RMS, record IFS coefficient changes next macro block of step 9 coding over to; Last and step 5 gained result compares, and the minimum conduct of Select Error predicts the outcome; If all macro blocks of present frame have all been encoded finish, then forward step 14 to;

Step 10: two points are searched in interlacing about the elder generation in vertical direction; If the RMS of these two points all forwards step 11 to greater than the RMS that puts on the polarized line, otherwise forward step 12 to;

Step 11: only search for, in search procedure, carry out the great-jump-forward search along the right side of initial point; Search for to two points of right septum, calculate RMS, change step 10 over to and carry out the vertical direction search; If the RMS of current point less than more preceding RMS, as new initial search point, rotates back into the step 11 starting position with current point, continue under the horizon scan a bit; Otherwise the RMS that current point is described is greater than more preceding RMS, and promptly optimal match point selects the proper cell territory to search for entirely between these 2, so far selects the minimum point of RMS as final matching results, finishes the DCP search procedure, continues execution in step nine;

Step 12: select 2 points that middle RMS is minimum; Search for the point between adjacent 2 in vertical direction again; Calculate its RMS, the point of Select Error minimum changes step 11 over to and continues to search for along the polarized line direction as the optimal match point on this vertical direction;

Step 13: this step individual processing boundary block; The boundary block of present frame only belongs to search matched in boundary block and the internal block in father's piece; The pixel of current block and father's piece is chosen criterion: in current block, only calculate the pixel value in the object video zone that is positioned at present encoding, only in the same video subject area of reference frame, carry out the piece coupling; In father's piece, if also belong to this object video zone, then adopt original pixel value with a certain pixel in father's piece of the opposite position of current block, otherwise, replace with other mean value that belongs to this object video area pixel; Forward step 5 to;

Step 14: all IFS coefficients are carried out the Huffman coding, reduce the statistical redundancy of IFS coefficient data; Judge whether present frame is last frame, if last frame finishes coding; Otherwise, return step 2 and continue to handle the next frame image.

2. said a kind of based on object and fractal binocular tri-dimensional video compaction coding method according to claim 1, it is characterized in that: the video sequence of processing is a yuv format, adopts above-mentioned 14 steps to handle in 3 components each respectively.

3. said a kind of based on object and fractal binocular tri-dimensional video compaction coding method according to claim 1, it is characterized in that: in the said step 5 macro block is adopted tree-shaped division, the piece coupling adopts the matching error criterion; The matching error RMS of sub-piece and father's piece is:

$\mathrm{RMS}=\frac{1}{N}[\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{r}_i^2+s(s\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_i^2-2\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{r}_i{d}_i+2o\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_i^2)+o(N\&CenterDot;o-2\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{r}_i)]---\left(1\right)$

Wherein parameter s and o are respectively:

$s=\frac{[N\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{r}_i{d}_i-\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{r}_i\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_i]}{[N\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_i^2-{\left(\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_i\right)}^2]}---\left(2\right)$

$o=\frac{1}{N}[\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{r}_i-s\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_i]---\left(3\right)$

Wherein, N is the number of sub-piece and father's piece pixel, r _iBe the pixel value of sub-piece, d _iPixel value for father's piece;

Calculate the block-matching error RMS of current macro in reference frame, wherein r _iBe the pixel value of sub-piece, d _iIt is the pixel value of father's piece; If RMS is less than pre-set threshold γ, record IFS coefficient, IFS coefficient comprise match block displacement vector (x, y) with formula 2,3 in s and o, handle next macro block; Otherwise, current macro is carried out tree-shaped division, calculate the RMS that divides the back fritter, if less than threshold gamma, then stop to divide, otherwise continue to divide, till sub-piece reaches predefined smallest blocks.

4. said a kind of based on object and fractal binocular tri-dimensional video compaction coding method according to claim 1; It is characterized in that: the searching method of vertical direction has used the polarizability of binocular tri-dimensional video in the said step 10; It is on the horizontal line that optimal match point is arranged on the polarized line: in the parallel camera system of solid, along continuous straight runs carries out the DCP search;

Two video cameras are watched the same characteristic point P (x in space at synchronization _c, y _c, z _c), be respectively p at the coordinate of left image and right image _Left=(X _Let, Y _Left), p _Right=(X _Right, Y _Right); Its mid point P (x _c, y _c, z _c) being called the plane of polarization with two determined planes of photocentre, the intersection of plane of polarization and left and right sides image is called polarized line; Obtain by geometrical relationship:

$\left\{\begin{array}{c}{X}_{\mathrm{left}}=f\frac{x_c}{z_c}\\ X_{\mathrm{right}}=f\frac{(x_c-B)}{z_c}\\ Y=f\frac{y_c}{z_c}\end{array}\right.---\left(4\right)$

Wherein f is the focal length of left and right cameras, and B is the parallax range between the left and right cameras.

5. said a kind of based on object and fractal binocular tri-dimensional video compaction coding method according to claim 1; It is characterized in that: the searching method of horizontal direction has used the directivity of binocular tri-dimensional video in the said step 11: the projected position difference vector of the same space o'clock in two planes of delineation be parallax

wherein f be the focal length of left and right cameras, B is the parallax range between the left and right cameras; Parallax D is the function of depth z, the expression relative depth; For same characteristic point, its perspective projection left side image is the local translation of right image along horizontal negative sense, and the image coordinate of the same space point on right image takes over than the image coordinate on left image, when search, carries out along a direction.

Images