GB2349535A - Estimation of fast motion in an image - Google Patents

Estimation of fast motion in an image Download PDF

Info

Publication number
GB2349535A
GB2349535A GB0003276A GB0003276A GB2349535A GB 2349535 A GB2349535 A GB 2349535A GB 0003276 A GB0003276 A GB 0003276A GB 0003276 A GB0003276 A GB 0003276A GB 2349535 A GB2349535 A GB 2349535A
Authority
GB
United Kingdom
Prior art keywords
blocks
sub
reference block
block
matching
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
GB0003276A
Other versions
GB0003276D0 (en
Inventor
Jong-Nam Kim
Yong-Lae Cha
Tae-Sun Choi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electro Mechanics Co Ltd
Original Assignee
Samsung Electro Mechanics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electro Mechanics Co Ltd filed Critical Samsung Electro Mechanics Co Ltd
Publication of GB0003276D0 publication Critical patent/GB0003276D0/en
Publication of GB2349535A publication Critical patent/GB2349535A/en
Withdrawn legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136Incoming video signal characteristics or properties
    • H04N19/137Motion inside a coding unit, e.g. average field, frame or block difference
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136Incoming video signal characteristics or properties
    • H04N19/14Coding unit complexity, e.g. amount of activity or edge presence estimation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Abstract

A method for estimating a fast motion in a video encoder is disclosed. In the matching scan for matching the respective candidate blocks of the search window of the preceding frame to the reference block of the present frame, there are adopted a spiral search direction and an adaptive matching scan for carrying out the matching scan in the decided direction based on the image complexity. Further, there is applied a PDE (partial distortion estimation) during the matching scan, so that the calculation amount can be significantly reduced in the full search, thereby making it possible to carry out the motion estimation at a high speed. Thus the full scan for the motion estimation can be embodied in the form of an algorithm.

Description

ESTIMATION OF FAST MOTION IN AN IMAGE The present invention relates to a method for estimating a fast motion in a video encoder which compresses the video data. Particularly, but not exclusively, the present invention relates to a method for estimating a fast motion based on the image complexity in a video encoder, in which in a matching scan for matching the respective candidate blocks of the search window of the preceding frame to the reference block of the present frame, there is applied an adaptive matching scan for carrying out the matching scan in the decided direction based on the image complexity, and there is applied a PDE (partial distortion estimation) during the matching scan, so that the calculation amount can be significantly reduced in the full search, thereby making it possible to carry out the motion estimation at a high speed.
Generally, the amount of an image data is greatly voluminous compared with the audio or character data, and therefore, if the image data is not compressed, a real time processing becomes impossible. Thus by compressing the image data by a proper method, a real time processing becomes possible when storing or transmitting the data. As the international standard for compressing the image data, there are JPEG for still picture standard, MPEG1 (television broadcasting) for moving picture used in the television broadcasting, MPEG2 used in the satellite broadcasting, and MPEG4 which is being developed at present for transmitting low speed bits.
Meanwhile, the compression of the picture data is realized by removing the repetitive data. In the repetition, there are a spatial repetition within one frame, a probability repetition, and a time repetition between two frames. First, the spatial repetition originates from the similarity of the values of the adjacent pixels, and this corresponds to the case where the value of an arbitrary pixel is similar to the values of the adjacent pixels. This spatial repetition is processed by using the DCT (discrete cosine transform), and this transform causes the picture information to be gathered at the left upper portion of the frame. Meanwhile, the probability repetition originates from the similarities of symbols, and this corresponds to the case where the data are not uniformly distributed in probability, but an arbitrary symbol has a value similar to the values of the adjacent symbols. In processing this probability repetition, there is used a variable length coding which is an entropy coding method. In this variable length coding, the bits are allocated proportionally to the sizes of the symbols.
Finally, the time repetition originates from the similarity between an arbitrary frame and the preceding frame. This time repetition is processed by using the ME/MC (motion estimation/ motion compensation). In the ME (motion estimation), a motion vector between the present frame and the preceding frame is detected. Then in the motion compensation, a new frame is formed by using the detected motion vector, and then, the newly formed frame is subtracted from the present frame, thereby removing the same data from between the present frame and the newly formed frame.
FIG. 1 illustrates the overall constitution of the image data transceiving system. As shown in FIG. 1, the image data transceiving system includes: an image data transmitter 100 for transmitting an image data after compressing it; a satellite 1 for receiving the image data from the image data transmitter 100 to transmit it to a receiver 200; and the receiver 200 receiving the picture signals from the satellite 1 to release the compression to restore it to the original form.
The image data transmitter 100 includes: an MPEG source encoder 110 for compressing the image data and a sound data; a text encoder 130 for compressing the text; a channel encoder 150 for inserting a channel information into the encoded data; and an RF section 170 for modulating the encoded data.
The image data receiver 200 includes: a base band processing section 210 for restoring the received data into a base band image data by removing the carrier waves after receipt of the image data from the satellite 1; a channel decoder 220 for carrying out distortion detections, distortion corrections and restorations on the image data of the base band processing section 210; and an MPEG decoder 230 for releasing the compression of the image data of the channel decoder 220 to restore it to the original form.
FIG. 2 is a block diagram showing the constitution of the MPEG source encoder of the transmitter of FIG. 1.
As shown in FIG. 2, the MPEG source encoder includes: an 8*8 block forming section 111 for dividing each of the incoming frame data into 8*8 blocks; a subtractor 112 for subtracting the newly formed frame from the present frame of the 8*8 block forming section 111; an 8*8 DCT section (discrete cosine transform) 113 for carrying out a DCT on the present frame of the subtractor 112; an 8*8 quantizer 114 for quantizing the frame of the 8*8 DCT section 113; and a variable length coding section 115 for carrying out a variable length coding on the present frame of the 8*8 quantizer 114.
The MPEG source encoder includes: an 8*8 inverse quantizer 117 for quantizing the frame of the 8*8 quantizer 114; an 8*8 inverse DCT section 118 for carrying out an inverse DCT on the frame of the 8*8 inverse quantizer 117; an adder 119 for adding the newly formed frame to the frame of the inverse DCT section 118; a frame memory 120 for storing the frames of the adder 119; a 16*16 block forming section 123 for dividing each of the incoming data into 16*16 blocks; a motion estimator 121 for estimating the motion vectors by comparing the pixel values of the present frame of the 16*16 block forming section 123 with the pixel values of the preceding frame of the frame memory; an MC (motion compensator) 122 for forming a new frame by applying the motion vector of the motion estimator 121 to the frame of the frame memory 120; and a multiplexer 116 for multiplexing the image data of an 8*8 variable length coding section 115 and the motion vector of the motion estimator 121.
Meanwhile, the resolution (number of lateral * number of longitudinal pixels) of a frame is various including 720*480, and 1192*1080. In such a frame, when estimating the motion vectors between the present frame and the preceding frame, a single frame is divided into blocks having 16*16 pixels, and thus, a frame is processed in units of blocks.
In this motion estimator 121, the present frame is compared with the preceding frame (f (t-l)), thereby estimating the varied direction of the motion. This will be described in detail referring to FIG. 3. FIG. 3 illustrates an example of the frame divided into the blocks. In the 16*16 blocking forming section 123 of FIG. 2, a frame is divided into blocks having 16*16 pixels (to be called"16*16 block"below) based on the MPEG regulation. This frame which has been divided into the blocks is illustrated in FIG. 3.
FIG. 4 illustrates a part of the blocks of the present frame. In FIG. 4, the present frame (f (t)) includes 9 blocks, with a reference block (B (t) 22) being disposed at the center, and with 8 blocks surrounding the reference block. FIG. 5 illustrates the blocks of the preceding frame, corresponding to the blocks of the present frame of FIG. 4. In FIG. 5, there is a candidate block (B (t-1) 22) corresponding to the reference block (B (t) 22), with 8 blocks surrounding it, thereby forming the preceding frame (f (t-1)) including 9 blocks. In FIG. 5, the one-dot chain line indicates a search window SRW which includes a plurality of blocks same as the reference block (B (t) 22). This search window SRW is decided by considering the movable range between two frames in an image stream showing about 24 frames per second. Generally, the search range is applied to the candidate block (B (t-1) 22), and for example, this search range is decided by the candidate block size divided by 2. The actual examples of the search range includes candidate block 8, candidate block 4 and the like.
Referring to FIG. 5, in the motion estimator 121 of FIG.
2, a matching scan is carried out between the reference block (B (t) 22) of FIG. 4 and the relevant candidate block which has been selected in the scan direction from among a plurality of candidate blocks of the search window SRW of FIG. 5. The scan starts from the center of FIG. 6a, and advances in a spiral direction as shown in FIG. 6b. The matching scan direction between the reference block and the selected candidate blocks is vertical and horizontal direction as shown in FIG. 6c. Thus, the candidate block which is most similarly matched with the reference block is found so as to estimate the motion vector.
In this context, the algorithm for finding the most similarly matched block is called"matching algorithm".
Under this condition, during the matching scan, the comparison objects are the pixels of the reference block of the present frame and the pixels of the candidate blocks of the preceding frame. Here, the matching blocks are found in the following manner. That is, the pixel values of the candidate blocks are subtracted from the pixel values of the reference block. Then the candidate block which shows the least distortion is taken, and then, the position vectors (by using x, y coordinates) are calculated, thereby estimating the motion vectors.
Generally, in carrying out the estimation of the motion vectors by the motion estimator, there are taken into account two factors including the high picture quality and the high processing speed.
In the conventional video encoder, the motion estimation occupies 60-70% in the total encoding time. There are various types in the block matching algorithms which are the high speed motion estimating algorithms. Among them, the one which shows the highest picture quality is the full search method.
In this full search method such as the 3-stage search method, the quality of the estimated image is superior, but there are required 24 bits including 8 bits for each of the R, G and B colors. Therefore, in the full search method, the arithmetic operation amount is too much voluminous, and therefore, if the search range is large, a real time processing becomes impossible.
Embodiments of the present invention seek to address one or more of the above described disadvantages of the conventional technique.
Therefore, in accordance with a first aspect of the present invention there is provided a method for estimating a fast motion based on the image complexity in a video encoder, in which in a matching scan for matching the respective candidate blocks of the search window of the preceding frame to the reference block of the present frame, there is applied an adaptive matching scan for carrying out the matching scan in the decided direction based on the image complexity, and there is applied a PDE (partial distortion estimation) during the matching scan, so that the calculation amount can be significantly reduced in the full search, thereby making it possible to carry out the motion estimation at a high speed.
In accordance with the above aspect, the method for estimating a fast motion based on the image complexity in a video encoder, with an encoder (for compressing an image data) detecting a matching block from among a plurality of candidate blocks of a preceding frame (f (t-1)) through a full search so as to match with a reference block of a present frame (f (t)), according to an embodiment the present invention includes the steps of: (a) calculating a complexity of the reference block of the present frame, and then, deciding a matching scan direction based on the complexity; (b) calculating a SAD (sum of absolute differences) between the reference block of the present frame and the candidate blocks set up in advance to decide an initial SAD (SADmin); (c) calculating the sum of absolute distortions (SADtep) between the reference block of the present frame and the present candidate blocks in the matching scan direction of the step a: (d) comparing as to whether SADtep is smaller than SADmin so as to advance to a next step if it is not smaller, and so as to advance to the next step after replacing SADmin with SADtep if it is smaller ; (e) judging as to whether the present candidate block is a last candidate block, so as to advance to the step c after moving to a next candidate block within the search window if it is not a last one, and so as to move to a next step if it is a last one; (f) deciding a motion vector based on a coordinate (x, y) of the present reference block and a candidate block of SADmin; and (g) judging as to whether the present reference block is a last reference block, to return to the step a after carrying out a preparatory procedure for a motion vector estimation for the next reference block if it is not a last reference block, and to terminate the motion estimation for one frame if it is a last reference block.
The above aspect and other advantages of embodiment5of the present invention will become more apparent by describing in detail, by way of example only, the preferred embodiment of the present invention with reference to the attached drawings, in which: FIG. 1 illustrates the overall constitution of the image data transceiving system; FIG. 2 is a block diagram showing the constitution of the MPEG source encoder of the transmitter of FIG. 1; FIG. 3 illustrates an example of the frame divided into the blocks; FIG. 4 illustrates a part of the blocks of the present frame ; FIG. 5 illustrates the blocks of the preceding frame, corresponding to the blocks of the present frame of FIG. 4; FIG. 6a illustrates the starting position for the search within the search window; FIG. 6b illustrates the search direction within the search window; FIG. 6c illustrates the matching scan direction; FIG. 7 is a flow chart showing the constitution of the fast motion estimation method according to an embodiment of the present invention; FIG. 8 is a flow chart showing the matching scan direction deciding procedure based on the complexity of FIG. 7; FIG. 9 illustrates an example of a sub-block of the reference block, for calculating the complexity of FIG. 7; FIGs. 10a-10e illustrate a first embodiment of the matching scan of FIG. 8; FIG. lla-lle illustrate a second embodiment of the matching scan of FIG. 8; FIGs. 12a-12e illustrate a third embodiment of the matching scan of FIG. 8; FIGs. 13a-13d illustrate a fourth embodiment of the matching scan of FIG. 8; FIG. 14 is a first modified example of the matching scan of FIG. 10 ; FIG. 15 is a second modified example of the matching scan of FIG. 10; FIG. 16 is a flow chart showing the execution procedure for the matching scan of FIG. 7; and FIGs. 17a-17d are graphical illustrations showing the number of the average checking rows per block with respect to the number of frames in embodiments of the present invention and the conventional method.
In the PDE (Partial distortion elimination) algorithm in accordance with the present invention adopting a spiral search method, there is applied an algorithm in which the matching scan direction is decided based on the image complexity, and the matching scan is carried out in the decided direction. In this manner, the impossible candidate blocks are eliminated as speedily as possible, thereby finding out the motion vector at a high speed.
An embodiment of the present invention will be described in detail referring to FIGs. 7 to 17. First, FIG. 7 is a flow chart showing the constitution of the fast motion estimation method according to an embodiment of the present invention. Referring to FIG. 7, first a step a is carried out as shown in a step 71 of FIG. 7. That is, the complexity of the present block is calculated, and then, the matching scan direction is decided based on the calculated complexity. This will be described in further details below.
FIG. 8 is a flow chart showing the matching scan direction deciding procedure based on the complexity of FIG. 7. FIGs. 10a -l0e illustrate a first embodiment of the matching scan of FIG.
8. Referring to FIGs. 8 and 10a to 10c, at a step al, a reference block is divided into sub-blocks of a certain size.
At a step a2, the complexities of the respective sub-blocks are calculated. At a step a3, the complexities of the sub-blocks are summed up in the longitudinal direction and in the lateral direction. At a step a4, the row or column of the largest complexity sum is taken from among the respective complexity sums of the step a3, and is decided as the matching scan direction.
FIG. 9 illustrates an example of a sub-block of the reference block, for calculating the complexity of FIG. 7.
Referring to FIG. 9, in order to reduce the repetitions when calculating the image complexities, the reference block of a 16*16 size is divided into sub-blocks. That is, this reference block of a 16*16 size is divided into 8*8 sub-blocks, each of which has a 1/4 size of the reference block. Then the complexity is calculated for each of the 8*8 sub-blocks, and the method of calculating the complexities will be described later.
In this manner, the complexities can be calculated by dividing the reference block of a 16*16 size into sub-blocks of an 8*8 size. However, in order to carry out a motion estimation quickly and accurately, the 16*16 reference block can also be divided into 4*4 sub-blocks as shown in a third embodiment of FIG. 12, so as to calculate the complexities of the sub-blocks.
Referring to FIGs. 12a-12e, the direction of the adaptive matching scan is decided to be top-to-bottom in the reference 1 of the sub-blocks, to be bottom-to-top in a sub-block 2, to be left-to-right in a sub-block 3, and to be right-to-left in a sub-block 4.
Alternatively, the 16*16 reference block can be divided into 2*2 blocks, and the complexities of them can be calculated.
In an embodiment of the present invention, it could be confirmed through repeated experiments and researches that the matching distortions are increased depending on the image complexities. Therefore, in an embodiment of the present invention, the image complexity is adopted as the criterion for the method of carrying out the matching scan. That is, the adaptive matching scan is adopted among the various matching scans. In an embodiment of the present invention, the complexities can be measured by using the Laplacian, or the gradient or their modified form. The mentioned formulas are the generally known ones, and therefore, descriptions on them will be skipped.
Referring to FIG. 9, in order to reduce the repetitions when calculating the image complexities, the reference block of a 16*16 size is divided into sub-blocks. That is, this reference block of a 16*16 size is divided into 8*8 sub-blocks, each of which has a 1/4 size of the reference block. Then the complexity is calculated for each of the 8*8 sub-blocks. Then by utilizing the calculated complexities of sub-blocks, there are calculated a complexity sum ( (i) + (ii)) for the two blocks of the top, a complexity sum ( (iii) +iiv)) for the two blocks of the bottom, a complexity sum ( (i) + (iii)) for the right two blocks, and a complexity sum ( (ii) + (iv)) for the left two blocks. Among these sums, the two blocks corresponding to the highest complexity are adopted as the starting point of the matching scan.
Among the four scans as shown in FIGs. 10b-10e, if only one of them is adopted to carry out the obtained matching distortion, then there is no significant difference between the embodiments of the present invention and the conventional method. Accordingly, embodiments of the present invention adaptively employ the above four scan methods, and the image complexity of the reference block is analyzed in order to adaptively employ the scan methods.
The matching scan direction which is decided at the step a4 is one selected from among a top-to-bottom direction, a bottomto-top direction, a left-to-right direction, and a right-toleft direction. This is specifically decided as follows.
In the blocks of FIG. 10a, if the complexity sum ( (i) + (ii)) of the two blocks of the top is largest, then a horizontal scan direction at the top-to-bottom direction as shown in FIG. 10b is selected. In the blocks of FIG. 10a, if the complexity sum ( (iii) + (iv)) of the two blocks of the bottom is largest, then a horizontal scan direction at the bottom-to-top direction as shown in FIG. 10c is selected. In the blocks of FIG. 10a, if the complexity sum ( (i) + (iii)) of the two blocks of the left is largest, then a vertical scan direction at the left-to-right direction as shown in FIG. 10d is selected. In the blocks of FIG. 10a, if the complexity sum ( (ii) + (iv)) of the two blocks of the right is largest, then a vertical scan direction at the right-to-left direction as shown in FIG. 10e is selected.
FIG. lla-lle illustrate a second embodiment of the matching scan of FIG. 8. Referring to FIG. lla, at a step al, the reference block is divided into sub-blocks of a certain size.
At a step a2, the complexities of the respective sub-blocks of the step al are calculated. At a step a3, the sub-blocks are arranged in the order of the complexities of the step a2, and a matching scan direction is decided in accordance with the magnitudes of the complexities of the sub-blocks. FIGs. llblle illustrates the case where the magnitudes of the complexities correspond to the sub-block numbers (1), (2), (3) and (4).
At the step al, the 8*8 sub-blocks can be utilized for the sub-blocks. However, for the sake of more accurate and faster matching scans, the 16*16 reference block can be divided into 4*4 sub-blocks as shown in the fourth embodiment of FIG. 13, and thus, the complexity of each of the sub-blocks can be calculated, thereby deciding the matching scan direction. As shown in FIGs. 13a-13e, the complexity of each of the 4*4 subblocks is calculated, and the magnitudes of the complexities correspond to the order of sub-blocks (1), (2),.... (15). Or the reference block can be divided into 2*2 sub-blocks. In this case, the matching scan direction which has been decided at the step a3 is in top-to-bottom.
FIG. 14 is a first modified example of the matching scan of FIG. 10. Referring to FIG. 14, the reference block is divided into sub-blocks of a certain size at a step al. At a step a2, the complexities of the sub-blocks of the step al are calculated. At a step a3, the complexities of the sub-blocks are summed up for each of the lateral and longitudinal rows. At a step a4, the matching scan direction is decided based on the sub-block row which corresponds to the largest complexity sum among the complexity sums of the step a3. At a step aS, if the matching scan direction is decided, the matching scan order is decided based on rows of the decided direction and based on the order of the magnitudes of the complexities.
At the step al, the 8*8 sub-blocks can be utilized for the sub-blocks. However, for the sake of more accurate and faster matching scans, the 16*16 reference block can be divided into 4*4 sub-blocks, or the reference block can be divided into 2*2 sub-blocks. In this case, the matching scan direction which has been decided at the step a4 is in the lateral or longitudinal direction.
FIG. 15 is a second modified example of the matching scan of FIG. 10. Referring to FIG. 15, the reference block is divided into sub-blocks of a certain size at a step al. At a step a2, the complexities of the sub-blocks of the step al are calculated. At a step a3, the complexities of the sub-blocks are arranged in the order of magnitudes of the complexities of the step a2, and the matching scan direction is decided to be in the diagonal direction starting from the sub-block which corresponds to the largest complexity.
At the step al, the 8*8 sub-blocks can be utilized for the sub-blocks. However, for the sake of more accurate and faster matching scans, the 16*16 reference block can be divided into 4*4 sub-blocks, or the reference block can be divided into 2*2 sub-blocks. In this case, the matching scan direction which has been decided at the step a3 is one selected from upper leftlower right, upper right-lower left, lower left-upper right, lower right-upper left.
The method of deciding the matching scan direction based on the image complexities has the following advantage. That is, the amount of the calculations which have to be done until the finding of the matching blocks is reduced as far as possible, so that the calculations can be more quickly done. This will be described in detail below.
After the step a, a step 72 of FIG. 7 is carried out. That is, at a step b, the SAD (sum of absolute differences) between the present reference block and the candidate blocks (which have been decided within the search window) is calculated to adopt it as an initial minimum value SADmin. This SAD calculation is carried out in the following manner. That is, the sum of the absolute values of the differences (distortions) between the pixel values of the present reference block and the pixel values of the present candidate blocks is calculated.
< Formula 1 > SADmin =
FIG. 16 is a flow chart showing the execution procedure for the matching scan of FIG. 7. Referring to FIGs. 7 and 16, at a step c, a full scan is carried out in the decided matching scan direction. That is, at the step c, the absolute distortion sum SADtep between the present reference block and the present candidate blocks is calculated in the matching scan direction which has been decided at the step a.
During the time when the absolute distortion sum SADtep is being calculated, a judgment is made as to whether the SADtep is larger than SADmin. If it is larger, then the calculation of SADtep is terminated, and the system advances to the next step. Under this condition, the absolute distortion sum is calculated for each pixel group or for each line.
In this manner, the calculation of SADtep is skipped over the blocks which are not related to the motion vector, so that the calculation amount can be reduced. This is called"partial distortion elimination", and this can be expressed mathematically by the following formula: < Formula 2 > SADtep =
where n is the size of the matching block. In the above formula, the conventional full scan algorithm uses n instead of k, and the smaller k is, the more the calculation amount can be reduced. During the calculation of the matching distortions, if k having a matching distortion larger than the minimum matching distortion (which has been the minimum so far) is found, then the remaining distortion calculation is skipped, thereby reducing the calculation amount by utilizing the partial distortion elimination method. In this context, the important matter is how early the minimum matching distortion is encountered, and how small k can be obtained. According to embodiments of the present invention, k is very small as shown in FIGs. 17a-17d.
Then as shown at steps of 74 and 75 of FIG. 7, a comparison is made as to whether SADtep of the step c is smaller than SADmin at a step d. If it is not smaller, then the system advances to the next step, while if it is smaller, then the system replaces SADmin with SADtep, and then, advances to the next step.
In this manner, by replacing SADmin with SADtep (which is smaller than SADmin), there is found out the optimum matching block which shows the least distortions between the present reference block and a plurality of candidate blocks.
Then steps 76a and 76b of FIG. 7 are carried out. That is, at a step e, a judgment is made as to whether the present candidate block is the last candidate block within the search window. If it is not the last candidate block, then the system advances to a step c to calculate SADtep for the next candidate block. If it is the last candidate block, then it is judged that the search for one block has been completed, and therefore, the system advances to the next step.
Thus when the search into between the present reference block and the plurality of the candidate blocks within the search window is completed at the step e, a step 77 of FIG. 7 is carried out. That is, at a step f, the motion vector is decided based on the center coordinate of the present reference block and the center coordinate of the candidate block corresponding to SADmin which has been the minimum value at the step d.
Finally, steps 78a and 78b of FIG. 7 are carried out. That is, at a step g, it is judged as to whether the present reference block is the last reference block within the frame (which has been divided into a plurality of blocks). If it is not the last referen motion estimation of the frame is terminated.
At the step g, if the present reference block is not the last reference block, then the system advances to the next reference block as described above. In this context, the search direction is spiral. If the spiral search is adopted, the minimum matching distortion of the total region can be obtained as fast as possible. To have a small k as small as possible is first attempted by embodiments of the present invention in this technical field.
The smaller the value of k is, the more the calculation amount can be reduced.
FIGs. 17a-17d are graphical illustrations showing the number of the average checking rows per block with respect to the number of frames in the embodiments of the present invention and the conventional method. FIG. 17a illustrates the number of the checking rows per block based on the foreman sequence. FIG. 17b illustrates the number of the checking rows per block based on the car phone sequence. FIG. 17c illustrates the number of the checking rows per block based on the trevor sequence. FIG. 17d illustrates the number of the checking rows per block based on the Claire sequence. As shown in FIGs. 17a-17d, according to the motion estimation of embodiments of the present invention, the number of the checking rows within the candidate blocks has been reduced compared with the motion estimation of the conventional method in which simply the spiral search is utilized. Further, the number of the checking rows has been reduced in each of the first to fourth embodiments of the matching scan.
Above descriptions are only some specific examples of embodiment5of the present invention, but it should be apparent to those ordinarily skilled in the art that various changes and modifications can be added without departing from the spirit and scope of the present invention which will be defined in the appended claims.
According to embodiments of the present invention as described above, in a matching scan for matching the respective candidate blocks of the search window of the preceding frame to the reference block of the present frame, there are adopted a spiral search direction and an adaptive matching scan for carrying out the matching scan in the decided direction based on the image complexity, and there is applied a PDE (partial distortion estimation) during the matching scan, so that the calculation amount can be significantly reduced in the full search, thereby making it possible to carry out the motion estimation at a high speed. Thus the full scan for the motion estimation can be embodied in the form of an algorithm.
The scope of the present disclosure includes any novel feature or combination of features disclosed therein either explicitly or implicitly or any generalisation thereof irrespective of whether or not it relates to the claimed invention or mitigates any or all of the problems addressed by the present invention. The applicant hereby gives notice that new claims may be formulated to such features during the prosecution of this application or of any such further application derived therefrom. In particular, with reference to the appended claims, features from dependent claims may be combined with those of the independent claims and features from respective independent claims may be combined in any appropriate manner and not merely in the specific combinations enumerated in the claims. For example, embodiment of the invention may comprise programmable logic devices such as Digital Signal Processors for implementing a video encoder operable in accordance with the invention.
Such programmable logic devices may be configurable by a computer program comprising computer implementable instructions for configuring a computer, including a computing device such as a programmable logic device or the like, to operate in accordance with the foregoing described methods.

Claims (31)

1. A method for estimating a fast motion based on an image complexity in a video encoder, with an encoder (for compressing an image data) detecting a matching block from among a plurality of candidate blocks of a preceding frame (f (t-1)) within a search window SRW through a full search so as to match it with a reference block of a present frame (f (t)), comprising the steps of: calculating a complexity of said reference block of said present frame ; deciding a matching scan direction based on the complexity thus calculated; and carrying out a matching scan in the matching scan direction thus decided, so as to estimate motions.
2. A method for estimating a fast motion based on an image complexity in a video encoder, with an encoder (for compressing an image data) detecting a matching block from among a plurality of candidate blocks of a preceding frame (f (t-1)) within a search window SRW through a full search so as to match it with a reference block of a present frame (f (t)), comprising the steps of: (a) calculating a complexity of said reference block of said present frame, and then, deciding a matching scan direction based on the complexity; (b) calculating a SAD (sum of absolute differences) between said reference block of said present frame and said candidate blocks set up in advance to decide an initial SAD (SADmin); (c) calculating a sum of absolute distortions (SADtep) between said reference block of said present frame and said present candidate blocks in the matching scan direction of the step a: (d) judging as to whether SADtep is smaller than SADmin so as to advance to a next step if it is not smaller, and so as to advance to a next step after replacing SADmin with SADtep if it is smaller; (e) judging as to whether a present candidate block is a last candidate block, so as to advance to the step c after moving to a next candidate block within said search window if it is not a last one, and so as to move to a next step if it is a last one; (f) deciding a motion vector based on a coordinate (x, y) of said present reference block and a candidate block of SADmin; and (g) judging as to whether said present reference block is a last reference block, to return to the step a after carrying out a preparatory procedure for a motion vector estimation for a next reference block if it is not a last reference block, and to terminate the motion estimation for one frame if it is a last reference block.
3. The method as claimed in claim 2, wherein the step a comprises the sub-steps of: (al) dividing said reference block into a plurality of subblocks of a certain size; (a2) calculating complexities of said respective sub-blocks of the sub-step al; (a3) summing the complexities of said sub-blocks in lateral and longitudinal rows; and (a4) selecting a block row or a block column having a largest complexity sum to decide the matching scan direction.
4. The method as claimed in claim 3, wherein the matching scan direction is one selected from top-to-bottom, bottom-to-top, left-to-right, and right-to-left.
5. The method as claimed in claim 3 or 4, wherein at the step al, said respective sub-blocks are 8*8 sub-blocks.
6. The method as claimed in claim 3 or 4, wherein at the step al, said respective sub-blocks are 4*4 sub-blocks.
7. The method as claimed in claim 3 or 4, wherein at the step al, said respective sub-blocks are 2*2 sub-blocks.
8. The method as claimed in claim 2, wherein the step a comprises the sub-steps of: (al) dividing said reference block into a plurality of subblocks of a certain size; (a2) calculating complexities of said respective sub-blocks of the sub-step al; and (a3) arranging said sub-blocks in an order of magnitudes of complexities as obtained at the sub-step a2, and deciding the matching scan direction in an order of magnitudes of the complexities of said sub-blocks.
9. The method as claimed in claim 8, wherein at the step al, said respective sub-blocks are 8*8 sub-blocks.
10. The method as claimed in claim 8, wherein at the step al, said respective sub-blocks are 4*4 sub-blocks.
11. The method as claimed in claim 8, wherein at the step al, said respective sub-blocks are 2*2 sub-blocks.
12. The method as claimed in any of claims 8 to 11, wherein the matching scan direction decided at the sub-step a3 is in top-to-bottom.
13. The method as claimed in claim 2, wherein the step a comprises the sub-steps of: (al) dividing said reference block into a plurality of subblocks of a certain size; (a2) calculating complexities of said respective sub-blocks of the sub-step al; (a3) summing the complexities of said sub-blocks in lateral and longitudinal rows; (a4) selecting a block row or a block column having a largest complexity sum to decide the matching scan direction; and (a5) deciding a matching scan sequence based on magnitudes of the complexities of said rows and columns.
14. The method as claimed in claim 13, wherein at the step al, said respective sub-blocks are 8*8 sub-blocks.
15. The method as claimed in claim 13, wherein at the step al, said respective sub-blocks are 4*4 sub-blocks.
16. The method as claimed in claim 13, wherein at the step al, said respective sub-blocks are 2*2 sub-blocks.
17. The method as claimed in any of claims 13 to 16, wherein at the step a4, the matching scan direction is decided to be one of a lateral direction and a longitudinal direction.
18. The method as claimed in claim 2, wherein the step a comprises the sub-steps of: (al) dividing said reference block into a plurality of subblocks of a certain size; (a2) calculating complexities of said respective sub-blocks of the sub-step al; and (a3) arranging said sub-blocks in an order of magnitudes of complexities as obtained at the sub-step a2, and deciding the matching scan direction to be in a diagonal direction in an order of magnitudes of complexities of said sub-blocks.
19. The method as claimed in claim 18, wherein at the step al, said respective sub-blocks are 8*8 sub-blocks.
20. The method as claimed in claim 18, wherein at the step al, said respective sub-blocks are 4*4 sub-blocks.
21. The method as claimed in claim 18, wherein at the step al, said respective sub-blocks are 2*2 sub-blocks.
22. The method as claimed in any of claims 18 to 21, wherein at the step a3, the matching scan direction is decided to be one selected from upper left-lower right, upper right-lower left, lower left-upper right, lower right-upper left.
23. The method as claimed in claim 2, wherein at the step c, during a calculation of absolute distortion sums between said present reference block and present candidate blocks, it is judged as to whether SADtep (calculated) is larger than SADmin, and if it is larger, then the calculation of SADtep is terminated at a corresponding search point, to advance to a next step.
24. The method as claimed in claim 2, wherein at the step c, during a calculation of absolute distortion sums between said present reference block and present candidate blocks for each of pixel groups, it is judged as to whether SADtep (calculated) is larger than SADmin, and if it is larger, then the calculation of SADtep is terminated at a corresponding search point, to advance to a next step.
25. The method as claimed in claim 2, wherein at the step c, during a calculation of absolute distortion sums between said present reference block and present candidate blocks for each line, it is judged as to whether SADtep (calculated) is larger than SADmin, and if it is larger, then the calculation of SADtep is terminated at a corresponding search point, to advance to a next step.
26. The method as claimed in claim 2, wherein at the step e, said candidate blocks of said search window are scanned in a spiral direction starting from a predetermined position.
27. A method for estimating fast motion based on image complexity substantially as hereinbefore described in respective embodiments and with reference to respective Figures of the drawings.
28. A video encoder configured to operate in accordance with any preceding claim.
29. A video encoder according to claim 28, comprising a programmable logic device configured to operate in accordance with any of claims 1 to 27.
30. A computer program comprising computer implementable instructions for configuring a device to operate in accordance with any of claims 1 to 27.
31. A video encoder substantially as hereinbefore described in respective embodiments and with reference to respective Figures of the drawings.
GB0003276A 1999-02-22 2000-02-11 Estimation of fast motion in an image Withdrawn GB2349535A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR19990005770 1999-02-22
KR1019990055072A KR100305582B1 (en) 1999-02-22 1999-12-06 Fast motion estimating method using adaptive scan based on image complexity

Publications (2)

Publication Number Publication Date
GB0003276D0 GB0003276D0 (en) 2000-04-05
GB2349535A true GB2349535A (en) 2000-11-01

Family

ID=26634729

Family Applications (1)

Application Number Title Priority Date Filing Date
GB0003276A Withdrawn GB2349535A (en) 1999-02-22 2000-02-11 Estimation of fast motion in an image

Country Status (2)

Country Link
KR (1) KR100305582B1 (en)
GB (1) GB2349535A (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100468844B1 (en) * 2002-01-07 2005-01-29 삼성전자주식회사 Optimal scanning method for transform coefficients in image and video coding/decoding
KR100961760B1 (en) * 2002-08-13 2010-06-07 퀄컴 인코포레이티드 Motion Estimation Method and Apparatus Which Refer to Discret Cosine Transform Coefficients
KR100910209B1 (en) * 2006-11-28 2009-07-30 한양대학교 산학협력단 Apparatus and Method for the fast full search motion estimation using the partitioned search window
KR100870554B1 (en) * 2007-01-08 2008-11-27 한양대학교 산학협력단 Motion compensated temporal filtering method for efficient wavelet-based scalable video coding and record-medium for executing method thereof
KR101037743B1 (en) * 2009-10-29 2011-05-27 숭실대학교산학협력단 Method for motion estimation using an efficient early termination scheme, and apparatus for video encoding using the same

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2313514A (en) * 1993-08-03 1997-11-26 Sony Uk Ltd Motion compensated video signal processing
US6014181A (en) * 1997-10-13 2000-01-11 Sharp Laboratories Of America, Inc. Adaptive step-size motion estimation based on statistical sum of absolute differences

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2313514A (en) * 1993-08-03 1997-11-26 Sony Uk Ltd Motion compensated video signal processing
US6014181A (en) * 1997-10-13 2000-01-11 Sharp Laboratories Of America, Inc. Adaptive step-size motion estimation based on statistical sum of absolute differences

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Intl. Conf. on consumer electronics, June 1999, JN KIM et al"Adaptive matching scan" pp 366-367 *
Picture coding symposium 1999, JN KIM et al, "Computational Reduction Using UESA", pp 107-111 *

Also Published As

Publication number Publication date
KR20000057040A (en) 2000-09-15
GB0003276D0 (en) 2000-04-05
KR100305582B1 (en) 2001-11-05

Similar Documents

Publication Publication Date Title
CA2452632C (en) Method for sub-pixel value interpolation
US7302015B2 (en) Motion estimation method for moving picture compression coding
US6418166B1 (en) Motion estimation and block matching pattern
EP1797722B1 (en) Adaptive overlapped block matching for accurate motion compensation
KR100772576B1 (en) Method for encoding and decoding video information, a motion compensated video encoder and a corresponding decoder
EP1466477B1 (en) Coding dynamic filters
US9948944B2 (en) Image coding apparatus and image coding method
AU2002324085A1 (en) Method for sub-pixel value interpolation
EP1413144A2 (en) Methods and apparatus for sub-pixel motion estimation
KR100929608B1 (en) Video motion estimation method and device using fast global search block matching algorithm
US20050226329A1 (en) Motion estimation method using multilevel succesive elimination algorithm
KR100229803B1 (en) Method and apparatus for detecting motion vectors
GB2349535A (en) Estimation of fast motion in an image
MXPA04009309A (en) Methods and apparatus for efficient global motion compensation encoding and associated decoding.
EP3754983B1 (en) Early intra coding decision
US7386050B2 (en) Fast half-pel searching method on the basis of SAD values according to integer-pel search and random variable corresponding to each macro block
KR100457231B1 (en) Moving Compensation Prediction Apparatus and Method for Shape Encoding of Image Signal
KR100212560B1 (en) Apparatus for determinating coding mode of contour in hybrid contour coding system
GB2379820A (en) Interpolating values for sub-pixels
AU2007237319B2 (en) Method for sub-pixel value interpolation
KR100203658B1 (en) Apparatus for estimating motion of contour in object based encoding
KR101037070B1 (en) Fast Motion Estimation Method Using Full Search
Liang et al. Adaptively asymmetric pattern search for fast block-matching motion estimation
EP1387587A1 (en) Image encoder and decoder with error concealment of motion vector losses
Mathur et al. A low bit rate video coder using content-based automatic segmentation and shape-adaptive DCT

Legal Events

Date Code Title Description
WAP Application withdrawn, taken to be withdrawn or refused ** after publication under section 16(1)