US20050025342A1 - Pattern analysis-based motion vector compensation apparatus and method - Google Patents
Pattern analysis-based motion vector compensation apparatus and method Download PDFInfo
- Publication number
- US20050025342A1 US20050025342A1 US10/887,897 US88789704A US2005025342A1 US 20050025342 A1 US20050025342 A1 US 20050025342A1 US 88789704 A US88789704 A US 88789704A US 2005025342 A1 US2005025342 A1 US 2005025342A1
- Authority
- US
- United States
- Prior art keywords
- motion vector
- values
- pattern
- projection
- motion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
- H04N7/0135—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes
- H04N7/014—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes involving the use of motion vectors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/215—Motion-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/223—Analysis of motion using block-matching
Definitions
- the present invention relates to a pattern analysis-based motion vector compensation apparatus and method, and more particularly to an apparatus and method for compensating for estimated motion vectors by deciding whether blocks for motion estimation are a pattern image.
- the motion estimation between image frames is necessarily carried out in image processing such as the frame rate up-conversion (FRC) and the Interlaced-to-progressive conversion (IPC).
- FRC frame rate up-conversion
- IPC Interlaced-to-progressive conversion
- BMA block matching algorithm
- the BMA compares two consecutive frames/fields matching one block with another, and estimates one motion vector per block.
- the motion vector is estimated by use of the known motion estimation error values, for example, the sum of Absolute Difference (SAD) values. Further, the motion compensation process compensates for motions by use of the estimated motion vectors.
- SAD Sum of Absolute Difference
- the conventional motion estimation based on the block-based estimation brings out inaccurate motion vectors from time to time.
- precise motion vectors are hardly estimated by use of the BMA only with respect to images of periodic pattern, that is, identical images periodically repeated as shown in FIG. 1A .
- the periodic pattern is a pattern repeating a certain image in certain intervals, such as stairs, zebra, lattice, and so on.
- the block artifact occurs in the frame/field image compression as shown in FIG. 1B .
- the block artifact refers to a boundary discontinuity among neighboring blocks, and causes visible and objectionable images together with lower image quality.
- Such block artifact is caused when the block-based motion compensation is applied by use of estimated motion vectors without considering compensation with neighboring blocks.
- An aspect of the present invention is to provide a pattern analysis-based motion vector compensation apparatus and method capable of compensating for motion vectors in consideration of correlation among blocks by deciding whether a certain pattern image exists in blocks, to thereby reduce block artifact resulting from motion vectors inaccurately estimated on certain blocks.
- a pattern analysis-based motion vector compensation apparatus comprising a motion estimation part for comparing a current block of a current frame/field to a certain search area established in a previous frame/field, calculating plural motion estimation error values, and estimating a temporary motion vector of the current block based on the plural motion estimation error values; a pattern decision part for analyzing the calculated motion estimation error values, and deciding whether the current block is a pattern image; a motion compensation part for producing a compensation motion vector of the current block by use of the plural motion estimation error values if the current block is decided as the pattern image; and a motion vector selection part for selecting as a final motion vector either the temporary motion vector estimated by the motion estimation part or the compensation motion vector compensated for by the motion compensation part based on a result of the pattern decision part.
- the motion estimation part includes a motion estimation error calculation unit for applying a block-matching approach to the current block and search area of the current frame/field divided into blocks in a certain size, and calculating the plural motion estimation error values; and a motion vector estimation unit for estimating the temporary motion vector of the current block based on a point of a minimum motion estimation error value of the plural motion estimation error values produced.
- the pattern decision part includes an integral projection unit for applying an integral projection to the plural motion estimation error values, and producing plural projection values; a projection value verification unit for verifying the number of at least one minimum local projection values between two points corresponding to predetermined first and second projection values but smaller than the first and second projection values of the plural projection values; a period calculation unit for calculating the number of repeated periods including at least one minimum local projection value sequentially appearing in the plural projection values, and dividing a length of the search area by the number of repeated periods to calculate a period; and a pattern decision unit for analyzing the verified number of minimum local projection values and the calculated number of repeated periods, and deciding whether the current block is the pattern image.
- the integral projection unit includes a vertical projector for applying the integral projection to the plural motion estimation error values in a vertical direction, and producing plural vertical projection values; and a horizontal projector for applying the integral projection to the plural motion estimation error values in a horizontal direction, and producing plural horizontal projection values.
- the projection value verification unit includes a vertical projection value verification portion for verifying the number of minimum vertical local projection values between two points between predetermined first and second vertical projection values but smaller than the first and second vertical projection values of the plural vertical projection values produced; and a horizontal projection value verification portion for verifying the number of minimum local horizontal projection values between two points corresponding to predetermined first and second horizontal projection values but smaller than the first and second horizontal projection values of the plural horizontal projection values produced.
- the period calculation unit includes a vertical period calculator for calculating the number of repeated vertical periods including at least one or more minimum local vertical projection values sequentially appearing in the plural vertical projection values; and a horizontal period calculator for calculating the number of repeated horizontal periods including at least one or more minimum local horizontal projection values sequentially appearing in the plural horizontal projection values.
- the vertical period calculator divides a horizontal length of the search area by the number of repeated vertical periods and calculates a vertical period of the pattern image
- the horizontal period calculator divides a vertical length of the search area by the number of repeated horizontal periods and calculates a horizontal period of the pattern image.
- the pattern decision part decides that the current block is a pattern image, if the number of the minimum local projection values is at least two and the number of repeated periods is larger than a predetermined first threshold value.
- the pattern analysis-based motion vector compensation apparatus further comprises a pattern classification part for classifying the pattern image into a static pattern image having zero motion of the current block or into a dynamic pattern image occurring due to motion of the current block, depending on a point of a minimum local error value that is located at a nearest distance to the center of the search area if the current block is the pattern image.
- the pattern classification part compares the minimum local error value located at the nearest distance to the center of the search area to the period calculated by the period calculation part, and classifies the pattern image, the minimum local error value being one of at least one minimum local error values between two points corresponding to predetermined first and second error values but smaller than the first and second error values of the plural motion estimation error values.
- the pattern classification part decides that the pattern image is the static pattern image, if the pattern image satisfies a condition as below:
- ⁇ p
- the motion compensation part includes a static motion compensation unit for estimating and producing the compensation motion vector based on a point corresponding to the first minimum local error value, if the pattern image is classified into the static pattern image; an average vector calculation unit for calculating an average vector of motion vectors of blocks neighboring to the current block, if the pattern image is classified into the dynamic pattern image; and a dynamic motion compensation unit for estimating and producing the compensation motion vector based on a point of a second minimum local error value of at least one or more minimum local error values that is located at a nearest distance to the calculated average vector.
- the motion vector selection part selects as the final motion vector the compensation motion vector estimated by the motion compensation part.
- the motion vector selection part selects a compensation motion vector estimated by the static motion compensation unit, and, if the pattern classification part classifies the pattern image into the dynamic pattern image, the motion vector selection part selects a compensation motion vector estimated by the dynamic motion compensation unit.
- first and second motion estimation error values are calculated by use of Sum of Absolute Difference (SAD), Mean Absolute Difference (MAD), or Mean Square Error (MSE).
- SAD Sum of Absolute Difference
- MAD Mean Absolute Difference
- MSE Mean Square Error
- a pattern analysis-based motion vector compensation method comprising a motion estimation step for comparing a current block of a current frame/field to a certain search area established in a previous frame/field, calculating plural motion estimation error values, and estimating a temporary motion vector of the current block based on the plural motion estimation error values; a pattern decision step for analyzing the calculated motion estimation error values, and deciding whether the current block is a pattern image; a motion compensation step for producing a compensation motion vector of the current block by use of the plural motion estimation error values if the current block is decided as the pattern image; and a motion vector selection step for selecting as a final motion vector either the temporary motion vector estimated by the motion estimation step or the compensation motion vector compensated for by the motion compensation step based on a result of the pattern decision step.
- the motion estimation step includes a motion estimation error calculation step for applying a block-matching approach to the current block and search area of the current frame/field divided into blocks in a certain size, and calculating the plural motion estimation error values; and a motion vector estimation step for estimating the temporary motion vector of the current block based on a point of a minimum motion estimation error value of the plural motion estimation error values produced.
- the pattern decision step includes an integral projection step for applying an integral projection to the plural motion estimation error values, and producing plural projection values; a projection value verification step for verifying the number of at least one minimum local projection values between two points corresponding to predetermined first and second projection values but smaller than the first and second projection values of the plural projection values; a period calculation step for calculating the number of repeated periods including at least one minimum local projection value sequentially appearing in the plural projection values, and dividing a length of the search area by the number of repeated periods to calculate a period; and a pattern decision step for analyzing the verified number of minimum local projection values and the calculated number of repeated periods, and deciding whether the current block is the pattern image.
- the integral projection step includes a vertical projection step for applying the integral projection to the plural motion estimation error values in a vertical direction, and producing plural vertical projection values; and a horizontal projection step for applying the integral projection to the plural motion estimation error values in a horizontal direction, and producing plural horizontal projection values.
- the projection value verification step includes a vertical projection value verification step for verifying the number of minimum vertical local projection values between two points corresponding to predetermined first and second vertical projection values but smaller than the first and second vertical projection values of the plural vertical projection values produced; and a horizontal projection value verification step for verifying the number of minimum local horizontal projection values between two points corresponding to predetermined first and second horizontal projection values but smaller than the first and second horizontal projection values of the plural horizontal projection values produced.
- the period calculation method includes a vertical period calculation step for calculating the number of repeated vertical periods including at least one or more minimum local vertical projection values sequentially appearing in the plural vertical projection values; and a horizontal period calculation step for calculating the number of repeated horizontal periods including at least one or more minimum local horizontal projection values sequentially appearing in the plural horizontal projection values.
- the vertical period calculation step divides a horizontal length of the search area by the number of repeated vertical periods and calculates a vertical period of the pattern image
- the horizontal period calculation step divides a vertical length of the search area by the number of repeated horizontal periods and calculates a horizontal period of the pattern image.
- the pattern decision step decides that the current block is a pattern image, if the number of the minimum local projection values is at least two and the number of repeated periods is larger than a predetermined first threshold value.
- the pattern analysis-based motion vector compensation method further comprises a pattern classification step for classifying the pattern image into a static pattern image having zero motion of the current block or into a dynamic pattern image occurring due to motion of the current block, depending on a point of a minimum local error value that is located at a nearest distance to the center of the search area if the current block is the pattern image.
- the pattern classification step compares a first minimum local error value located at the nearest distance to the center of the search area to the period calculated by the period calculation step, and classifies the pattern image, the first minimum local error value being one of at least one minimum local error values between two points corresponding to predetermined first and second error values but smaller than the first and second error values of the plural motion estimation error values.
- the pattern classification step decides that the pattern image is the static pattern image, if the pattern image satisfies a condition as below:
- ⁇ p
- the motion compensation step includes a static motion compensation step for estimating and producing the compensation motion vector based on a point corresponding to the first minimum local error value, if the pattern image is classified into the static pattern image; an average vector calculation step for calculating an average vector of motion vectors of blocks neighboring to the current block, if the pattern image is classified into the dynamic pattern image; and a dynamic motion compensation step for estimating and producing the compensation motion vector based on a point of a second minimum local error value of at least one minimum local error values that is located at a nearest distance to the calculated average vector.
- the motion vector selection step selects as the final motion vector the compensation motion vector estimated by the motion compensation step.
- the motion vector selection step selects a compensation motion vector estimated by the static motion compensation step, and, if the pattern classification step classifies the pattern image into the dynamic pattern image, the motion vector selection step selects a compensation motion vector estimated by the dynamic motion compensation step.
- FIG. 1A is a view for showing an image of periodic pattern in general
- FIG. 1B is a view for showing a simulation of an image of periodic pattern with block artifact by use of a conventional motion estimation method
- FIG. 2 is a block diagram for schematically showing a pattern-based motion vector compensation apparatus according to a preferred embodiment of the present invention
- FIG. 3 is a block diagram for showing in detail a pattern decision part shown in FIG. 2 ;
- FIG. 4 is a graph for showing in three dimensions a part of a SAD map calculated by a motion estimation error calculation unit shown in FIG. 2 ;
- FIG. 5 is a graph for illustrating plural vertical projection values resulting from the certain SAD map shown in FIG. 4 ;
- FIG. 6A and FIG. 6B are views for showing a two-dimensional SAD map resulting from an image with a static pattern and a two-dimensional SAD map resulting from an image with a dynamic pattern, respectively;
- FIG. 7 is a view for simulating an image with reduced block artifact by the pattern decision part of FIG. 2 ;
- FIG. 8 is a flow chart for schematically explaining a pattern-based motion vector compensation process of FIG. 2 .
- FIG. 2 is a block diagram for schematically showing a pattern analysis-based motion vector compensation apparatus according to an exemplary embodiment of the present invention
- FIG. 3 is a block diagram for showing in detail a pattern decision part provided in FIG. 2 .
- the pattern analysis-based motion vector compensation apparatus 200 has a motion estimation part 210 , a pattern decision part, a pattern classification part 230 , a motion compensation part 240 , and a motion vector selection part 250 .
- the motion estimation part 210 includes a motion estimation error calculation unit 212 and a motion vector estimation unit 214 .
- the motion estimation error calculation unit 212 divides a current input frame/field f n (hereinafter, referred to as ‘current frame’) into blocks each having a certain size, and estimates a motion vector v of each block. In more detail, the motion estimation error calculation unit 212 compares the current block for an estimated motion vector, which is one of the divided blocks, to search an area established in a previous frame/field f n ⁇ 1 (hereinafter, referred to as ‘previous frame’), so as to calculate plural motion estimation error values.
- the plural motion estimation error values can be estimated by bidirectional BMA, unidirectional BMA, or all the known approaches that can estimate motion vectors. Further, the motion estimation error values can be calculated by various approaches such as Sum of Absolute Difference(SAD), Mean Absolute Difference(MAD), Mean Square Error(MSE), and so on, but the present invention applies the SAD values, so the values are referred to as “SAD values”.
- the motion vector estimation unit 214 estimates a temporary motion vector (temporary MV) of a current block from the minimum error value of the plural motion estimation error values calculated.
- the motion estimation part 210 estimates the SAD values and temporary motion vectors of the divided blocks.
- FIG. 4 is a graph for showing a part of a calculated SAD map in three dimensions.
- x denotes a horizontal direction axis of a search area
- y a vertical direction axis of the search area.
- the plural SAD values forming the SAD map are obtained by using the BMA as stated above.
- the SAD map of a current block of the blocks to which motion compensation is applied is sent to the motion vector estimation unit 214 , the pattern decision part 220 , the pattern classification part 230 , and the motion compensation part 240 that will be later described. Further, the temporary motion vector of the current block is sent to the motion vector selection part 250 to be later described.
- the pattern decision part 220 analyzes the calculated SAD map, that is, the plural SAD values forming a search area, and decides whether there exists a pattern image in the current block.
- the pattern image is an image periodically repeated in certain intervals, which is referred to as a ‘periodic pattern image’.
- the pattern decision part 220 includes an integral projection unit 222 , a projection value verification unit 224 , a period calculation unit 226 , and a pattern decision unit 228 . Further, the pattern decision part 220 decides a periodic pattern image by considering the vertical and horizontal direction characteristics of the SAD map respectively.
- the integral projection unit 222 applies the integral projection to the SAD map of a current block to produce plural integral vectors (hereinafter, referred to as ‘projection values’). Accordingly, degradation of image caused by noise can be minimized.
- the integral projection unit 222 applies the vertical and horizontal projections, so the integral projection unit 222 has a vertical projector 222 a and a horizontal projector 222 b.
- the vertical projector 222 a applies the vertical integral projection to the SAD map of a current block in the vertical direction to produce plural vertical projection values. That is, the vertical projector 222 a applies the vertical integral projection in order to decide a periodic pattern in the vertical direction, by which the SAD map of a K ⁇ K matrix is converted into a vertical projection value of a K ⁇ 1 matrix, for example.
- FIG. 5 is a graph for showing plural vertical projection values produced based on the SAD map as shown in FIG. 4 .
- x denotes a vertical direction axis of a search area
- S v (x) denotes a vertical projection value at the xth point
- E(x,y) denotes a SAD value corresponding to a motion vector(x,y)
- K denotes the total number of SAD values located on the y axis of the search area.
- the horizontal projector 222 b applies the integral projection to the SAD map of a current block in the horizontal direction to produce plural horizontal projection values. That is, the horizontal projector 222 b applies the horizontal integral projection to decide a periodic pattern in the vertical direction, by which the SAD map of a (K ⁇ K) matrix is converted into a horizontal projection value of a (K ⁇ 1) matrix.
- Equation 2 y denotes a vertical direction axis of a search area
- S h (y) denotes a horizontal projection value at the yth point
- E(x,y) denotes a SAD value corresponding to a motion vector(x,y)
- K denotes the total number of SAD values located on the x axis of the search area.
- the plural vertical and horizontal projection values produced are sent to the projection value verification unit 224 .
- the projection value verification unit 224 verifies the number of minimum local projection values n of the plural projection values produced by the integral projection unit 222 .
- the minimum local projection values are located between the first and second projection values of the plural projection values, but smaller than the first and second projection values.
- the verified minimum local projection values are used when the pattern decision part 228 decides whether there exists a pattern image, wherein the pattern decision part 228 will be described later.
- the projection value verification unit 224 has a vertical projection value verification portion 224 a and a horizontal projection value verification portion 224 b.
- the vertical projection value verification portion 224 a verifies the number of minimum local vertical projection values n v of the plural vertical projection values produced.
- the minimum local vertical projection values are located between the first and second vertical projection values of the plural vertical projection values, but smaller than the first and second vertical projection values. Referring to FIG. 5 , when a certain vertical projection value S v (x) is smaller than two neighboring vertical projection values S v (x ⁇ 1) and S v (x+1), the S v (x) corresponds to the minimum local vertical projection value.
- the horizontal projection value verification portion 224 b verifies the number of minimum local horizontal projection values nh of the plural horizontal projection values produced.
- the minimum local horizontal projection values are located between the first and second horizontal projection values of the plural horizontal projection values, but smaller than the first and second horizontal projection values.
- the vertical and horizontal projection verification portions 224 a and 224 b send the position information of the respective verified minimum local values to the pattern classification part 230 , a static motion compensation unit 242 , and a dynamic motion compensation unit 246 that will be later described.
- the period calculation unit 226 calculates the number of repeated periods ⁇ including at least one of the minimum local projection values that sequentially appear in the plural projection values produced from the integral projection unit 222 . Further, the period calculation unit 226 divides the length or size of a search area by the number of repeated periods, and calculates a period T.
- the period calculation unit 226 has a vertical period calculator 226 a and a horizontal period calculator 226 b.
- the vertical period calculator 226 a calculates the number of repeated vertical periods, using Equation 3.
- the vertical periods refer to ones including at least one of the minimum local vertical projection values that sequentially appear in the plural vertical projection values produced from the vertical projector 222 a .
- the horizontal period calculator 226 b calculates the number of repeated horizontal periods based on Equation 5.
- the horizontal periods refer to ones including at least one of the minimum local horizontal projection values that sequentially appear in the plural horizontal projection values produced from the horizontal projector 222 b .
- the pattern decision unit 228 analyzes the number of the minimum local projection values n verified by the projection value verification unit 224 and the number of repeated periods ⁇ calculated from the period calculation unit 226 , and decides whether there exists a pattern image in a current block. In more detail, the pattern decision unit 228 decides the current block as a periodic pattern image if there exist at least two minimum local projection values and the number of repeated periods is larger than a predetermined first threshold value.
- the pattern decision unit 228 decides that the current block is a periodic pattern image in the vertical direction if there exist at least two minimum local vertical projection values verified by the vertical projection value verification portion 224 a and the vertical period calculated by the vertical period calculator 226 a is larger than the first threshold value of 1.75.
- the established first threshold value can be identical to or different from the value of 1.75.
- the pattern decision unit 228 sends the decided result to the pattern classification part 230 , motion compensation part 240 , and motion vector selection part 250 that will be later described.
- the pattern classification part 230 classifies the pattern image into any of a static pattern image and a dynamic pattern image based on where the minimum local error value exists that is nearest to the center of a search area (that is, a zero vector), which means there is no motion.
- the static pattern image refers to a periodic pattern image having no substantial motion (that is, a zero vector) of the current block
- the dynamic pattern image refers to a periodic pattern image having substantial motion of the current block.
- the pattern classification part 230 compares the minimum local error value, which is a value of at least one minimum local error value located at a nearest distance to the center of the search area, with a period of the periodic pattern image calculated by the period calculation unit 226 , and classifies a pattern image into one in the vertical direction or one in the horizontal direction.
- the minimum local error value refers to a value of the SAD map that is between predetermined first and second error values, but smaller than the first and second error values.
- FIG. 6A is a graph for explaining a SAD map of a static pattern image
- FIG. 6B is a graph for explaining a SAD map of a dynamic pattern image, which are two-dimensional graphs of the center region of the search area of the SAD map of FIG. 4 .
- points P and R each denote the minimum local error values
- a point C denotes the center of a search area indicating a location at which there exists no motion of a current block
- L denotes a range moving by a in the left and by T v in the right from the center of the search area C, the range being established in advance for pattern image classification.
- the pattern classification part 230 classifies into a static pattern image a periodic pattern image in the vertical direction that the pattern decision part 220 has decided, if the point R, that is, the minimum local error value nearest to the center C of the search area is included in the range L.
- the minimum global error value refers to the minimum motion estimation error value of the plural motion estimation error values forming the SAD map.
- the pattern classification part 230 classifies into a dynamic pattern image the periodic pattern image in the horizontal direction that the pattern decision part 220 has decided, if the point R, that is, the minimum local error value nearest to the center of the search area is not included in the established range L.
- the pattern classification part 230 classifies periodic pattern images in the vertical direction based on Equation 8.
- periodic pattern image in vertical direction static pattern image, if
- y′ denotes a point of the minimum local error value that is nearest at distance to the center of a search area
- T h denotes a horizontal period
- ⁇ denotes a predetermined constant for which the present invention uses 1 ⁇ 8, but the constant is not limited to that value of ⁇ fraction ( 1 / 8 ) ⁇ .
- the motion compensation part 240 uses the SAD map to produce a compensation motion vector of a current block, if the current block is decided to be a pattern image.
- the motion compensation part 240 produces a motion vector adaptively compensated depending on pattern images in the vertical and horizontal directions decided by the pattern decision part 220 .
- the point of the minimum local error value of a periodic pattern moves with the motion of the periodic pattern, so the motion compensation part 240 carries out compensation, considering the motion of the periodic pattern. That is, the motion compensation part 240 adaptively carries out motion compensation according to a classification result done by the pattern classification part 230 .
- the motion compensation part 240 has a static motion compensation unit 242 , an average vector calculation unit 244 , and a dynamic motion compensation unit 246 .
- the static motion compensation unit 242 produces the compensation motion vector of a current block decided as a static pattern image by the pattern classification part 230 .
- the static motion compensation unit 242 produces compensation motion vectors from the point of the minimum local error value, for example, the point R of FIG. 6A , that is located at a nearest distance to the center of a search area.
- the compensation process in the static motion compensation unit 242 decides consistent motion vectors from plural blocks forming one periodic pattern.
- estimated vectors are used in compensating for images, so the mismatching of patterns between blocks, that is, the block artifact phenomenon occurring in a periodic pattern image can be greatly reduced.
- the average vector calculation unit 244 calculates an average vector of motion vectors of the blocks neighboring to the current block prior to producing a compensation motion vector.
- the motion vectors of the respective neighboring blocks are final motion vectors provided by the motion vector selection part 250 which will be later described.
- the dynamic motion compensation unit 246 approximately estimates the motion vector of the current block based on the average vector of the motion vectors of the neighboring blocks as described above, which will be described in detail with reference to FIG. 6B . That is, the dynamic motion compensation unit 246 produces a compensation motion vector based on the point R of the minimum local error value that is located at a nearest distance to a calculated average vector of at least one minimum local error value. However, since it is uncertain that any of the plural minimum local error values indicates a substantial motion in case that the pattern image of a current block is a dynamic pattern image, correlation among motion vectors of neighboring blocks is used to produce a compensated motion vector (MV) of the current block.
- MV compensated motion vector
- the motion vector selection part 250 selects as a final motion vector (final MV) one of a temporary motion vector estimated by the motion estimation part 210 and a compensation motion vector compensated by the motion compensation part 240 based on a decision result of the pattern decision unit 228 .
- final MV final motion vector
- the motion vector selection part 250 selects as a final motion vector the temporary motion vector estimated by the motion estimation part 210 .
- the motion vector selection part 250 selects as a final motion vector the compensation motion vector produced by any of the static and dynamic motion compensation units 242 and 246 . That is, if the pattern decision unit 228 decides the current block as a pattern image and the pattern classification part 230 classifies the pattern image into a static pattern image, the motion vector selection part 250 selects as a final motion vector a compensation motion vector produced from the static motion compensation unit 242 .
- the motion vector selection part 250 selects as the final motion vector the compensation motion vector produced from the dynamic motion compensation unit 246 .
- FIG. 8 is a flow chart for schematically explaining a motion vector compensation process based on the pattern of FIG. 2 .
- the motion estimation error calculation unit 212 compares a current block to a search area established on a previous frame, and calculates plural motion estimation error values (S 805 ). Any of the SAD, MAD, and MSE values can be applied for the plural motion estimation error values, and the present invention applies the SAD values therefor. Accordingly, hereinafter, the plural motion estimation error values calculated are referred to as a SAD map.
- the motion vector estimation unit 214 estimates a temporary motion vector of the current block based on a point having the minimum SAD value in the SAD map (S 810 ).
- the integral projection unit 222 applies an integral projection to the SAD map, and produces plural projection values (S 815 ).
- the projection value verification unit 224 verifies the number of minimum local projection values of the plural projection values produced in step S 815 (S 820 ).
- the period calculation unit 226 calculates the number of repeated periods including at least one of the minimum local projection values sequentially appearing in the plural projection values, and divides a length of the search area by the number of repeated periods to calculate an actual period (S 825 ).
- step S 825 the pattern decision unit 228 analyzes the number of the minimum local projection values verified in step S 820 and the number of repeated periods calculated in step S 825 , and decides whether the current block is a pattern image (S 830 ). If step S 830 decides that the current block is not the pattern image, the motion vector selection part 250 selects the temporary motion vector estimated in step S 810 as a final motion vector of the current block (S 835 ).
- step S 830 decides that the current block is the pattern image
- the pattern classification part 230 classifies the pattern image into a static pattern image or a dynamic pattern image (S 840 ).
- step S 840 classifies the pattern image into the static pattern image
- the static motion compensation unit 242 estimates the compensation motion vector based on the point of a minimum local error value that is located at a nearest distance to the center of the search area (S 845 ). Further, the motion vector selection part 250 selects the compensation motion vector estimated in step S 845 as the final motion vector of the current block (S 850 ).
- step S 840 decides that the pattern image is the dynamic pattern image
- the average vector calculation unit 244 calculates an average vector of the motion vectors of the blocks neighboring to the current block (S 855 ). If step S 855 is carried out, the dynamic motion compensation unit 246 estimates a compensation motion vector based on the point of the minimum local error value of the SAD map that is nearest to the average vector calculated in step S 855 (S 860 ). Further, the motion vector selection part 250 selects the compensation motion vector estimated in step S 860 as the final motion vector of the current block (S 850 ).
- the pattern decision unit 228 decides that the current block is the pattern image
- the pattern classification part 230 , static motion compensation part 242 , and dynamic motion compensation part 246 read information of the points of the minimum local error values respectively verified by the vertical projection value verification portion 224 a and the horizontal projection value verification portion 224 b , and carry out their own operations.
- the pattern classification part 230 static motion compensation unit 242 , and dynamic motion compensation unit 246 extract information of the points of the minimum local projection values respectively verified by the vertical projection value verification portion 224 a and the horizontal projection value verification portion 224 b , and carry out their own operations.
- the motion estimation part 210 estimates SAD values and temporary motion vectors of the blocks equally divided as above. Further, the motion estimation part 210 sends the SAD map consisting of the SAD values to the pattern decision part 220 and the pattern classification part 230 .
- the pattern decision part 220 analyzes the SAD map that has been sent, and decides whether the current block is a pattern image. To do this, the integral projection unit 222 of the pattern decision part 220 applies the integral projection to the SAD map of the current block, and produces plural integral vectors. Preferably, the integral projection unit 222 applies the vertical integral projection and the horizontal integral projection to respectively produce plural vertical integral vectors and horizontal integral vectors.
- the vertical integral vector is referred to as a ‘vertical projection value’
- the horizontal integral vector is referred to as a ‘horizontal projection value’.
- the projection value verification unit 224 verifies the number of minimum local vertical projection values of the plural vertical projection values produced, n v , and the number of minimum local horizontal projection values of the horizontal projection values produced, n h . Further, the projection value verification unit 224 verifies information of the points of the plural minimum local vertical and horizontal projection values, and send the information to the pattern classification part 230 , static motion compensation unit 242 , and dynamic motion compensation unit 246 .
- the pattern classification part 230 classifies a pattern image into a static pattern image or a dynamic pattern image based on the point of the minimum local projection value that is nearest to the center of the search area and indicates that there exists no motion.
- the pattern classification part 230 compares the minimum local projection value of at least one or more minimum local projection values at a point located at a nearest distance to the center of the search area to a period of the periodic pattern image calculated by the period calculation unit 226 , and classifies the pattern image into one in the vertical direction or one in the horizontal direction.
- the minimum local projection value refers to a value smaller than the SAD projection values corresponding to predetermined first and second projection values of the SAD projection values.
- FIG. 6A and FIG. 6B show the classification of the pattern image of the current block, establishing the minimum global error value as the minimum global projection value and the minimum local error value as the minimum local projection value.
- the motion compensation part 240 carries out motion compensation based on a result of the pattern classification part 230 .
- the motion compensation part 240 uses information of the points of the plural integral vectors, that is, integral projection values verified by the projection value verification unit 224 , and produces a compensation motion vector of the current block.
- the static motion compensation unit 242 uses information of the points of the plural integral projection values, and produces a compensation motion vector of the current block decided as the static pattern image
- the dynamic motion compensation unit 246 uses the information of the points of the plural integral projection values and an average vector calculated by the average vector calculation unit 244 , and produces a compensation motion vector of the current block decided as the dynamic pattern image.
- the motion vector selection part 250 selects as a final motion vector(final MV) either a temporary motion vector estimated by the motion estimation part 210 or a compensation motion vector compensated for by the motion compensation part 240 based on a decision result of the pattern decision unit 228 .
- the motion vector selection part 250 selects as the final motion vector the temporary motion vector estimated by the motion estimation part 210 .
- the motion vector selection part 250 selects as the final motion vector a compensation motion vector produced by either the static motion compensation unit 242 or the dynamic motion compensation unit 246 .
- the vertical and horizontal period calculators 226 a and 226 b of the present invention calculate vertical and horizontal periods regardless of a decision result of the pattern decision unit 228 , but can be adapted to calculate the vertical and horizontal periods only if the pattern decision unit 228 decides that the current block is a pattern image.
- the pattern analysis-based motion vector compensation apparatus and method according to the present invention can analyze, based on motion estimation error values, a pattern of a pattern area at which a precise motion vector is hardly estimated, and estimate precise motion vectors by use of correlation among neighboring blocks. Therefore, when applied to the FRC, IPC, and so on, the present invention can effectively reduce the image quality degradation phenomenon such as block artifact occurring in the compensation images due to motion vector estimation errors.
Abstract
Description
- This application claims the benefit of Korean Patent Application No. 2003-53232, filed on Jul. 31, 2003, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference.
- 1. Field of the Invention
- The present invention relates to a pattern analysis-based motion vector compensation apparatus and method, and more particularly to an apparatus and method for compensating for estimated motion vectors by deciding whether blocks for motion estimation are a pattern image.
- 2. Description of the Related Art
- The motion estimation between image frames is necessarily carried out in image processing such as the frame rate up-conversion (FRC) and the Interlaced-to-progressive conversion (IPC). The motion estimation is used to estimate motion vectors and compensate for motions, and, in general, the block matching algorithm (BMA) is used for the estimation.
- The BMA compares two consecutive frames/fields matching one block with another, and estimates one motion vector per block. Here, the motion vector is estimated by use of the known motion estimation error values, for example, the sum of Absolute Difference (SAD) values. Further, the motion compensation process compensates for motions by use of the estimated motion vectors.
- However, the conventional motion estimation based on the block-based estimation brings out inaccurate motion vectors from time to time. In particular, precise motion vectors are hardly estimated by use of the BMA only with respect to images of periodic pattern, that is, identical images periodically repeated as shown in
FIG. 1A . The periodic pattern is a pattern repeating a certain image in certain intervals, such as stairs, zebra, lattice, and so on. - Thus, when motion compensation is applied to the images of the periodic pattern based on the motion vectors estimated by use of the BMA, the block artifact occurs in the frame/field image compression as shown in
FIG. 1B . The block artifact refers to a boundary discontinuity among neighboring blocks, and causes visible and objectionable images together with lower image quality. Such block artifact is caused when the block-based motion compensation is applied by use of estimated motion vectors without considering compensation with neighboring blocks. - The present invention has been developed in order to solve the above drawbacks and other problems associated with the conventional arrangement. An aspect of the present invention is to provide a pattern analysis-based motion vector compensation apparatus and method capable of compensating for motion vectors in consideration of correlation among blocks by deciding whether a certain pattern image exists in blocks, to thereby reduce block artifact resulting from motion vectors inaccurately estimated on certain blocks.
- The forgoing and other objects and advantages are substantially realized by providing a pattern analysis-based motion vector compensation apparatus comprising a motion estimation part for comparing a current block of a current frame/field to a certain search area established in a previous frame/field, calculating plural motion estimation error values, and estimating a temporary motion vector of the current block based on the plural motion estimation error values; a pattern decision part for analyzing the calculated motion estimation error values, and deciding whether the current block is a pattern image; a motion compensation part for producing a compensation motion vector of the current block by use of the plural motion estimation error values if the current block is decided as the pattern image; and a motion vector selection part for selecting as a final motion vector either the temporary motion vector estimated by the motion estimation part or the compensation motion vector compensated for by the motion compensation part based on a result of the pattern decision part.
- In detail, the motion estimation part includes a motion estimation error calculation unit for applying a block-matching approach to the current block and search area of the current frame/field divided into blocks in a certain size, and calculating the plural motion estimation error values; and a motion vector estimation unit for estimating the temporary motion vector of the current block based on a point of a minimum motion estimation error value of the plural motion estimation error values produced.
- In an exemplary embodiment, the pattern decision part includes an integral projection unit for applying an integral projection to the plural motion estimation error values, and producing plural projection values; a projection value verification unit for verifying the number of at least one minimum local projection values between two points corresponding to predetermined first and second projection values but smaller than the first and second projection values of the plural projection values; a period calculation unit for calculating the number of repeated periods including at least one minimum local projection value sequentially appearing in the plural projection values, and dividing a length of the search area by the number of repeated periods to calculate a period; and a pattern decision unit for analyzing the verified number of minimum local projection values and the calculated number of repeated periods, and deciding whether the current block is the pattern image.
- Further, the integral projection unit includes a vertical projector for applying the integral projection to the plural motion estimation error values in a vertical direction, and producing plural vertical projection values; and a horizontal projector for applying the integral projection to the plural motion estimation error values in a horizontal direction, and producing plural horizontal projection values.
- Further, the projection value verification unit includes a vertical projection value verification portion for verifying the number of minimum vertical local projection values between two points between predetermined first and second vertical projection values but smaller than the first and second vertical projection values of the plural vertical projection values produced; and a horizontal projection value verification portion for verifying the number of minimum local horizontal projection values between two points corresponding to predetermined first and second horizontal projection values but smaller than the first and second horizontal projection values of the plural horizontal projection values produced.
- Further, the period calculation unit includes a vertical period calculator for calculating the number of repeated vertical periods including at least one or more minimum local vertical projection values sequentially appearing in the plural vertical projection values; and a horizontal period calculator for calculating the number of repeated horizontal periods including at least one or more minimum local horizontal projection values sequentially appearing in the plural horizontal projection values.
- In an exemplary embodiment, the vertical and horizontal period calculators calculate the numbers of repeated vertical and horizontal periods based on the following equations:
-
- where, γv denotes the number of repeated vertical periods, Lv,a a vertical projection length obtained from adding all absolute differences between two neighboring vertical projection values of the plural vertical projection values, Lv,p an absolute difference between the maximum and minimum vertical projection values of the plural vertical projection values, γh the number of repeated horizontal periods, Lh,a a horizontal projection length obtained from adding all absolute differences between two neighboring horizontal projection values of the plural horizontal projection values, and Lh,p an absolute difference between the maximum and minimum horizontal projection values of the plural horizontal projection values.
- Further, the vertical period calculator divides a horizontal length of the search area by the number of repeated vertical periods and calculates a vertical period of the pattern image, and the horizontal period calculator divides a vertical length of the search area by the number of repeated horizontal periods and calculates a horizontal period of the pattern image.
- In detail, the pattern decision part decides that the current block is a pattern image, if the number of the minimum local projection values is at least two and the number of repeated periods is larger than a predetermined first threshold value.
- In another exemplary embodiment, the pattern analysis-based motion vector compensation apparatus further comprises a pattern classification part for classifying the pattern image into a static pattern image having zero motion of the current block or into a dynamic pattern image occurring due to motion of the current block, depending on a point of a minimum local error value that is located at a nearest distance to the center of the search area if the current block is the pattern image.
- Further, the pattern classification part compares the minimum local error value located at the nearest distance to the center of the search area to the period calculated by the period calculation part, and classifies the pattern image, the minimum local error value being one of at least one minimum local error values between two points corresponding to predetermined first and second error values but smaller than the first and second error values of the plural motion estimation error values.
- Further, the pattern classification part decides that the pattern image is the static pattern image, if the pattern image satisfies a condition as below:
|u|=α·p -
- where, u denotes a point of the first minimum local error value that is located at a nearest distance, a denotes a certain constant, and p denotes a period calculated by the period calculation part.
- In detail, the motion compensation part includes a static motion compensation unit for estimating and producing the compensation motion vector based on a point corresponding to the first minimum local error value, if the pattern image is classified into the static pattern image; an average vector calculation unit for calculating an average vector of motion vectors of blocks neighboring to the current block, if the pattern image is classified into the dynamic pattern image; and a dynamic motion compensation unit for estimating and producing the compensation motion vector based on a point of a second minimum local error value of at least one or more minimum local error values that is located at a nearest distance to the calculated average vector.
- Further, if the pattern decision part decides that the current block is the pattern image, the motion vector selection part selects as the final motion vector the compensation motion vector estimated by the motion compensation part.
- Further, if the pattern classification part classifies the pattern image into the static pattern image, the motion vector selection part selects a compensation motion vector estimated by the static motion compensation unit, and, if the pattern classification part classifies the pattern image into the dynamic pattern image, the motion vector selection part selects a compensation motion vector estimated by the dynamic motion compensation unit.
- In an exemplary embodiment, first and second motion estimation error values are calculated by use of Sum of Absolute Difference (SAD), Mean Absolute Difference (MAD), or Mean Square Error (MSE).
- In the meantime, the forgoing and other objects and advantages are substantially realized by providing a pattern analysis-based motion vector compensation method, comprising a motion estimation step for comparing a current block of a current frame/field to a certain search area established in a previous frame/field, calculating plural motion estimation error values, and estimating a temporary motion vector of the current block based on the plural motion estimation error values; a pattern decision step for analyzing the calculated motion estimation error values, and deciding whether the current block is a pattern image; a motion compensation step for producing a compensation motion vector of the current block by use of the plural motion estimation error values if the current block is decided as the pattern image; and a motion vector selection step for selecting as a final motion vector either the temporary motion vector estimated by the motion estimation step or the compensation motion vector compensated for by the motion compensation step based on a result of the pattern decision step.
- In detail, the motion estimation step includes a motion estimation error calculation step for applying a block-matching approach to the current block and search area of the current frame/field divided into blocks in a certain size, and calculating the plural motion estimation error values; and a motion vector estimation step for estimating the temporary motion vector of the current block based on a point of a minimum motion estimation error value of the plural motion estimation error values produced.
- Further, the pattern decision step includes an integral projection step for applying an integral projection to the plural motion estimation error values, and producing plural projection values; a projection value verification step for verifying the number of at least one minimum local projection values between two points corresponding to predetermined first and second projection values but smaller than the first and second projection values of the plural projection values; a period calculation step for calculating the number of repeated periods including at least one minimum local projection value sequentially appearing in the plural projection values, and dividing a length of the search area by the number of repeated periods to calculate a period; and a pattern decision step for analyzing the verified number of minimum local projection values and the calculated number of repeated periods, and deciding whether the current block is the pattern image.
- Further, the integral projection step includes a vertical projection step for applying the integral projection to the plural motion estimation error values in a vertical direction, and producing plural vertical projection values; and a horizontal projection step for applying the integral projection to the plural motion estimation error values in a horizontal direction, and producing plural horizontal projection values.
- Further, the projection value verification step includes a vertical projection value verification step for verifying the number of minimum vertical local projection values between two points corresponding to predetermined first and second vertical projection values but smaller than the first and second vertical projection values of the plural vertical projection values produced; and a horizontal projection value verification step for verifying the number of minimum local horizontal projection values between two points corresponding to predetermined first and second horizontal projection values but smaller than the first and second horizontal projection values of the plural horizontal projection values produced.
- Further, the period calculation method includes a vertical period calculation step for calculating the number of repeated vertical periods including at least one or more minimum local vertical projection values sequentially appearing in the plural vertical projection values; and a horizontal period calculation step for calculating the number of repeated horizontal periods including at least one or more minimum local horizontal projection values sequentially appearing in the plural horizontal projection values.
- In an exemplary embodiment, the vertical and horizontal period calculation steps calculate the numbers of repeated vertical and horizontal periods based on the following equations:
-
- where, γv denotes the number of repeated vertical periods, Lv,a a vertical projection length obtained from adding all absolute differences between two neighboring vertical projection values of the plural vertical projection values, Lv,p an absolute difference between the maximum and minimum vertical projection values of the plural vertical projection values, γh the number of repeated horizontal periods, Lh,a a horizontal projection length obtained from adding all absolute differences between two neighboring horizontal projection values of the plural horizontal projection values, and Lh,p an absolute difference between the maximum and minimum horizontal projection values of the plural horizontal projection values.
- Further, the vertical period calculation step divides a horizontal length of the search area by the number of repeated vertical periods and calculates a vertical period of the pattern image, and the horizontal period calculation step divides a vertical length of the search area by the number of repeated horizontal periods and calculates a horizontal period of the pattern image.
- The pattern decision step decides that the current block is a pattern image, if the number of the minimum local projection values is at least two and the number of repeated periods is larger than a predetermined first threshold value.
- In detail, the pattern analysis-based motion vector compensation method further comprises a pattern classification step for classifying the pattern image into a static pattern image having zero motion of the current block or into a dynamic pattern image occurring due to motion of the current block, depending on a point of a minimum local error value that is located at a nearest distance to the center of the search area if the current block is the pattern image.
- Further, the pattern classification step compares a first minimum local error value located at the nearest distance to the center of the search area to the period calculated by the period calculation step, and classifies the pattern image, the first minimum local error value being one of at least one minimum local error values between two points corresponding to predetermined first and second error values but smaller than the first and second error values of the plural motion estimation error values.
- The pattern classification step decides that the pattern image is the static pattern image, if the pattern image satisfies a condition as below:
|u|=α·p -
- where, u denotes a point of the first minimum local error value that is located at the nearest distance, a denotes a certain constant, and p denotes a period calculated by the period calculation step.
- In detail, the motion compensation step includes a static motion compensation step for estimating and producing the compensation motion vector based on a point corresponding to the first minimum local error value, if the pattern image is classified into the static pattern image; an average vector calculation step for calculating an average vector of motion vectors of blocks neighboring to the current block, if the pattern image is classified into the dynamic pattern image; and a dynamic motion compensation step for estimating and producing the compensation motion vector based on a point of a second minimum local error value of at least one minimum local error values that is located at a nearest distance to the calculated average vector.
- Further, if the pattern decision step decides that the current block is the pattern image, the motion vector selection step selects as the final motion vector the compensation motion vector estimated by the motion compensation step.
- If the pattern classification step classifies the pattern image into the static pattern image, the motion vector selection step selects a compensation motion vector estimated by the static motion compensation step, and, if the pattern classification step classifies the pattern image into the dynamic pattern image, the motion vector selection step selects a compensation motion vector estimated by the dynamic motion compensation step.
- The above aspects and features of the present invention will be more apparent by describing certain embodiments of the present invention with reference to the accompanying drawings, in which:
-
FIG. 1A is a view for showing an image of periodic pattern in general; -
FIG. 1B is a view for showing a simulation of an image of periodic pattern with block artifact by use of a conventional motion estimation method; -
FIG. 2 is a block diagram for schematically showing a pattern-based motion vector compensation apparatus according to a preferred embodiment of the present invention; -
FIG. 3 is a block diagram for showing in detail a pattern decision part shown inFIG. 2 ; -
FIG. 4 is a graph for showing in three dimensions a part of a SAD map calculated by a motion estimation error calculation unit shown inFIG. 2 ; -
FIG. 5 is a graph for illustrating plural vertical projection values resulting from the certain SAD map shown inFIG. 4 ; -
FIG. 6A andFIG. 6B are views for showing a two-dimensional SAD map resulting from an image with a static pattern and a two-dimensional SAD map resulting from an image with a dynamic pattern, respectively; -
FIG. 7 is a view for simulating an image with reduced block artifact by the pattern decision part ofFIG. 2 ; and -
FIG. 8 is a flow chart for schematically explaining a pattern-based motion vector compensation process ofFIG. 2 . - Hereinafter, the present invention will be described in detail with respect to the accompanying drawings.
-
FIG. 2 is a block diagram for schematically showing a pattern analysis-based motion vector compensation apparatus according to an exemplary embodiment of the present invention, andFIG. 3 is a block diagram for showing in detail a pattern decision part provided inFIG. 2 . - Referring to
FIG. 2 , the pattern analysis-based motionvector compensation apparatus 200 has amotion estimation part 210, a pattern decision part, apattern classification part 230, amotion compensation part 240, and a motionvector selection part 250. - The
motion estimation part 210 includes a motion estimationerror calculation unit 212 and a motionvector estimation unit 214. - The motion estimation
error calculation unit 212 divides a current input frame/field fn (hereinafter, referred to as ‘current frame’) into blocks each having a certain size, and estimates a motion vector v of each block. In more detail, the motion estimationerror calculation unit 212 compares the current block for an estimated motion vector, which is one of the divided blocks, to search an area established in a previous frame/field fn−1 (hereinafter, referred to as ‘previous frame’), so as to calculate plural motion estimation error values. - Here, the plural motion estimation error values can be estimated by bidirectional BMA, unidirectional BMA, or all the known approaches that can estimate motion vectors. Further, the motion estimation error values can be calculated by various approaches such as Sum of Absolute Difference(SAD), Mean Absolute Difference(MAD), Mean Square Error(MSE), and so on, but the present invention applies the SAD values, so the values are referred to as “SAD values”.
- The motion
vector estimation unit 214 estimates a temporary motion vector (temporary MV) of a current block from the minimum error value of the plural motion estimation error values calculated. - In such an approach, the
motion estimation part 210 estimates the SAD values and temporary motion vectors of the divided blocks. - The calculated SAD values form a SAD map, and the size of the SAD map is identical to the size of search area K×K.
FIG. 4 is a graph for showing a part of a calculated SAD map in three dimensions. Referring toFIG. 4 , x denotes a horizontal direction axis of a search area, and y a vertical direction axis of the search area. For example, the graph shows that a SAD value at (x,y)=(0,5) is ‘1500’. Here, the plural SAD values forming the SAD map are obtained by using the BMA as stated above. - The SAD map of a current block of the blocks to which motion compensation is applied is sent to the motion
vector estimation unit 214, thepattern decision part 220, thepattern classification part 230, and themotion compensation part 240 that will be later described. Further, the temporary motion vector of the current block is sent to the motionvector selection part 250 to be later described. - The
pattern decision part 220 analyzes the calculated SAD map, that is, the plural SAD values forming a search area, and decides whether there exists a pattern image in the current block. The pattern image is an image periodically repeated in certain intervals, which is referred to as a ‘periodic pattern image’. Referring toFIG. 3 , thepattern decision part 220 includes anintegral projection unit 222, a projectionvalue verification unit 224, aperiod calculation unit 226, and apattern decision unit 228. Further, thepattern decision part 220 decides a periodic pattern image by considering the vertical and horizontal direction characteristics of the SAD map respectively. Hereinafter, a detailed description will be made on a process of each block to decide the periodic pattern in the horizontal direction, and a description on the decision of the periodic pattern in the vertical direction will be briefly made for the sake of explanation since such a decision is nearly the same as the decision of the periodic pattern in the horizontal direction. - The
integral projection unit 222 applies the integral projection to the SAD map of a current block to produce plural integral vectors (hereinafter, referred to as ‘projection values’). Accordingly, degradation of image caused by noise can be minimized. - In the present invention, the
integral projection unit 222 applies the vertical and horizontal projections, so theintegral projection unit 222 has avertical projector 222 a and ahorizontal projector 222 b. - The
vertical projector 222 a applies the vertical integral projection to the SAD map of a current block in the vertical direction to produce plural vertical projection values. That is, thevertical projector 222 a applies the vertical integral projection in order to decide a periodic pattern in the vertical direction, by which the SAD map of a K×K matrix is converted into a vertical projection value of a K×1 matrix, for example. -
FIG. 5 is a graph for showing plural vertical projection values produced based on the SAD map as shown inFIG. 4 . - An xth vertical projection value produced by the vertical integral projection can be expressed in
Equation 1 as below: - Referring to
FIG. 5 andEquation 1, x denotes a vertical direction axis of a search area, Sv(x) denotes a vertical projection value at the xth point, E(x,y) denotes a SAD value corresponding to a motion vector(x,y), and K denotes the total number of SAD values located on the y axis of the search area. - The
horizontal projector 222 b applies the integral projection to the SAD map of a current block in the horizontal direction to produce plural horizontal projection values. That is, thehorizontal projector 222 b applies the horizontal integral projection to decide a periodic pattern in the vertical direction, by which the SAD map of a (K×K) matrix is converted into a horizontal projection value of a (K×1) matrix. The yth horizontal projection value produced by the horizontal integral projection can be expressed in Equation 2 as below: - Referring to Equation 2, y denotes a vertical direction axis of a search area, Sh(y) denotes a horizontal projection value at the yth point, E(x,y) denotes a SAD value corresponding to a motion vector(x,y), and K denotes the total number of SAD values located on the x axis of the search area.
- The plural vertical and horizontal projection values produced are sent to the projection
value verification unit 224. - The projection
value verification unit 224 verifies the number of minimum local projection values n of the plural projection values produced by theintegral projection unit 222. Here, the minimum local projection values are located between the first and second projection values of the plural projection values, but smaller than the first and second projection values. The verified minimum local projection values are used when thepattern decision part 228 decides whether there exists a pattern image, wherein thepattern decision part 228 will be described later. - In the present invention, the projection
value verification unit 224 has a vertical projectionvalue verification portion 224 a and a horizontal projectionvalue verification portion 224 b. - The vertical projection
value verification portion 224 a verifies the number of minimum local vertical projection values nv of the plural vertical projection values produced. The minimum local vertical projection values are located between the first and second vertical projection values of the plural vertical projection values, but smaller than the first and second vertical projection values. Referring toFIG. 5 , when a certain vertical projection value Sv (x) is smaller than two neighboring vertical projection values Sv (x−1) and Sv (x+1), the Sv (x) corresponds to the minimum local vertical projection value. - The horizontal projection
value verification portion 224 b verifies the number of minimum local horizontal projection values nh of the plural horizontal projection values produced. Here, the minimum local horizontal projection values are located between the first and second horizontal projection values of the plural horizontal projection values, but smaller than the first and second horizontal projection values. - Further, the vertical and horizontal
projection verification portions pattern classification part 230, a staticmotion compensation unit 242, and a dynamicmotion compensation unit 246 that will be later described. - The
period calculation unit 226 calculates the number of repeated periods γ including at least one of the minimum local projection values that sequentially appear in the plural projection values produced from theintegral projection unit 222. Further, theperiod calculation unit 226 divides the length or size of a search area by the number of repeated periods, and calculates a period T. - In the present invention, the
period calculation unit 226 has avertical period calculator 226 a and ahorizontal period calculator 226 b. - The
vertical period calculator 226 a calculates the number of repeated vertical periods, using Equation 3. In here, the vertical periods refer to ones including at least one of the minimum local vertical projection values that sequentially appear in the plural vertical projection values produced from thevertical projector 222 a. -
- where, γv denotes the number of repeated vertical periods, Lv,a denotes a vertical projection length obtained from adding all absolute differences between two neighboring vertical projection values of the plural vertical projection values, and Lv,p denotes an absolute difference of the maximum and minimum vertical projection values (Sv,max, Sv,min) of the plural vertical projection values. In Equation 3, the absolute difference between the maximum and minimum vertical projection values (Sv,max, Sv,min) of the plural vertical projection values refers to a difference between the vertical projection values that corresponds to half a period, so the number of repeated vertical periods can be calculated by using Equation 3. Accordingly, the 2Lv,p refers to a difference corresponding to one period.
- Further, the
vertical period calculator 226 a calculates a vertical period based on Equation 4 as below: -
- where, Tv denotes a vertical period, K denotes a horizontal direction length of a search area, and γv denotes the number of repeated vertical periods. Referring to Equation 4, the
vertical period calculator 226 a divides the horizontal direction length of a search area by the number of repeated vertical periods, and then calculates a vertical period.
- where, Tv denotes a vertical period, K denotes a horizontal direction length of a search area, and γv denotes the number of repeated vertical periods. Referring to Equation 4, the
- On the other hand, the
horizontal period calculator 226 b calculates the number of repeated horizontal periods based onEquation 5. Here, the horizontal periods refer to ones including at least one of the minimum local horizontal projection values that sequentially appear in the plural horizontal projection values produced from thehorizontal projector 222 b. -
- where, γh denotes the number of repeated vertical periods, Lh,a denotes a horizontal projection length obtained from adding all absolute differences between two neighboring horizontal projection values of the plural horizontal projection values, and Lh,p denotes an absolute difference between the maximum and minimum horizontal projection values(Sh,max, Sh,min) of the plural horizontal projection values. Referring to
Equation 5, the difference between the maximum and minimum horizontal projection values of the horizontal projection values refers to a difference between the horizontal projection values corresponding to half a period, so the number of repeated horizontal periods can be calculated by use ofEquation 5. Thus, the 2Lh,p refers to a difference corresponding to one period.
- where, γh denotes the number of repeated vertical periods, Lh,a denotes a horizontal projection length obtained from adding all absolute differences between two neighboring horizontal projection values of the plural horizontal projection values, and Lh,p denotes an absolute difference between the maximum and minimum horizontal projection values(Sh,max, Sh,min) of the plural horizontal projection values. Referring to
- Further, the
horizontal period calculator 226 b calculates a horizontal period using Equation 6 as below: -
- where, Th denotes a horizontal period, K denotes a vertical direction length of a search area, and γh denotes the number of repeated horizontal periods. Referring to Equation 6, the
horizontal period calculator 226 b divides the vertical direction length of a search area by the number of repeated horizontal periods, and then calculates a horizontal period.
- where, Th denotes a horizontal period, K denotes a vertical direction length of a search area, and γh denotes the number of repeated horizontal periods. Referring to Equation 6, the
- The
pattern decision unit 228 analyzes the number of the minimum local projection values n verified by the projectionvalue verification unit 224 and the number of repeated periods γ calculated from theperiod calculation unit 226, and decides whether there exists a pattern image in a current block. In more detail, thepattern decision unit 228 decides the current block as a periodic pattern image if there exist at least two minimum local projection values and the number of repeated periods is larger than a predetermined first threshold value. - For example, when deciding a periodic pattern in the vertical direction, the
pattern decision unit 228 decides that the current block is a periodic pattern image in the vertical direction if there exist at least two minimum local vertical projection values verified by the vertical projectionvalue verification portion 224 a and the vertical period calculated by thevertical period calculator 226 a is larger than the first threshold value of 1.75. Further, as for the decision on a periodic pattern in the vertical direction, the established first threshold value can be identical to or different from the value of 1.75. - Further, the
pattern decision unit 228 sends the decided result to thepattern classification part 230,motion compensation part 240, and motionvector selection part 250 that will be later described. - If the
pattern decision unit 228 decides the current block as a pattern image, thepattern classification part 230 classifies the pattern image into any of a static pattern image and a dynamic pattern image based on where the minimum local error value exists that is nearest to the center of a search area (that is, a zero vector), which means there is no motion. Here, the static pattern image refers to a periodic pattern image having no substantial motion (that is, a zero vector) of the current block, and the dynamic pattern image refers to a periodic pattern image having substantial motion of the current block. - In more detail, the
pattern classification part 230 compares the minimum local error value, which is a value of at least one minimum local error value located at a nearest distance to the center of the search area, with a period of the periodic pattern image calculated by theperiod calculation unit 226, and classifies a pattern image into one in the vertical direction or one in the horizontal direction. The minimum local error value refers to a value of the SAD map that is between predetermined first and second error values, but smaller than the first and second error values. - For example, the
pattern classification part 230 classifies the periodic pattern images in horizontal direction, using Equation 7 as below:
periodic pattern image in horizontal direction=static pattern image, if |x′|<αTv, or, dynamic pattern image, otherwise [Equation 7] -
- where, x′ denotes a location of the minimum local error value nearest at distance to the center of a search area, Tv denotes a vertical period, and a denotes a certain constant of ⅛ in the present invention but not limited to the constant.
-
FIG. 6A is a graph for explaining a SAD map of a static pattern image, andFIG. 6B is a graph for explaining a SAD map of a dynamic pattern image, which are two-dimensional graphs of the center region of the search area of the SAD map ofFIG. 4 . - First, referring to Equation 7 and
FIG. 6A , points P and R each denote the minimum local error values, a point C denotes the center of a search area indicating a location at which there exists no motion of a current block, and L denotes a range moving by a in the left and by Tv in the right from the center of the search area C, the range being established in advance for pattern image classification. In such an occasion, thepattern classification part 230 classifies into a static pattern image a periodic pattern image in the vertical direction that thepattern decision part 220 has decided, if the point R, that is, the minimum local error value nearest to the center C of the search area is included in the range L. Here, the minimum global error value refers to the minimum motion estimation error value of the plural motion estimation error values forming the SAD map. - On the other hand, referring to Equation 7 and
FIG. 6B , thepattern classification part 230 classifies into a dynamic pattern image the periodic pattern image in the horizontal direction that thepattern decision part 220 has decided, if the point R, that is, the minimum local error value nearest to the center of the search area is not included in the established range L. - Further, the
pattern classification part 230 classifies periodic pattern images in the vertical direction based on Equation 8.
periodic pattern image in vertical direction=static pattern image, if |y′|<αTh, or, dynamic pattern image, otherwise [Equation 8] - Referring to Equation 8, y′ denotes a point of the minimum local error value that is nearest at distance to the center of a search area, Th denotes a horizontal period, and α denotes a predetermined constant for which the present invention uses ⅛, but the constant is not limited to that value of {fraction (1/8)}. The classification into pattern images in the horizontal direction is carried out in a way similar to that into pattern images in the vertical direction, so explanation on such a classification will be omitted.
- The
motion compensation part 240 uses the SAD map to produce a compensation motion vector of a current block, if the current block is decided to be a pattern image. In particular, themotion compensation part 240 produces a motion vector adaptively compensated depending on pattern images in the vertical and horizontal directions decided by thepattern decision part 220. The point of the minimum local error value of a periodic pattern moves with the motion of the periodic pattern, so themotion compensation part 240 carries out compensation, considering the motion of the periodic pattern. That is, themotion compensation part 240 adaptively carries out motion compensation according to a classification result done by thepattern classification part 230. To do this, themotion compensation part 240 has a staticmotion compensation unit 242, an averagevector calculation unit 244, and a dynamicmotion compensation unit 246. - The static
motion compensation unit 242 produces the compensation motion vector of a current block decided as a static pattern image by thepattern classification part 230. In more detail, the staticmotion compensation unit 242 produces compensation motion vectors from the point of the minimum local error value, for example, the point R ofFIG. 6A , that is located at a nearest distance to the center of a search area. Thus, it is prevented that the vectors corresponding to the other minimum local error values, for example, the points P and Q ofFIG. 6A are decided as a final motion vector of the current block. The compensation process in the staticmotion compensation unit 242 decides consistent motion vectors from plural blocks forming one periodic pattern. Thus, estimated vectors are used in compensating for images, so the mismatching of patterns between blocks, that is, the block artifact phenomenon occurring in a periodic pattern image can be greatly reduced. - Further, if the
pattern classification part 230 decides the pattern image of a current block as a dynamic pattern image, the averagevector calculation unit 244 calculates an average vector of motion vectors of the blocks neighboring to the current block prior to producing a compensation motion vector. Here, the motion vectors of the respective neighboring blocks are final motion vectors provided by the motionvector selection part 250 which will be later described. - The dynamic
motion compensation unit 246 approximately estimates the motion vector of the current block based on the average vector of the motion vectors of the neighboring blocks as described above, which will be described in detail with reference toFIG. 6B . That is, the dynamicmotion compensation unit 246 produces a compensation motion vector based on the point R of the minimum local error value that is located at a nearest distance to a calculated average vector of at least one minimum local error value. However, since it is uncertain that any of the plural minimum local error values indicates a substantial motion in case that the pattern image of a current block is a dynamic pattern image, correlation among motion vectors of neighboring blocks is used to produce a compensated motion vector (MV) of the current block. - The motion
vector selection part 250 selects as a final motion vector (final MV) one of a temporary motion vector estimated by themotion estimation part 210 and a compensation motion vector compensated by themotion compensation part 240 based on a decision result of thepattern decision unit 228. In more detail, if thepattern decision unit 228 decides that the current block is not a pattern image, the motionvector selection part 250 selects as a final motion vector the temporary motion vector estimated by themotion estimation part 210. - On the other hand, if the
pattern decision unit 228 decides the current block as the pattern image, the motionvector selection part 250 selects as a final motion vector the compensation motion vector produced by any of the static and dynamicmotion compensation units pattern decision unit 228 decides the current block as a pattern image and thepattern classification part 230 classifies the pattern image into a static pattern image, the motionvector selection part 250 selects as a final motion vector a compensation motion vector produced from the staticmotion compensation unit 242. Further, if thepattern decision unit 228 decides the current block as a pattern image and thepattern classification part 230 classifies the pattern image as a dynamic pattern image, the motionvector selection part 250 selects as the final motion vector the compensation motion vector produced from the dynamicmotion compensation unit 246. - As stated above, by deciding whether a current block is a pattern image and then deciding a final motion vector, images can be obtained at image output terminals(not shown) of which block artifact has been remarkably reduced as shown in
FIG. 7 , compared toFIG. 1A . -
FIG. 8 is a flow chart for schematically explaining a motion vector compensation process based on the pattern ofFIG. 2 . - Referring to
FIG. 2 toFIG. 8 , first, the motion estimationerror calculation unit 212 compares a current block to a search area established on a previous frame, and calculates plural motion estimation error values (S805). Any of the SAD, MAD, and MSE values can be applied for the plural motion estimation error values, and the present invention applies the SAD values therefor. Accordingly, hereinafter, the plural motion estimation error values calculated are referred to as a SAD map. The motionvector estimation unit 214 estimates a temporary motion vector of the current block based on a point having the minimum SAD value in the SAD map (S810). - If the SAD map is calculated, the
integral projection unit 222 applies an integral projection to the SAD map, and produces plural projection values (S815). The projectionvalue verification unit 224 verifies the number of minimum local projection values of the plural projection values produced in step S815 (S820). Further, theperiod calculation unit 226 calculates the number of repeated periods including at least one of the minimum local projection values sequentially appearing in the plural projection values, and divides a length of the search area by the number of repeated periods to calculate an actual period (S825). - If step S825 is carried out, the
pattern decision unit 228 analyzes the number of the minimum local projection values verified in step S820 and the number of repeated periods calculated in step S825, and decides whether the current block is a pattern image (S830). If step S830 decides that the current block is not the pattern image, the motionvector selection part 250 selects the temporary motion vector estimated in step S810 as a final motion vector of the current block (S835). - On the other hand, if step S830 decides that the current block is the pattern image, the
pattern classification part 230 classifies the pattern image into a static pattern image or a dynamic pattern image (S840). - If step S840 classifies the pattern image into the static pattern image, the static
motion compensation unit 242 estimates the compensation motion vector based on the point of a minimum local error value that is located at a nearest distance to the center of the search area (S845). Further, the motionvector selection part 250 selects the compensation motion vector estimated in step S845 as the final motion vector of the current block (S850). - On the other hand, if step S840 decides that the pattern image is the dynamic pattern image, the average
vector calculation unit 244 calculates an average vector of the motion vectors of the blocks neighboring to the current block (S855). If step S855 is carried out, the dynamicmotion compensation unit 246 estimates a compensation motion vector based on the point of the minimum local error value of the SAD map that is nearest to the average vector calculated in step S855 (S860). Further, the motionvector selection part 250 selects the compensation motion vector estimated in step S860 as the final motion vector of the current block (S850). - In the meantime, in the present invention as described above, if the
pattern decision unit 228 decides that the current block is the pattern image, thepattern classification part 230, staticmotion compensation part 242, and dynamicmotion compensation part 246 read information of the points of the minimum local error values respectively verified by the vertical projectionvalue verification portion 224 a and the horizontal projectionvalue verification portion 224 b, and carry out their own operations. - If the
pattern decision unit 228 decides that the current block is the pattern image, thepattern classification part 230, staticmotion compensation unit 242, and dynamicmotion compensation unit 246 extract information of the points of the minimum local projection values respectively verified by the vertical projectionvalue verification portion 224 a and the horizontal projectionvalue verification portion 224 b, and carry out their own operations. - A description will be made as below on the extraction of the information of the points of the minimum local projection values and the respective operations with reference to
FIG. 2 . - The
motion estimation part 210 estimates SAD values and temporary motion vectors of the blocks equally divided as above. Further, themotion estimation part 210 sends the SAD map consisting of the SAD values to thepattern decision part 220 and thepattern classification part 230. - The
pattern decision part 220 analyzes the SAD map that has been sent, and decides whether the current block is a pattern image. To do this, theintegral projection unit 222 of thepattern decision part 220 applies the integral projection to the SAD map of the current block, and produces plural integral vectors. Preferably, theintegral projection unit 222 applies the vertical integral projection and the horizontal integral projection to respectively produce plural vertical integral vectors and horizontal integral vectors. Hereinafter, the vertical integral vector is referred to as a ‘vertical projection value’, and the horizontal integral vector is referred to as a ‘horizontal projection value’. - Further, the projection
value verification unit 224 verifies the number of minimum local vertical projection values of the plural vertical projection values produced, nv, and the number of minimum local horizontal projection values of the horizontal projection values produced, nh. Further, the projectionvalue verification unit 224 verifies information of the points of the plural minimum local vertical and horizontal projection values, and send the information to thepattern classification part 230, staticmotion compensation unit 242, and dynamicmotion compensation unit 246. - If the
pattern decision unit 228 decides that the current block is the pattern image, thepattern classification part 230 classifies a pattern image into a static pattern image or a dynamic pattern image based on the point of the minimum local projection value that is nearest to the center of the search area and indicates that there exists no motion. - In more detail, the
pattern classification part 230 compares the minimum local projection value of at least one or more minimum local projection values at a point located at a nearest distance to the center of the search area to a period of the periodic pattern image calculated by theperiod calculation unit 226, and classifies the pattern image into one in the vertical direction or one in the horizontal direction. The minimum local projection value refers to a value smaller than the SAD projection values corresponding to predetermined first and second projection values of the SAD projection values. - The description will be omitted on the pattern image classified by the
pattern classification part 230 since the above description with reference toFIG. 6A andFIG. 6B can be similarly applied. However,FIG. 6A andFIG. 6B show the classification of the pattern image of the current block, establishing the minimum global error value as the minimum global projection value and the minimum local error value as the minimum local projection value. - The
motion compensation part 240 carries out motion compensation based on a result of thepattern classification part 230. In particular, if the current block is a pattern image, themotion compensation part 240 uses information of the points of the plural integral vectors, that is, integral projection values verified by the projectionvalue verification unit 224, and produces a compensation motion vector of the current block. - In more detail, the static
motion compensation unit 242 uses information of the points of the plural integral projection values, and produces a compensation motion vector of the current block decided as the static pattern image, whereas the dynamicmotion compensation unit 246 uses the information of the points of the plural integral projection values and an average vector calculated by the averagevector calculation unit 244, and produces a compensation motion vector of the current block decided as the dynamic pattern image. - The motion
vector selection part 250 selects as a final motion vector(final MV) either a temporary motion vector estimated by themotion estimation part 210 or a compensation motion vector compensated for by themotion compensation part 240 based on a decision result of thepattern decision unit 228. - That is, if the
pattern decision unit 228 decides that the current block is not a pattern image, the motionvector selection part 250 selects as the final motion vector the temporary motion vector estimated by themotion estimation part 210. - On the other hand, if the
pattern decision unit 228 decides that the current block is the pattern image, the motionvector selection part 250 selects as the final motion vector a compensation motion vector produced by either the staticmotion compensation unit 242 or the dynamicmotion compensation unit 246. - Further, the vertical and
horizontal period calculators pattern decision unit 228, but can be adapted to calculate the vertical and horizontal periods only if thepattern decision unit 228 decides that the current block is a pattern image. - As described so far, the pattern analysis-based motion vector compensation apparatus and method according to the present invention can analyze, based on motion estimation error values, a pattern of a pattern area at which a precise motion vector is hardly estimated, and estimate precise motion vectors by use of correlation among neighboring blocks. Therefore, when applied to the FRC, IPC, and so on, the present invention can effectively reduce the image quality degradation phenomenon such as block artifact occurring in the compensation images due to motion vector estimation errors.
- The foregoing embodiment and advantages are merely exemplary and are not to be construed as limiting the present invention. The present teaching can be readily applied to other types of apparatuses. Also, the description of the embodiments of the present invention is intended to be illustrative, and not to limit the scope of the claims, and many alternatives, modifications, and variations will be apparent to those skilled in the art.
Claims (36)
|u|=α·p
|u|=α·p
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR20030053232A KR100573696B1 (en) | 2003-07-31 | 2003-07-31 | Apparatus and method for correction motion vector |
KR2003-53232 | 2003-07-31 |
Publications (2)
Publication Number | Publication Date |
---|---|
US20050025342A1 true US20050025342A1 (en) | 2005-02-03 |
US7555166B2 US7555166B2 (en) | 2009-06-30 |
Family
ID=34101805
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/887,897 Expired - Fee Related US7555166B2 (en) | 2003-07-31 | 2004-07-12 | Pattern analysis-based motion vector compensation apparatus and method |
Country Status (4)
Country | Link |
---|---|
US (1) | US7555166B2 (en) |
JP (1) | JP4155952B2 (en) |
KR (1) | KR100573696B1 (en) |
CN (1) | CN100372381C (en) |
Cited By (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070109448A1 (en) * | 2005-11-14 | 2007-05-17 | Lsi Logic Corporation | Noise adaptive 3D composite noise reduction |
US20070236579A1 (en) * | 2006-01-19 | 2007-10-11 | Jingqiang Li | Hand jitter reduction for compensating for linear displacement |
US20070296855A1 (en) * | 2006-06-22 | 2007-12-27 | Yunwei Jia | Video processing using region-based statistical measurements |
US20080031338A1 (en) * | 2006-08-02 | 2008-02-07 | Kabushiki Kaisha Toshiba | Interpolation frame generating method and interpolation frame generating apparatus |
US20090059068A1 (en) * | 2005-09-30 | 2009-03-05 | Toshiharu Hanaoka | Image display device and method |
US20090161011A1 (en) * | 2007-12-21 | 2009-06-25 | Barak Hurwitz | Frame rate conversion method based on global motion estimation |
US20090161763A1 (en) * | 2007-12-20 | 2009-06-25 | Francois Rossignol | Motion estimation with an adaptive search range |
US20090161010A1 (en) * | 2007-12-20 | 2009-06-25 | Integrated Device Technology, Inc. | Image interpolation with halo reduction |
WO2010008655A1 (en) * | 2008-07-16 | 2010-01-21 | Sony Corporation | Simple next search position selection for motion estimation iterative search |
US20110141369A1 (en) * | 2009-12-11 | 2011-06-16 | Renesas Electronics Corporation | Video signal processing device, video signal processing method, and non-transitory computer readable medium storing image processing program |
US20110176013A1 (en) * | 2010-01-19 | 2011-07-21 | Sony Corporation | Method to estimate segmented motion |
US20110216831A1 (en) * | 2010-03-08 | 2011-09-08 | Francois Rossignol | Apparatus and method for motion vector filtering based on local image segmentation and lattice maps |
US20110229056A1 (en) * | 2010-03-19 | 2011-09-22 | Sony Corporation | Method for highly accurate estimation of motion using phase correlation |
US20110249747A1 (en) * | 2010-04-12 | 2011-10-13 | Canon Kabushiki Kaisha | Motion vector decision apparatus, motion vector decision method and computer readable storage medium |
US8064722B1 (en) * | 2006-03-07 | 2011-11-22 | The United States Of America As Represented By The Secretary Of The Navy | Method and system for analyzing signal-vector data for pattern recognition from first order sensors |
US20120169890A1 (en) * | 2010-12-30 | 2012-07-05 | Stmicroelectronics Inc. | Motion estimation in imaging systems |
US8473474B1 (en) * | 2012-03-28 | 2013-06-25 | Sap Ag | Granularity-adaptive extraction of correlation structures in databases |
US20130176487A1 (en) * | 2012-01-11 | 2013-07-11 | Panasonic Corporation | Image processing apparatus, image capturing apparatus, and computer program |
US8537283B2 (en) | 2010-04-15 | 2013-09-17 | Qualcomm Incorporated | High definition frame rate conversion |
US20140072241A1 (en) * | 2012-09-12 | 2014-03-13 | Canon Kabushiki Kaisha | Image processing apparatus and image processing method |
US8676866B2 (en) | 2012-03-19 | 2014-03-18 | Sap Ag | Computing canonical hierarchical schemas |
US20150043785A1 (en) * | 2011-08-29 | 2015-02-12 | Veldandi Muninder | Method and Apparatus for Feature Computation and Object Detection Utilizing Temporal Redundancy between Video Frames |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8120658B2 (en) * | 2006-01-19 | 2012-02-21 | Qualcomm Incorporated | Hand jitter reduction system for cameras |
JP4743048B2 (en) * | 2006-08-31 | 2011-08-10 | カシオ計算機株式会社 | Imaging apparatus and program |
JP4752685B2 (en) * | 2006-08-31 | 2011-08-17 | カシオ計算機株式会社 | Imaging apparatus and program |
JP4869045B2 (en) | 2006-11-30 | 2012-02-01 | 株式会社東芝 | Interpolation frame creation method and interpolation frame creation apparatus |
JP4869049B2 (en) | 2006-12-08 | 2012-02-01 | 株式会社東芝 | Interpolated frame image creation method and interpolated frame image creation apparatus |
EP2188978A2 (en) * | 2007-09-10 | 2010-05-26 | Nxp B.V. | Method and apparatus for line-based motion estimation in video image data |
JP4406892B1 (en) * | 2008-11-28 | 2010-02-03 | 日本ビクター株式会社 | Repeated object detection apparatus and method |
KR101548269B1 (en) | 2008-12-02 | 2015-08-31 | 삼성전자주식회사 | Apparatus and method for estimating motion by block segmentation and combination |
KR101630688B1 (en) | 2010-02-17 | 2016-06-16 | 삼성전자주식회사 | Apparatus for motion estimation and method thereof and image processing apparatus |
JP5091994B2 (en) | 2010-09-02 | 2012-12-05 | 株式会社東芝 | Motion vector detection device |
WO2013005917A1 (en) * | 2011-07-06 | 2013-01-10 | 에스케이플래닛 주식회사 | Multicast-based content transmitting system and method, and device and method for estimating high-speed movement |
CN107942731B (en) * | 2017-12-11 | 2021-03-30 | 上海电机学院 | Pedestrian trestle active wave compensation device and method based on prediction |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5786860A (en) * | 1994-07-30 | 1998-07-28 | Korean Advanced Institute Of Science And Technology | High speed block matching for bi-directional motion vector estimation |
US6430317B1 (en) * | 1997-12-31 | 2002-08-06 | Sarnoff Corporation | Method and apparatus for estimating motion using block features obtained from an M-ary pyramid |
US7336707B2 (en) * | 2003-06-06 | 2008-02-26 | Samsung Electronics Co., Ltd. | Method and apparatus for detecting improper area for motion compensation in video signal |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR0174597B1 (en) | 1995-06-29 | 1999-04-01 | 김광호 | Register signaling control method using database in exchange |
JP3363039B2 (en) * | 1996-08-29 | 2003-01-07 | ケイディーディーアイ株式会社 | Apparatus for detecting moving objects in moving images |
KR19990005282A (en) | 1997-06-30 | 1999-01-25 | 배순훈 | Hierarchical Motion Estimation Method Using Pattern Classification in Wavelet Transform Domain |
JPH1169356A (en) * | 1997-08-25 | 1999-03-09 | Mitsubishi Electric Corp | Dynamic image encoding system and dynamic image decoding system |
JPH11243551A (en) | 1997-12-25 | 1999-09-07 | Mitsubishi Electric Corp | Motion compensation device and dynamic image corder and its method |
JP4356140B2 (en) | 1999-05-20 | 2009-11-04 | 富士通株式会社 | Image motion vector measuring method and apparatus |
JP2001112009A (en) * | 1999-10-14 | 2001-04-20 | Oki Electric Ind Co Ltd | Moving picture encoder and decoder |
EP1440581B1 (en) | 2001-07-10 | 2014-10-08 | Entropic Communications, Inc. | Unit for and method of motion estimation, and image processing apparatus provided with such motion estimation unit |
-
2003
- 2003-07-31 KR KR20030053232A patent/KR100573696B1/en not_active IP Right Cessation
-
2004
- 2004-07-02 CN CNB2004100621395A patent/CN100372381C/en not_active Expired - Fee Related
- 2004-07-12 US US10/887,897 patent/US7555166B2/en not_active Expired - Fee Related
- 2004-07-23 JP JP2004216508A patent/JP4155952B2/en not_active Expired - Fee Related
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5786860A (en) * | 1994-07-30 | 1998-07-28 | Korean Advanced Institute Of Science And Technology | High speed block matching for bi-directional motion vector estimation |
US6430317B1 (en) * | 1997-12-31 | 2002-08-06 | Sarnoff Corporation | Method and apparatus for estimating motion using block features obtained from an M-ary pyramid |
US7336707B2 (en) * | 2003-06-06 | 2008-02-26 | Samsung Electronics Co., Ltd. | Method and apparatus for detecting improper area for motion compensation in video signal |
Cited By (40)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090059068A1 (en) * | 2005-09-30 | 2009-03-05 | Toshiharu Hanaoka | Image display device and method |
US9881535B2 (en) * | 2005-09-30 | 2018-01-30 | Sharp Kabushiki Kaisha | Image display device and method |
US7551232B2 (en) * | 2005-11-14 | 2009-06-23 | Lsi Corporation | Noise adaptive 3D composite noise reduction |
US20070109448A1 (en) * | 2005-11-14 | 2007-05-17 | Lsi Logic Corporation | Noise adaptive 3D composite noise reduction |
US8019179B2 (en) * | 2006-01-19 | 2011-09-13 | Qualcomm Incorporated | Hand jitter reduction for compensating for linear displacement |
US20070236579A1 (en) * | 2006-01-19 | 2007-10-11 | Jingqiang Li | Hand jitter reduction for compensating for linear displacement |
US8064722B1 (en) * | 2006-03-07 | 2011-11-22 | The United States Of America As Represented By The Secretary Of The Navy | Method and system for analyzing signal-vector data for pattern recognition from first order sensors |
US20070296855A1 (en) * | 2006-06-22 | 2007-12-27 | Yunwei Jia | Video processing using region-based statistical measurements |
US20080031338A1 (en) * | 2006-08-02 | 2008-02-07 | Kabushiki Kaisha Toshiba | Interpolation frame generating method and interpolation frame generating apparatus |
US8265158B2 (en) | 2007-12-20 | 2012-09-11 | Qualcomm Incorporated | Motion estimation with an adaptive search range |
US20090161763A1 (en) * | 2007-12-20 | 2009-06-25 | Francois Rossignol | Motion estimation with an adaptive search range |
US8649437B2 (en) | 2007-12-20 | 2014-02-11 | Qualcomm Incorporated | Image interpolation with halo reduction |
US20090161010A1 (en) * | 2007-12-20 | 2009-06-25 | Integrated Device Technology, Inc. | Image interpolation with halo reduction |
WO2009085232A1 (en) * | 2007-12-20 | 2009-07-09 | Integrated Device Technology, Inc. | Estimation of true motion vectors using an adaptive search range |
US20090161011A1 (en) * | 2007-12-21 | 2009-06-25 | Barak Hurwitz | Frame rate conversion method based on global motion estimation |
KR101407852B1 (en) * | 2008-07-16 | 2014-06-17 | 소니 일렉트로닉스 인코포레이티드 | Simple next search position selection for motion estimation iterative search |
US20100014001A1 (en) * | 2008-07-16 | 2010-01-21 | Sony Corporation, A Japanese Corporation | Simple next search position selection for motion estimation iterative search |
US8144766B2 (en) | 2008-07-16 | 2012-03-27 | Sony Corporation | Simple next search position selection for motion estimation iterative search |
WO2010008655A1 (en) * | 2008-07-16 | 2010-01-21 | Sony Corporation | Simple next search position selection for motion estimation iterative search |
US8411200B2 (en) * | 2009-12-11 | 2013-04-02 | Renesas Electronics Corporation | Video signal processing device, method, and non-transitory computer readable medium storing image processing program capable of producing an appropriate interpolation frame |
US20110141369A1 (en) * | 2009-12-11 | 2011-06-16 | Renesas Electronics Corporation | Video signal processing device, video signal processing method, and non-transitory computer readable medium storing image processing program |
US8488007B2 (en) | 2010-01-19 | 2013-07-16 | Sony Corporation | Method to estimate segmented motion |
US20110176013A1 (en) * | 2010-01-19 | 2011-07-21 | Sony Corporation | Method to estimate segmented motion |
US20110216831A1 (en) * | 2010-03-08 | 2011-09-08 | Francois Rossignol | Apparatus and method for motion vector filtering based on local image segmentation and lattice maps |
US8594199B2 (en) * | 2010-03-08 | 2013-11-26 | Qualcomm Incorporated | Apparatus and method for motion vector filtering based on local image segmentation and lattice maps |
US8285079B2 (en) | 2010-03-19 | 2012-10-09 | Sony Corporation | Method for highly accurate estimation of motion using phase correlation |
US20110229056A1 (en) * | 2010-03-19 | 2011-09-22 | Sony Corporation | Method for highly accurate estimation of motion using phase correlation |
US20110249747A1 (en) * | 2010-04-12 | 2011-10-13 | Canon Kabushiki Kaisha | Motion vector decision apparatus, motion vector decision method and computer readable storage medium |
US9113170B2 (en) * | 2010-04-12 | 2015-08-18 | Canon Kabushiki Kaisha | Motion vector decision apparatus, motion vector decision method and computer readable storage medium |
US8537283B2 (en) | 2010-04-15 | 2013-09-17 | Qualcomm Incorporated | High definition frame rate conversion |
US20120169890A1 (en) * | 2010-12-30 | 2012-07-05 | Stmicroelectronics Inc. | Motion estimation in imaging systems |
US8675080B2 (en) * | 2010-12-30 | 2014-03-18 | Stmicroelectronics, Inc. | Motion estimation in imaging systems |
US20150043785A1 (en) * | 2011-08-29 | 2015-02-12 | Veldandi Muninder | Method and Apparatus for Feature Computation and Object Detection Utilizing Temporal Redundancy between Video Frames |
US9508155B2 (en) * | 2011-08-29 | 2016-11-29 | Nokia Technologies Oy | Method and apparatus for feature computation and object detection utilizing temporal redundancy between video frames |
US8929452B2 (en) * | 2012-01-11 | 2015-01-06 | Panasonic Intellectual Property Management Co., Ltd. | Image processing apparatus, image capturing apparatus, and computer program |
US20130176487A1 (en) * | 2012-01-11 | 2013-07-11 | Panasonic Corporation | Image processing apparatus, image capturing apparatus, and computer program |
US8676866B2 (en) | 2012-03-19 | 2014-03-18 | Sap Ag | Computing canonical hierarchical schemas |
US8473474B1 (en) * | 2012-03-28 | 2013-06-25 | Sap Ag | Granularity-adaptive extraction of correlation structures in databases |
US20140072241A1 (en) * | 2012-09-12 | 2014-03-13 | Canon Kabushiki Kaisha | Image processing apparatus and image processing method |
US9292907B2 (en) * | 2012-09-12 | 2016-03-22 | Canon Kabushiki Kaisha | Image processing apparatus and image processing method |
Also Published As
Publication number | Publication date |
---|---|
US7555166B2 (en) | 2009-06-30 |
CN1581982A (en) | 2005-02-16 |
CN100372381C (en) | 2008-02-27 |
KR100573696B1 (en) | 2006-04-26 |
JP4155952B2 (en) | 2008-09-24 |
JP2005056410A (en) | 2005-03-03 |
KR20050014569A (en) | 2005-02-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7555166B2 (en) | Pattern analysis-based motion vector compensation apparatus and method | |
US8325812B2 (en) | Motion estimator and motion estimating method | |
US7680186B2 (en) | Apparatus for estimating motion considering correlation between blocks and method thereof | |
US6990148B2 (en) | Apparatus for and method of transforming scanning format | |
US6782054B2 (en) | Method and apparatus for motion vector estimation | |
US8315436B2 (en) | Robust camera pan vector estimation using iterative center of mass | |
US8199252B2 (en) | Image-processing method and device | |
US20030086498A1 (en) | Apparatus and method of converting frame and/or field rate using adaptive motion compensation | |
US7321626B2 (en) | System and method for predictive motion estimation using a global motion predictor | |
EP1734769B1 (en) | An encoder and encoding method for motion estimation | |
US20050201626A1 (en) | Global motion-compensated sequential-scanning method considering horizontal and vertical patterns | |
US20070140346A1 (en) | Frame interpolator, frame interpolation method and motion reliability evaluator | |
US20060193535A1 (en) | Image matching method and image interpolation method using the same | |
US7949205B2 (en) | Image processing unit with fall-back | |
US20080095399A1 (en) | Device and method for detecting occlusion area | |
US8045619B2 (en) | Motion estimation apparatus and method | |
KR20070079411A (en) | Method and apparatus for estimating motion vector based on block | |
US20040022320A1 (en) | Image matching device and method for motion pictures | |
KR20110022133A (en) | Move estimating method and image processing apparauts | |
US7881500B2 (en) | Motion estimation with video mode detection | |
US7327787B1 (en) | Method and apparatus for motion estimation | |
US20050025243A1 (en) | Motion type decision apparatus and method thereof | |
KR100289862B1 (en) | Global motion compensation on/off decision method using 2D translational motion vectors and regression | |
US20080107179A1 (en) | Global Motion Estimation | |
KR20090084311A (en) | Frame rate conversion method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SAMSUNG ELECTRONICS CO., LTD., KOREA, REPUBLIC OF Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LEE, SUNG-HEE;KWON, OHJAE;REEL/FRAME:015585/0707 Effective date: 20040607 |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Free format text: PAYER NUMBER DE-ASSIGNED (ORIGINAL EVENT CODE: RMPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
REMI | Maintenance fee reminder mailed | ||
LAPS | Lapse for failure to pay maintenance fees | ||
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20170630 |