GB2327008A - Detecting motion vectors by classifying search blocks according to patterns - Google Patents

Detecting motion vectors by classifying search blocks according to patterns Download PDF

Info

Publication number
GB2327008A
GB2327008A GB9812513A GB9812513A GB2327008A GB 2327008 A GB2327008 A GB 2327008A GB 9812513 A GB9812513 A GB 9812513A GB 9812513 A GB9812513 A GB 9812513A GB 2327008 A GB2327008 A GB 2327008A
Authority
GB
United Kingdom
Prior art keywords
search
block
blocks
current
subimage
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
GB9812513A
Other versions
GB9812513D0 (en
Inventor
Sung-Joo Park
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
WiniaDaewoo Co Ltd
Original Assignee
Daewoo Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Daewoo Electronics Co Ltd filed Critical Daewoo Electronics Co Ltd
Publication of GB9812513D0 publication Critical patent/GB9812513D0/en
Publication of GB2327008A publication Critical patent/GB2327008A/en
Withdrawn legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/14Picture signal circuitry for video frequency region
    • H04N5/144Movement detection
    • H04N5/145Movement estimation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • G06T7/223Analysis of motion using block-matching
    • G06T7/238Analysis of motion using block-matching using non-full search, e.g. three-step search
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • H04N19/53Multi-resolution motion estimation; Hierarchical motion estimation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20048Transform domain processing
    • G06T2207/20064Wavelet transform [DWT]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/63Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Image Processing (AREA)

Abstract

A method and apparatus for detecting motion vectors between a current and a previous frame in a video signal derives wavelet representations of the current and previous frames to thereby generate M layers and stores a current and previous subimage of a lowest frequency band for each layer, wherein each layer has a corresponding resolution and M is a positive integer. Then, a current subimage of a lowest frequency band in an Mth layer having a resolution of a lowest level is motion estimated with respect to a corresponding previous subimage by classifying search blocks in the current subimage and candidate blocks in the previous subimage into predetermined patterns and with these generating motion vectors for the current subimage of the lowest frequency band in the Mth layer. candidate blocks are selected in an (M-i)th layer by using motion vectors for the current subimage of the lowest frequency band in the (M-i+1)st layer, wherein i is an integer ranging from 1 to M-1. The patterns, 720, 750, indicate the comparative values of the pixels within each block.

Description

METHOD AND APPARATUS FOR DETECTING MOTION VECTORS OF A WAVELET TRANSFORMED VIDEO SIGNAL BY PATTERN CLASSIFICATION The present invention relates to a motion estimation method and apparatus; and, more particularly, to a method and apparatus for detecting motion vectors of a wavelet transformed video signal by pattern classification.
The discrete wavelet transform(DWT) technique has recently attracted a considerable amount of attention in the art of image processing due to its flexibility in representing nonstationary image signals and its ability to adapt to human visual characteristics. A wavelet representation provides a multi-resolution/multi-frequency expression of a signal localized in both time and frequency.
Such versatilities are desirable in image and video coding applications. Since natural image and video signals are nonstationary in nature and a wavelet transform decomposes a nonstationary signal into a set of multi-scaled wavelets where each component becomes relatively more stationary, such transform method makes it easier to encode such nonstationary signals. Also, coding schemes and parameters can be adapted to the statistical properties of each wavelet, and hence coding each stationary component is more efficient than coding the whole nonstationary signal. In addition, the wavelet representation matches well with the spatially-tuned, frequency modulated properties experienced in human vision as reported by the research in psychophysics and physiology.
In a typical wavelet decomposition technique(see, e.g., U.S. Patent No. 5,477,272 issued to Ya-Qin Zhang on Dec. 19, 1995), a video frame is decomposed into a plurality of layers with different resolutions, each subimage being in a same layer corresponding to each of different frequency bands.
Fig. 1 illustrates a conventional wavelet decomposition process wherein a current frame S1 is applied to a first wavelet decomposition block 110 and decomposed into subimages of layer 1, i.e., S2, W21, W22 and W23. Then, the subimage S2 is applied to a second wavelet decomposition block 120 and decomposed into subimages of layer 2, i.e., S4, W41, W42, W43.
Thereafter, the subimage S4 is applied to a third wavelet decomposition block 130 and decomposed into subimages of layer 3, i.e., S8, W81, W82, W83.
These subimages can be organized into a pyramid structure to provide a pictorial representation as shown in Fig. 2. The wavelet transformed current frame S1 has a resolution depth of 3 and consists of 10 subimages, with 3 subimages at each layer and one lowpass subimage. The subimage S4 is formed by combining the subimage S8 with the sub images W81 to W83 in the layer 3; the subimage S2 is formed by combining the subimage S4 with the sub images W41 to w43 in the layer 2; and the current frame S1 is formed by combining the subimage S2 with the subimages W21 to W23 in the layer 1.
Referring to Fig. 3A, there is depicted a conventional multi-resolution motion estimation(MRME) scheme. First, a current frame S1 is decomposed to thereby generate subimages S8' W81, W82, W83, W41, W42, W43, W21, W22 and W23, and a previous frame PS1 is also decomposed to yield subimages PS8, PW81, PW82, PW83, PW41, PW42, PW43, PW21, PW22 and PW23 wherein the previous frame PS1 and its subimages PS8, PW81, PW82, PW83, PW41, PW42, PW43 PW21, PW22 and PW23 are not shown, for the sake of simplicity.
Then, each of the subimages of the S1 is divided into a plurality of search blocks, wherein the sizes of search blocks within subimages of a same layer are identical. If the size of a search block within a subimage of a highest layer M is p x p, the size of a search block within a subimage of a layer m is p' 2Mm x p. 2M-m, M, p and m being positive integers, respectively, wherein typical values of M and p are 3 and 2, respectively.
Thereafter, each search block in each subimage is motion estimated with reference to a corresponding subimage of the PS1. For example, assuming that a search block 302 in the S8 of Fig. 3A is motion estimated by using a conventional block matching algorithm, a search region corresponding to the search block 302 in the S8 is formed in the PS8 and a plurality of candidate blocks are generated in the search region. Then, error values between the search block 302 in the S8 and the candidate blocks are calculated, wherein an error value is, e.g., a mean absolute error between a pixel value of the search block 302 in the S8 and a corresponding pixel value of a candidate block.
Among the calculated error values, a minimum error value is selected and a difference between the search block 302 in the S8 and an optimum candidate block 304 which yields the minimum error value is detected as a motion vector MVS8 of the search block 302 in the In motion estimating a search block 306 in the W a search region corresponding to the search block 306 in the W81 is formed in the PW81 based on the MVSa. Specifically, a location which is same as that of the search block 306 in the W81 is detected and the detected location is displaced by as much as the MVSB. The search region is formed around the displaced location and an optimum candidate block 308 is detected in the search region by motion estimating the search block 306 in the W81 in a same manner as that of the search block 302 in the S8. Search blocks in the W82 and the W83 are also motion estimated in a similar manner as that of the search block 306 in the W81.
In motion estimating a search block 310 in the W41, a search region corresponding to the search block 310 in the W is formed in the PW41 based on a scaled motion vector 2MVSB.
That is, a location which is same as that of the search block 310 in the W41 is detected and the detected location is displaced by as much as the 2MVSa. The search region is formed around the displaced location and an optimum candidate block 312 is detected in the search region by motion estimating the search block 310 in the W41 in a same manner as that of the search block 306 in the W81. Search blocks in the W42 and the W43 are also motion estimated in a similar manner as that of the search block 310 in the W 4.
In motion estimating a search block 314 in the W21, a search region corresponding to the search block 314 in the W is formed in the Pw; based on a scaled motion vector 4MVSa.
Specifically, a location which is same as that of the search block 314 in the W21 is detected and the detected location is displaced by as much as the 4MVSa. The search region is formed around the displaced location and an optimum candidate block 316 is detected in the search region by motion estimating the search block 314 in the W21 in a same manner as that of the search block 310 in the W41. Search blocks in the W22 and the W23 are also motion estimated in a similar manner as that of the search block 314 in the W Meanwhile, if an optimum candidate block 316 corresponding to the search block 314 in the W21 is detected as is shown in Fig. 3A, a displacement from the search block 314 in the W21 to an optimum candidate block 318 is MVW 1 Then, a difference between the 4MVS8 and the MEW21 is calculated and provided as a motion vector difference MVDW21 of the search block 314 of the W 2 In such a MRME scheme, motion vectors for all subimages are detected and transferred, thereby complicating the computational process. Thus, a motion estimation scheme only for subimages of low frequency bands is developed as shown in Fig. 3B. The new scheme is based on the concept that the subimage Ss contains a major portion of the total energy present in the current frame S1 although its size is only 1/64 of that of S1 and human vision is more perceptible to errors in lower frequency bands than those occurring in higher bands.
According to the new scheme depicted in Fig. 3B, a motion vector MV8 between a search block 318 in the S8 and a corresponding optimum candidate block 320 is scaled by 2 and used as an initial vector of a search block 322 in the S4 to thereby detect a motion vector MV4 and a motion vector difference MVD4, wherein the MVD4 is a difference between 2MV8 and the MV4; the motion vector MV4 between the search block 322 in the S4 and a corresponding candidate block 324 is scaled by 2 and used as an initial vector of a search block 326 in the Sz to thereby detect a motion vector MV2 and a motion vector difference MVD2, wherein the MVD2 is a difference between 2MV4 and the MV2; and the motion vector MV2 between the search block 326 in the S2 and a corresponding candidate block 328 is scaled by 2 and used as an initial vector of a search block 330 in the S1 to thereby detect a motion vector MV1 and a motion vector difference MVD1, wherein the MV1 is a displacement between the search block 330 and a corresponding candidate block 332 and the MVD1 is a difference between 2MV2 and the MV1.
By motion estimating only the subimages of low frequency bands, the computational process can be simplified. Since, however, a motion vector of a search block in the S8 is detected by block matching the search block with all candidate blocks generated in a corresponding search region, it is still needed to further simplify the computational process.
It is, therefore, a primary object of the invention to provide a method and apparatus for detecting motion vectors of a wavelet transformed video signal by pattern classification.
In accordance with one aspect of the present invention, there is provided a method for detecting motion vectors between a current and a previous frames in a video signal, comprising the steps of: (a) deriving a wavelet representation of the current frame to thereby generate M layers, M being a positive integer and storing a current subimage of a lowest frequency band for each layer, wherein each layer has a corresponding resolution level and a current subimage of a lowest frequency band for a 1st layer corresponds to the current frame; (b) deriving a wavelet representation of the previous frame to thereby generate M layers and storing a previous subimage of a lowest frequency band for each layer, wherein each layer has a corresponding resolution level and a previous subimage of a lowest frequency band for a 1st layer corresponds to the previous frame; (c) motion estimating a current subimage of a lowest frequency band in an Mth layer having a resolution of a lowest level with respect to a corresponding previous subimage by classifying patterns of search blocks in the current sub image and candidate blocks in the previous subimage to thereby generate motion vectors for the current subimage of the lowest frequency band in the Mth layer; and (d) motion estimating a current subimage of a lowest frequency band in an (M-i)th layer with respect to a corresponding previous subimage by using scaled motion vectors for the current subimage of the lowest frequency band in the (M-i+l)st layer to thereby generate motion vectors for the current subimage of the lowest frequency band for the 1st layer, wherein i is an integer ranging from 1 to M-l.
In accordance with another aspect of the present invention, there is provided an apparatus for detecting motion vectors between a current and a reference frames of a video signal, comprising: means for wavelet transforming the current frame to thereby produce an M number of hierarchical current images with M being greater than 1, wherein a current image of a highest hierarchy corresponds to the current frame, a current image of a higher hierarchy has a higher resolution than a current image of a lower hierarchy, and the respective current images have an identical number of search blocks, each search block of the lower hierarchy corresponding to a search block of the higher hierarchy; means for wavelet transforming the reference frame to provide an M number of hierarchical reference images, wherein a reference image of a highest hierarchy corresponds to the reference frame, a reference subimage of a hierarchy including candidate blocks corresponding to each search block of the hierarchy and sizes of each search block of the hierarchy and the candidate blocks corresponding thereto being identical to each other; a first classifying means for categorizing a search block of the lowest hierarchy into one of predetermined patterns; a second classifying means for categorizing each of candidate blocks corresponding to the search block into one of the predetermined patterns; means for finding an optimum candidate block based on patterns of the search block and the candidate blocks to thereby detect a displacement vector for the search block, the displacement vector representing the displacement between the search block and the optimum candidate block; and means for obtaining a displacement vector of a search block of the highest hierarchy based on the detected displacement vector and setting the obtained displacement vector as a motion vector of the search block of the highest hierarchy.
The above and other objects and features of the present invention will become apparent from the following description of preferred embodiments given in conjunction with the accompanying drawings, in which: Fig. 1 represents a conventional wavelet decomposition process; Fig. 2 provides a pictorial representation of a pyramid structure of wavelet transformed subimages; Fig. 3A presents a conventional multi-resolution motion estimation(MRME) scheme; Fig. 3B shows a motion estimation scheme only for subimages of low frequency bands; Fig. 4 is a block diagram of an encoding system in accordance with a preferred embodiment of the present invention; Fig. 5 illustrates a detailed structure of a motion estimation block shown in Fig. 4; Fig. 6 depicts a motion estimation scheme of a pattern block matching unit shown in Fig. 5; and Fig. 7 describes a block classifying process.
Referring to Fig. 4, there is provided an apparatus 400 for selectively detecting motion vectors of a wavelet transformed video signal in accordance with a preferred embodiment of the present invention.
A current frame S1 is applied to a current wavelet decomposition block 410 and a motion estimation block 420 via a line L402. The current wavelet decomposition block 410 decomposes the S1 into subimages as is shown in Figs. 1 and 2, and provides current subimages of low frequency bands, i.e., S2, S4 and S8 to the motion estimation block 420 via lines L412, L414 and L416, respectively.
Meanwhile, a previous frame PS1 is applied from a memory 490 to a previous wavelet decomposition block 430, the motion estimation block 420 and a motion compensation block 440 via a line L492. Previous subimages of low frequency bands, i.e., PS2, PS4 and PS8 are respectively provided from the previous wavelet decomposition block 430 to the motion estimation block 420 via lines L432, L434 and L436, wherein the PS2, the PS4 and the PS8 are obtained by decomposing the PS1 from the memory 490.
The motion estimation block 420 detects a motion vector for each search block in the S1 and provides the same to the motion compensation block 440 and a transmitter(not shown).
The detailed motion estimation process of the motion estimation block 420 will be further described with reference to Figs. 5, 6 and 7.
The motion compensation block 440 is provided with the motion vector from the motion estimation block 420 and a corresponding optimum candidate block in the PS1 from the memory 490 via the line L492. The motion compensation block 440 motion compensates the optimum candidate block by using the motion vector and provides a motion compensated optimum candidate block to a subtraction block 450 and an addition block 480.
The subtraction block 450 subtracts the motion compensated optimum candidate block from the search block in the S1 to thereby provide an error signal to an discrete cosine transform(DCT) and quantization(Q) block 460. The DCT & Q block 460 discrete cosine transforms and quantizes the error signal and provides a quantized discrete cosine transformed coefficient set to a variable length coding(VLC) block 465 and an inverse discrete cosine transform(IDCT) and inverse quantization(IQ) block 470. The VLC block 465 variable length codes the quantized discrete cosine transformed coefficient set to thereby provide the variable length coded signal to the transmitter. The IDCT & IQ block 470 inverse discrete cosine transforms and inverse quantizes the quantized discrete cosine transformed coefficient set and sends the restored error signal to the addition block 480.
The addition block 480 adds the restored error signal to the motion compensated optimum candidate block to thereby generate a reconstructed search block in the S1. Reconstructed search blocks of the S1 are stored at the memory 480 as a previous frame for a next frame.
Referring to Fig. 5, there is illustrated a detailed block diagram of the motion estimation block 420 shown in Fig.
4.
A pattern block matching unit 422 is provided with the via the line L416 and the PS8 via the line L436 to thereby motion estimate each search block in the S8 by classifying patterns of pixel value distribution. The detailed motion estimation process of the pattern block matching unit 422 is illustrated with reference to Fig. 6.
The S8 on the line L416 is provided to a search block formation unit 422-1. The search block formation unit 422-1 divides the S8 into a plurality of search blocks of a predetermined size p x p, e.g., 2 x 2 pixels, and provides each search block to a search block pattern classification unit 422-2.
The search block pattern classification unit 422-2 classifies a pattern of each search block of the S8, as is shown in Fig. 7. Since the size of the S8 is 12 x 8 pixels and the size of a search block of the S8 is 2 x 2 pixels, the number of the search blocks is 24. For example, a search block 710 of the S8 has 4 pixels, with an upper-left pixel having a value of 92, an upper-right 124, a lower left 115 and a lower right 109. The search block pattern classification unit 422-2 compares 4 pixel values with each other and ranks the pixels. Then, the pixel values of the search block 710 of the S8 are ranked to thereby generate a ranked search block 720 of the S8 with an upper-left pixel having a value of 1, an upper-right 4, a lower-left 3 and a lower-right 2 and a pattern to which the search block 710 of the S8 belongs. If more than one pixel has a same pixel value, the pixels are ranked same.
Search block information and pattern information for each search block of the S8 are provided to a first block matching unit 422-6, wherein the search block information represents pixel values of each search block of the S8 and the pattern information indicates which pattern each search block of the S8 belongs to.
Meanwhile, the PS8 is provided to a candidate block formation unit 422-3 of Fig. 6 via the line L436. The candidate block formation unit 422-3 forms a multiplicity of candidate blocks in PS8. Referring to Fig. 7, 77 candidate blocks of the PS8 of a same size as that of the search block of the Sg' e.g., 2 x 2 pixels size, are formed in the PS a since the size of the PS8 is 12 x 8 pixels. Candidate block information of the PS8 are provided to a candidate block pattern classification unit 422-4, wherein the candidate block information of the PS8 represents pixel values of each candidate block of the PS8.
The candidate block pattern classification unit 422-4 classifies a pattern of each candidate block of the PS8, as is shown in Fig. 7. For example, a candidate block 740 of the PS8 has 4 pixels, with an upper-left pixel having a value of 120, an upper-right 112, a lower left 108 and a lower right 98. The candidate block pattern classification unit 422-4 compares 4 pixel values with each other and ranks the pixels.
Then, the pixel values of the candidate block 740 of the PS8 are ranked to thereby generate a ranked candidate block 750 of the PS8 with an upper-left pixel having a value of 4, an upper-right 3, a lower-left 2 and a lower-right 1 and a pattern to which the candidate block 740 of the PS8 belongs.
If more than one pixel has a same pixel value, the pixels are ranked same.
The candidate block information and pattern information for each candidate block of the PS8 are provided to a search region formation unit 422-5, wherein the pattern information indicates to which pattern each candidate block of the PS8 belongs.
The search region formation unit 422-5 forms a plurality of search regions of the PS8, each of the search regions of the PS8 corresponding to one of the search blocks of the Back to the above example of Fig. 7, a location in the PS8 which is same as that of the search block 710 of the S8 is detected and a search region 730 of the PS8 is formed around the location. The candidate block information and the pattern information for each candidate block of the PS8 and search region information are stored at the search region formation unit 422-5, wherein the search region information indicates to which search block of the S8 each search region of the PS8 corresponds.
The first block matching unit 422-6 motion estimates each search block of the S8. When motion estimating a search block of the Sg, the first block matching unit 422-6 extracts candidate blocks of the PS8 which are in a search region of the PS8 corresponding to the search block of the S8 and whose patterns are same as that of the search block of the S8 from the search region formation unit 422-5. If there is no candidate block of PS8 whose pattern is same as that of the search block of S8, all the candidate blocks of the PS8, which are in the corresponding search region of PS8, are extracted from the search region formation unit 422-5.
Thereafter, error values between each of the extracted candidate blocks of the PS8 and the search block of the S8 are calculated, wherein an error value is, e.g., a mean absolute error between pixel values of the search block of the S8 and corresponding pixel values of an extracted candidate block of the PS8. Among the extracted candidate blocks of the PS8, an extracted candidate block of the PS8 that yields a minimum error value is determined as an optimum candidate block of the PS8 and a displacement from the optimum candidate block of the PS8 to the search block of the S8 is determined as a motion vector MV8 of the search block of the S8. The motion vector MV8 of the search block of the S8 determined in the above steps is provided to a second block matching unit 424 of Fig.
5.
The second block matching unit 424 is provided with the S4 on the line L414, the PS4 on the line L434 and the motion vectors MVa'S of the search blocks in the S8 from the first block matching unit 422-6 in the pattern block matching unit 422. The S4 is divided into a plurality of search blocks of a predetermined size 2p x 2p, e.g., 4 x 4 pixels. Then, a search region of the PS4 corresponding to a search block of the S4 is formed in the PS4 based on a scaled motion vector, e.g., 2MV8 of a corresponding search block of the In detail, a location which is same as that of the search block of the S4 is detected in the PS4 and the detected location is displaced by as much as 2MV8 to thereby generate a displaced location. Then, a search region of the PS4 is formed around the displaced location and the search block of the S4 is motion estimated in the search region. In the search region of the S4, a displacement which yields a minimum error is detected and determined as a motion vector MV4 of the search block of the S4. The motion vector MV4 of the search block of the S4 determined in the above steps is provided to a third block matching unit 426.
At the third block matching unit 426, a motion vector MV2 of a search block of the S2 is detected by using the S2 on the line L412 and PS2 on the line L432 based on the 2MV4 and is provided to a fourth block matching unit 428. And at the fourth block matching unit 428, a motion vector MV1 of a search block of the S1 is detected by using the S1 on the line L402 and PS1 on the line L492 based on the 2MV2. A process to detect the MV2 and the MV1 is similar to the process to detect the MV4, wherein the third block matching unit 426 and a fourth block matching unit 428 play the role of the second block matching unit 424.
The motion vector MV1 of the search block of the S1 which is detected through the above process is provided to the motion compensation block 440 and the transmitter.
In accordance with the present invention, the computational process is much simplified since a motion estimation for a search block of the S8 is executed between the search block of the S8 and candidate blocks of the PS8 which have a same pattern as that of the search block.
While the present invention has been described with respect to certain preferred embodiments only, other modifications and variations may be made without departing from the scope of the present invention as set forth in the following claims.

Claims (21)

Claims:
1. A method for detecting motion vectors between a current and a previous frames in a video signal, comprising the steps of: (a) deriving a wavelet representation of the current frame to thereby generate M layers, M being a positive integer and storing a current sub image of a lowest frequency band for each layer, wherein each layer has a corresponding resolution level and a current subimage of a lowest frequency band for a 1st layer corresponds to the current frame; (b) deriving a wavelet representation of the previous frame to thereby generate M layers and storing a previous subimage of a lowest frequency band for each layer, wherein each layer has a corresponding resolution level and a previous subimage of a lowest frequency band for a 1st layer corresponds to the previous frame; (c) motion estimating a current subimage of a lowest frequency band in an Mth layer having a resolution of a lowest level with respect to a corresponding previous subimage by classifying patterns of search blocks in the current subimage and candidate blocks in the previous subimage to thereby generate motion vectors for the current subimage of the lowest frequency band in the Mth layer; and (d) motion estimating a current subimage of a lowest frequency band in an (M-i)th layer with respect to a corresponding previous subimage by using scaled motion vectors for the current sub image of the lowest frequency band in the (M-i+l)st layer to thereby generate motion vectors for the current subimage of the lowest frequency band for the 1st layer, wherein i is an integer ranging from 1 to M-l.
2. The method as recited in claim 1, wherein the step (c) includes the steps of: (cl) dividing the current subimage of the lowest frequency band in the Mth layer into a multiplicity of search blocks, the size of the search blocks being p x p pixels, p being a positive integer; (c2) determining a pattern of each search block; (c3) generating a plural number of candidate blocks in the corresponding previous subimage, the size of the candidate blocks being same as that of the search blocks; (c4) determining a pattern of each candidate block; and (c5) forming a multiplicity of search regions in the corresponding previous subimage, each search region corresponding to one search block.
3. The method as recited in claim 2, wherein the step (c) further includes the steps of: (c6) selecting, for each of the search blocks, candidate blocks which are included in a search region corresponding to said each of the search blocks and have a same pattern as that of said each of the search blocks; (c7) calculating error values between each of the search blocks and corresponding selected candidate blocks to thereby select an optimum candidate block for each search block, wherein the optimum candidate block yields a minimum error value; and (c8) providing displacements from the search blocks to the corresponding optimum candidate blocks as the motion vectors of the search blocks in the current subimage.
4. The method as recited in claim 3, wherein the step (c2) includes the steps of: (c21) grouping pixels in each search block into 4 groups; (c22) generating a representative value of each group for each search block by using pixel values in each group; and (c23) determining a pattern of each search block by using 4 representative values thereof.
5. The method as recited in claim 4, wherein the representative value of each group for each search block indicates an order of an average value of the pixel values in each group of each search block.
6. The method as recited in claim 5, wherein the step (c4) includes the steps of: (c41) grouping pixels in each candidate block into 4 groups; (c42) generating a representative value of each group for each candidate block by using pixel values in each group; and (c43) determining a pattern of each candidate block by using 4 representative values thereof.
7. The method as recited in claim 6, wherein the representative value of each group for each candidate block indicates an order of an average value of the pixel values in each group of each candidate block.
8. The method as recited in claim 7, wherein the step (d) includes the steps of: (dl) dividing the current subimage of the lowest frequency band in the (M-i)th layer into a multiplicity of search blocks, the size of the search blocks being p2i x p2 pixels and i being 1; (d2) forming a multiplicity of search regions in the corresponding previous subimage based on scaled motion vectors for the current sub image of the lowest frequency band in the (M-i+l)st layer, each search region corresponding to one search block; (d3) generating a plural number of candidate blocks in each search region, the size of the candidate blocks being same as that of the search blocks; (d4) calculating error values between each of the search blocks and candidate blocks in a corresponding search region to thereby select an optimum candidate block for each search block, wherein the optimum candidate block yields a minimum error value; (d5) providing displacements from the search blocks to the corresponding optimum candidate blocks as motion vectors of the search blocks in the current subimage; (d6) increasing i by 1 and repeating the steps (dl) to (d5) until i becomes M-l; and (d7) providing motion vectors of search blocks in the current sub image of the lowest frequency band for the 1st layer when i becomes M-l.
9. The method as recited in claim 8, wherein the step (d2) includes the steps of: (d21) detecting a location, which is same as that of a search block, in the previous subimage of the lowest frequency band in the (M-i)th layer, for each search block; (d22) displacing the detected location by as much as a scaled motion vector of a corresponding search block in the current subimage of the lowest frequency band in the (M-i+l)st layer, for each search block, wherein the scaled motion vector is obtained by multiplying a motion vector of the corresponding search block in the current sub image of the lowest frequency band in the (M-i+l)st layer by 2; and (d23) forming a search region around the displaced location, for each search block.
10. The method as recited in claim 9, wherein the value of M is 4 and the value of p is 2.
11. An apparatus for detecting motion vectors between a current and a reference frames of a video signal, comprising: means for wavelet transforming the current frame to thereby produce an M number of hierarchical current images with M being greater than 1, wherein a current image of a highest hierarchy corresponds to the current frame, a current image of a higher hierarchy has a higher resolution than a current image of a lower hierarchy, and the respective current images have an identical number of search blocks, each search block of the lower hierarchy corresponding to a search block of the higher hierarchy; means for wavelet transforming the reference frame to provide an M number of hierarchical reference images, wherein a reference image of a highest hierarchy corresponds to the reference frame, a reference subimage of a hierarchy including candidate blocks corresponding to each search block of the hierarchy and sizes of each search block of the hierarchy and the candidate blocks corresponding thereto being identical to each other; a first classifying means for categorizing a search block of the lowest hierarchy into one of predetermined patterns; a second classifying means for categorizing each of candidate blocks corresponding to the search block into one of the predetermined patterns; means for finding an optimum candidate block based on patterns of the search block and the candidate blocks to thereby detect a displacement vector for the search block, the displacement vector representing the displacement between the search block and the optimum candidate block; and means for obtaining a displacement vector of a search block of the highest hierarchy based on the detected displacement vector and setting the obtained displacement vector as a motion vector of the search block of the highest hierarchy.
12. The apparatus according to claim 11, wherein sizes of search blocks of a hierarchy are identical to each other.
13. The apparatus according to claim 12, wherein a size of each search block of the lowest hierarchy is P x Q pixels, P and Q being greater than 1.
14. The apparatus according to claim 13, wherein each of the classifying means includes means for determining a pattern of a block based on values of P x Q pixels therein.
15. The apparatus according to claim 14, wherein the means for determining the pattern of the block includes means for replacing each of the P x Q pixels in the block with a number representing an order of magnitudes of the P x Q pixel values to thereby determine the pattern of the block.
16. The apparatus according to claim 15, wherein the means for finding the optimum candidate block includes: means for selecting candidate blocks having a pattern identical to that of the search block; and means for deciding the optimum candidate block among the selected candidate blocks.
17. The apparatus according to claim 16, wherein the means for obtaining the displacement vector includes: means for motion estimating a search block of a hierarchy with respect to its corresponding candidate blocks based on a displacement vector obtained for a search block of a next lower hierarchy in order to find an optimum candidate block of the hierarchy; means for deciding a displacement vector for the search block of the hierarchy, the displacement vector for the hierarchy corresponding to the displacement between the optimum candidate block and the search block of the hierarchy; and means for repeatedly motion estimating a search block of a next higher hierarchy and deciding a displacement vector therefor until the displacement vector of the search block of the highest hierarchy is obtained.
18. The apparatus according to claim 17, wherein a size of a search block of a hierarchy is 2 > < x 2 times larger than a size of a search block of a next lower hierarchy.
19. The apparatus according to claim 18, wherein M, P and Q are 4, 2 and 2, respectively.
20. An apparatus for detecting motion vectors constructed and arranged substantially as herein described with reference to or as shown in Figures 4, 5 and 6 of the accompanying drawings.
21. A method for detecting motion vectors substantially as herein described, with reference to or as shown in Figures 4, 5 and 6 of the accompanying drawings.
GB9812513A 1997-06-30 1998-06-10 Detecting motion vectors by classifying search blocks according to patterns Withdrawn GB2327008A (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1019970029477A KR19990005282A (en) 1997-06-30 1997-06-30 Hierarchical Motion Estimation Method Using Pattern Classification in Wavelet Transform Domain

Publications (2)

Publication Number Publication Date
GB9812513D0 GB9812513D0 (en) 1998-08-05
GB2327008A true GB2327008A (en) 1999-01-06

Family

ID=19512453

Family Applications (1)

Application Number Title Priority Date Filing Date
GB9812513A Withdrawn GB2327008A (en) 1997-06-30 1998-06-10 Detecting motion vectors by classifying search blocks according to patterns

Country Status (3)

Country Link
JP (1) JPH1175203A (en)
KR (1) KR19990005282A (en)
GB (1) GB2327008A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180232861A1 (en) * 2017-02-10 2018-08-16 Fujifilm Corporation Image processing apparatus, image processing method, and image processing program

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100573696B1 (en) 2003-07-31 2006-04-26 삼성전자주식회사 Apparatus and method for correction motion vector
KR100815977B1 (en) * 2005-04-20 2008-03-24 주식회사 메디슨 Method and system for estimating motion from continuous images
US7894527B2 (en) * 2005-09-16 2011-02-22 Sony Corporation Multi-stage linked process for adaptive motion vector sampling in video compression
KR101678654B1 (en) * 2010-06-11 2016-11-22 에스케이 텔레콤주식회사 Method and Apparatus of adaptive motion vector predictors selection for competition-based motion vector coding, and Recording Medium therefor
KR101241508B1 (en) * 2011-04-01 2013-03-11 한양대학교 산학협력단 Methods of motion vector estimation using classfier and apparatuses for using the same

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0643539A2 (en) * 1993-09-09 1995-03-15 Sony Corporation Motion vector detection apparatus and method
US5477272A (en) * 1993-07-22 1995-12-19 Gte Laboratories Incorporated Variable-block size multi-resolution motion estimation scheme for pyramid coding

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5477272A (en) * 1993-07-22 1995-12-19 Gte Laboratories Incorporated Variable-block size multi-resolution motion estimation scheme for pyramid coding
EP0643539A2 (en) * 1993-09-09 1995-03-15 Sony Corporation Motion vector detection apparatus and method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180232861A1 (en) * 2017-02-10 2018-08-16 Fujifilm Corporation Image processing apparatus, image processing method, and image processing program
US10672108B2 (en) * 2017-02-10 2020-06-02 Fujifilm Corporation Image processing apparatus, image processing method, and image processing program

Also Published As

Publication number Publication date
GB9812513D0 (en) 1998-08-05
KR19990005282A (en) 1999-01-25
JPH1175203A (en) 1999-03-16

Similar Documents

Publication Publication Date Title
US6219383B1 (en) Method and apparatus for selectively detecting motion vectors of a wavelet transformed video signal
US5510841A (en) Encoding apparatus in which flat subblocks are identified in blocks of a motion signal prior to coding the blocks and complementary decoding apparatus
US7245659B2 (en) Image encoding method and apparatus, image decoding method and apparatus, and image processing system
US5748247A (en) Refinement of block motion vectors to achieve a dense motion field
US6208692B1 (en) Apparatus and method for performing scalable hierarchical motion estimation
US6560371B1 (en) Apparatus and method for employing M-ary pyramids with N-scale tiling
KR100461210B1 (en) Recorded and readable media containing segmented image coding methods and systems and corresponding decoding methods and systems, and coded signals and computer readable storage media storing encoded signals
CN1110204C (en) Apparatus for estimating motion vactors for feature points of video signal
US5477272A (en) Variable-block size multi-resolution motion estimation scheme for pyramid coding
JP2823770B2 (en) Multilevel signal encoding method
EP0720377A2 (en) Method for detecting motion vectors for use in a segmentation-based coding system
EP1523196A2 (en) Encoding system of motion image containing arbitrary object
US20010046326A1 (en) Video encoding method using a wavelet transform
KR20010013198A (en) Method and apparatus for performing hierarchical motion estimation using nonlinear pyramid
US5923777A (en) Method for irregular triangle mesh representation of an image based on adaptive control point removal
GB2327008A (en) Detecting motion vectors by classifying search blocks according to patterns
JPH0955943A (en) Video signal encoding device
US5808685A (en) Block matching motion estimation apparatus employing a weight function
GB2322250A (en) Adaptive contour encoding of a video signal
US6249548B1 (en) Motion vector processing
EP0720384B1 (en) Block matching motion estimation apparatus employing a weight function
KR100440567B1 (en) A method for forming binary plane for motion search and a motion estimating apparatus using the same
Earl et al. Spread transform watermarking for video sources
JP3697727B2 (en) Quantization apparatus and quantization method
CN1222283A (en) Video encoder and decoder using motion-based segmentation and merging

Legal Events

Date Code Title Description
WAP Application withdrawn, taken to be withdrawn or refused ** after publication under section 16(1)