GB2378345A - Method for scanning a reference macroblock window in a search area - Google Patents

Method for scanning a reference macroblock window in a search area Download PDF

Info

Publication number
GB2378345A
GB2378345A GB0213247A GB0213247A GB2378345A GB 2378345 A GB2378345 A GB 2378345A GB 0213247 A GB0213247 A GB 0213247A GB 0213247 A GB0213247 A GB 0213247A GB 2378345 A GB2378345 A GB 2378345A
Authority
GB
United Kingdom
Prior art keywords
macroblock
current
storage element
reference macroblock
image processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
GB0213247A
Other versions
GB0213247D0 (en
GB2378345B (en
Inventor
Jin-Hyun Cho
Hyung-Lae Roh
Byeung-Woo Jeon
Yun-Tae Lee
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US10/112,011 external-priority patent/US20030012281A1/en
Application filed by Samsung Electronics Co Ltd filed Critical Samsung Electronics Co Ltd
Publication of GB0213247D0 publication Critical patent/GB0213247D0/en
Publication of GB2378345A publication Critical patent/GB2378345A/en
Application granted granted Critical
Publication of GB2378345B publication Critical patent/GB2378345B/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • H04N19/55Motion estimation with spatial constraints, e.g. at image or region borders
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/43Hardware specially adapted for motion estimation or compensation
    • H04N19/433Hardware specially adapted for motion estimation or compensation characterised by techniques for memory access
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/423Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/43Hardware specially adapted for motion estimation or compensation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51Motion estimation or motion compensation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

A motion estimation technique compares a current macroblock with different reference macroblocks in a reference frame search area. A motion vector for the current macroblock is derived from the reference macroblock most closely matching the current macroblock. To reduce the number of instructions required to load new reference macroblocks, overlapping portions between reference macroblocks are reused and only nonoverlapping portions are loaded into a memory storage device.

Description

<Desc/Clms Page number 1>
MOTION ESTIMATION APPARATUS AND METHOD FOR SCANNING A REFERENCE MACROBLOCK WINDOW IN A SEARCH AREA BACKGROUND This application relies for priority upon Korean Patent Application No. 2001-40904, filed on July 9,2001, the contents of which are herein incorporated by reference in their entirety.
Video encoders generate bit streams that comply with International standards for video compression, such as H. 261, H. 263, MPEG-1, MPEG-2, MPEG-4, MPEG-7, and MPEG-21.
These standards are widely applied in the fields of data storage, Internet based image service, entertainment, digital broadcasting, portable video terminals, etc.
Video compression standards use motion estimation where a current frame is divided into a plurality of macroblocks (MBs). Dissimilarities are computed between a current MB and other reference MBs existing in a search area of a reference frame. The reference MB in the search area most similar to the current MB is referred to as the"matching block"and is selected.
A motion vector is encoded for the current MB that indicates a phase difference between the current MB and the matching block. The phase difference refers to the location difference between the current MB and the matching block. Since only the motion vector for the current MB is transmitted, a smaller amount of data has to be transmitted or stored.
The relationship between the current MB and a search area is shown in FIG. 1.
According to a Quarter Common Intermediate Format (QCIF), one frame consists of 176 x 144 pixels, a current frame 2 consists of 99 current MBs, and each current MB 10 consists of 16 x
<Desc/Clms Page number 2>
16 pixels. A motion vector is computed tor the current MB 10 111 the reference trame 4. A search area 12 in the reference frame 4 includes 48 x 48 pixels.
In the search area 12, a 16 x 16 reference MB that is most similar to the current MB 10 is identified as the matching block. The differences between the current MB and the reference MBs can be computed by a variety of different methods. For example by using the Mean of the Absolute Difference (MAD), the Mean of the Absolute Error (MAE), or the Sum of the Absolute Difference (SAD). The SAD is most popular because it only requires subtraction and accumulation operations.
FIG. 2 shows a basic full search in which each pixel 10~1 and 14~1 are loaded into 32-bit registers 15 and 17, respectively. The SAD is then computed using an Arithmetic Logic Unit (ALU) 30. Both the current MB 10 and the reference MB 14a are stored in a memory and loaded into the 32-bit registers 15 and 17 pixel by pixel before being compared by the ALU 30.
Reference MBs 14a, 14b, 14c,... etc. existing in the search area 12 are compared with the current MB 10 on a pixel by pixel basis.
This simple ideal estimation method provides high accuracy. However, the transmission rate is restricted because there are so many computations. This method is also unsuitable for real-time encoding with some general purpose Central Processing Units (CPUs) limited processing capacity, such as some CPUs used in hand held Personal Computers (PCs).
A fast search method algorithm (not shown) is used to compute the SAD by comparing a current MB with only a limited number of the reference MBs in the search area. This fast search algorithm can dramatically reduce the number of computations compared to the full search method described above. However, the fast search algorithm has reduced picture
<Desc/Clms Page number 3>
quality.
A quick computation of the SAD has been developed using a full search method. The SAD for a plurality of pixels is computed at the same time using a Single Instruction Multiple Data (SIMD) method. This reduced number of operations improves the transmission rate.
FIG. 3 illustrates the computation of the SAD using a SIMD device. Eight pixels 10~8 and 14~8 for the current MB 10 and reference MB 14a, respectively, are loaded into 64-bit registers 16 and 18, respectively. The SIMD machine 20 computes SAD for eight pixels loaded into each of the 64-bit registers 16 and 18 at the same time. Unlike a typical full search algorithm in which the SAD is separately computed for each pixel, a simultaneous parallel computation of the SAD for a plurality of pixels is achieved using the SIMD technique.
The amount of computation varies depending on the direction the next MB is shifted in the search area 12. As shown in FIG. 3, whenever a next MB is selected by horizontal shifting, 8 pixels in both the current MB 10 and the reference MB 14 must be accessed from memory and loaded into the registers 16 and 18. This large number of memory accesses increases the amount of time required for deriving motion vectors and increases power consumption.
These conventional motion estimation methods are unsuitable in mobile environments because of the large number of memory accesses and associated large power consumption. The present invention addresses this and other problems associated with the prior art.
SUMMARY OF THE INVENTION A motion estimation technique compares a current macroblock with different reference macroblocks in a reference frame search area. A motion vector for the current macroblock is
<Desc/Clms Page number 4>
derived from the reference macro block most closely matching the current macroblock To reduce the number of instructions required to load new reference macroblocks, overlapping portions between reference macroblocks are reused and only nonoverlapping portions are loaded into a memory storage device.
The foregoing and other objects, features and advantages of the invention will become more readily apparent from the following detailed description of a preferred embodiment of the invention which proceeds with reference to the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a prior art diagram showing how a motion vector is derived.
FIG. 2 is a prior art diagram illustrating a conventional method for performing a motion vector search using Sum of the Absolute Difference (SAD) using full search method.
FIG. 3 is a prior art diagram showing a conventional method for performing a motion vector search using a Single Instruction Multiple Data (SIMD) method.
FIG. 4 is a block diagram of a system for performing motion estimation according to the present invention.
FIG. 5 is a diagram of a decimation filter.
FIG. 6 is a diagram showing a current macroblock and a corresponding search area after decimation.
FIG. 7 is a diagram showing how two groups of registers are used according to the invention.
FIG. 8 shows how a reference macroblock is shifted in a search area according to the
<Desc/Clms Page number 5>
mention.
FIG. 9 is a flowchart showing how motion vectors are identified according to the invention.
FIGS. 10A-10D are charts comparing instruction counts for different motion estimation techniques.
FIG. 11 shows other differences between conventional motion estimation methods and motion estimation according to the present invention.
FIG. 12 compares a vertical scanning technique according to the invention with other scanning techniques and shows the difference in memory access.
FIG. 13 shows conceptually a part of the dissimilarity computing unit 110 of FIG. 4.
DETAILED DESCRIPTION OF THE INVENTION The present invention provides efficient motion estimation that reduces memory accesses by reusing common registers when scanning reference MBs in a search area.
FIG. 4 is a block diagram of the preferred embodiment of a motion estimation system according to the present invention. The motion estimation system includes a current frame (C/F) 100, a first register group 102, a dissimilarity computing unit 110, a search area (S/A) 104, a second register group 106, and a controller 108. The first and second register groups 102 and 106 store pixels for one macroblock (MB) of the current frame 100 and one macroblock of the search area 104, respectively. In one example, the size of one MB is 16 x 16 pixels. Each of the first and second register groups 102 and 106 can store an array of 16 x 16 pixels. The controller 108 may be constructed by software or hardware.
<Desc/Clms Page number 6>
FIG. 5 shows a pre-process step carried out using 4 : 1 decunation filters A If 1 decimation filer is used on the current frame 100 (FIG. 4) to reduce required hardware resources. The current frame is represented by input frame 130 in FIG. 5. Frame 130 is divided into four decimation frames a, b, c and d by four 4: 1 decimation filters 126a, 126b, 126c and 126d, and stored in a frame memory 128. A video signal output from a charge coupled image capture device (CCD) 120 is converted into digital signals through an Analog-to-Digital Converter (ADC) 122. The signal output from the ADC 122 is a RGB signal.
A pre-processor 124 converts the RGB signal to a YCbCr signal. In one embodiment, only the Y signal is subjected to decimation by the decimation filter 126.
The decimation filter 126a is for pixels a in the input frame 130, the decimation filter 126b is for pixels b, the decimation filter 126c is for pixels c, and the decimation filter 126d is for pixels d. After the decimation, decimated frames a, b, c, and d are stored in the frame memory 128.
As a result of the 4: 1 decimation for the input frame 130, the size of one MB reduces to 8 x 8 pixels. The search area 104 is decimated in the same ratio as the current frame 130. For example, 4: 1 decimation for a search area of 48 x 48 pixels reduces the size of the search area to 24 x 24 pixels. FIG. 6 shows one current MB 140 and a corresponding search area 150 after 4: 1 decimation.
For convenience of explanation, the current frame is described as one of the four decimation frames a, b, c, and d passed through the 4: 1 decimation filters of FIG. 5. The size of each MB in the current frame 100 has a size of 8 x 8 pixels and the search area 104 after being passed through the 4: 1 decimation filters has a size of 24 x 24 pixels.
<Desc/Clms Page number 7>
The tirst register group 102 (FIG. 4) stores one current MB of the current frame 100, and the second register group 106 stores one reference MB of the search area 104. The first and second register groups 102 and 106 store the pixels in a predetermined order showed as the circled numbers in FIG 7. The computing order in each of the first and second register groups 140 and 160 is determined for groups of 8 pixels.
FIG. 7 shows the structures and loading sequences of the first and second register groups 102 and 106 in FIG. 4. The first register group 140 stores the current MB and includes registers each storing eight pixels. The registers are designated in a predetermined order from 0 to 7. The second register group 160 includes registers each storing eight pixels and designated in a predetermined order from 8 to 15. To calculate the difference between the current MB stored in the first register group 102 and the reference MB stored in the second register group 106, the SAD and motion vectors MV for a current reference block are calculated using the following equation.
where, k (m, n) is the pixel value of the k-th frame at (M, N). The motion vector (MVx, MVy) represents the displacement of the current block to the best match in the reference frame.
The dissimilarity computing unit 110 (FIG. 4) computes the differences of 8 pixels at the same time using the Single Instruction Multiple Data (SIMD) method in FIG. 3.
FIG. 13 shows conceptually the dissimilarity computing unit 110 of FIG. 4. An absolute difference value between each pixel of each register 142 of the first register group 102 and each pixel of each register 144 of the second register group 106 is stored in a register 132.
<Desc/Clms Page number 8>
For example, the absolute difference value between 142a and 144b is stored m 132a, and the absolute difference value between 142b and 144b is stored in 132b. To calculate the absolute difference between 142 and 144, one iimer sum instruction is carried out adding each difference value stored in a register 132 in dotted block of Fig. 13.
As shown in the dotted block of FIG. 13, one inner sum instruction is carried out using only multiple adders. In the conventional method in order to add each value, a summation is carried out using an add instruction and shift instruction, therefore additional cycles are required compared with the present method. Thus, to calculate the matching block wholly between the decimated current MB and the decimated reference MB eight inner sum instructions are carried out.
Once the SADs for all the pixels of the current MB 10 and the reference MB 14 are computed, an internal sum for the reference MB 14a is calculated by adding up the SADs for each pixel. After the internal sum for all the reference MBs of the search area 12 are calculated, the reference MB having the least internal sum is identified as the matching block, and the result of the computation is output as a difference ofMB (E~MB) in FIG. 4. The controller 108 in FIG. 4 controls how the reference MB window is shifted in the search area 104 using the SIMD scanning method to reduce the number of memory accesses.
FIG. 12 shows in more detail some differences between conventional scanning methods and the scanning method according to the invention. For a full search, according to the conventional scanning method, a next reference block is shifted from a current reference block by one pixel in a horizontal or vertical direction, as shown in FIGS. 12~1 and 12~2,
<Desc/Clms Page number 9>
respectively. In these cases, most pixels in the currently compared reference block overlap with the pixels used in a next compared reference block.
For the horizontal scanning shown in FIG. 12~1, only the far right region of the next register group 106'~2 includes new pixels from those pixels in register group 106'~1.
Likewise, for the vertical scanning shown in FIG. 12~2, only the lower region of the next register group 106"~2 includes new pixels compared with the current register group 106"~1.
Even though only the edge regions include new pixels, memory accesses are performed for the entire reference macroblock 106.
A vertical scanning for SIMD scheme according to the present invention is shown in FIG. 12~3. Only new pixels 106"'~2 are loaded from main memory into the second register group 106 in FIG. 4. As shown in FIG. 7, the second register group 160b reuses the overlapping pixels stored in register regions 9 through 15 of the first register group 160a. Only the first register region 8 of the second register group 160a is loaded with a new row of pixel values. The first register region 8 is moved down to the last position in the second register group 160b. The other register regions 9-15 that store rows of pixels that overlap with a next reference block are moved up in the sequence by one. For example, register region 9 is moved to a first position, register 10 is moved to a second position, register 11 is moved to a third position, etc.
This shifting of the reference MB requires only one memory access to read a new nonoverlapping row of pixels for each vertical shift in the search area 104 (FIG. 4). Since the entire 8 x 8 pixel array for the next reference MB does not have to be read from memory, the number of memory accesses for scanning the search area 104 is reduced.
<Desc/Clms Page number 10>
FIG 8 shows the shifting of the reference MB in the search area 104 The reference MB window is vertically scanned under the control of the controller 108 in FIG. 4. The reference MB window is vertically shifted by one row of pixels at a time. While this shows vertical window shifting, the same technique can be used for horizontal window shifting.
Horizontal shifting could be used when pixels are stored in sequential locations in memory along vertical columns of the current and reference frames.
As described above, when registers capable of storing data for one MB are used and a reference MB window is vertically shifted in a search area, overlapping pixels between a current reference MB and a next reference MB are reused. This reduces the number of memory accesses required by the controller 108 to scan the search area. The current MB is stored in the first register group, and the current reference MB is stored in the second register group.
FIG. 9 is a flowchart showing in more detail the SIMD scanning scheme according to the present invention. A current frame and a reference frame are decimated in a ratio of n : 1 in step 170. For convenience of explanation, n=4 in the present embodiment. A parameter HS indicates the position of the last column of the first reference MB in the search area, a parameter VS indicates the position of the last low of the first reference MB in the search area, and a parameter DCM indicates four decimation frames.
Here, the first reference MB is the left uppermost MB in the search area, and the first parameter HS and the second parameter VS for the first reference MB are zero. In step 172, the parameters HS, VS, DCM are all initialized to zero, and a minimum dissimilarity E~MIN is initialized with a value as large as possible, for example, infinity.
<Desc/Clms Page number 11>
Identification Nos. 0, 1, 2, and 3 are assigned to the four decimation frames, respectively. The parameter DCM is compared to the value 4 in step 174 to determine whether motion estimation is completed for the last decimation frame. If motion estimation is not completed for the last decimation frame, a current MB is loaded into the first register group 140 (see FIG. 7) in step 176.
It is determined in step 178 whether the HS parameter is less than 17. When the HS parameter is not less than 17, the motion estimation is completed for the last column (HS 16) in the search area. HS is reset to zero in step 192 and DCM is incremented to the next DCM frame in block 198. The process then returns to step 174.
If motion estimation is not completed up to HS16, it is determined whether the VS parameter is less than 17 in step 180. If VS is less than 17, a pipelining procedure is performed in steps 182 and 184. Only the last row VS 1 is loaded into the reference MB in step 182 (see FIG. 8). If the motion estimation is not completed up to the last low, i. e. , if a reference MB window is not shifted to the last row VS 16, the reference MB is loaded into the second register group 160a in step 182. The difference between the current MB and the reference MB is calculated in step 184.
In this case, the new row VS 1 in the vertical direction is stored in the first register position in the sequence of register regions. For example, $register 8 of the second register group 160a is loaded with the next new nonoverlapping row of pixels for the next reference MB. The other register regions, i. e. , $register 9 through $registerl5, are moved up in the sequence by one. That is, the second register group 106b in FIG. 7 reuses the pixels stored in the register regions $register9 through $register 15. Thus, only the pixels of the new row VS1
<Desc/Clms Page number 12>
(FIG. 8) are accessed from memory and stored m the register region $register8 of the second register group 160a.
In step 184, the difference between MBs loaded into the first and second register groups 140 and 160 in FIG. 7 are computed. The MB dissimilarity EMB is compared with the minimum dissimilarity EMIN in step 186. If the MB dissimilarity EMB is less than the minimum dissimilarity EMIN, the minimum dissimilarity E~MIN is set to the MB dissimilarity E~MB in step 188. If the MB dissimilarity EMB is not less than the minimum dissimilarity E~MIN, the current minimal dissimilarity EMIN is maintained, and the parameter VS is incremented in step 190. Then steps 180 through 190 are repeated until vertical scanning of the reference MB reaches the last low VS16 (FIG. 8).
If it is determined in step 180 that the second parameter VS is not less than 17 as a result of scanning the last row VS16, the parameter VS is initialized to zero in step 200. The parameter HS is incremented in step 202, and the process returns to step 178. In other words, the reference MB window is shifted one pixel position to the right. Steps 180-190 are then repeated.
After the reference MB window is shifted in a horizontal direction to the last column HS16, i. e. , if it is determined in step 178 that the parameter HS is not less than 17, the first parameter HS is reinitialized to zero in step 192. The DCM parameter is incremented in step 198 and the process returns to step 174. Incrementing the DCM parameter means that motion estimation for another decimation frame is performed.
When motion estimation is completed for all the decimation frames, i. e. , if it is determined in step 174 that the DCM parameter is not less than 4, the reference MB with the
<Desc/Clms Page number 13>
least dissimilarity is identified as the matching block 111 step 204. Motion estimation for the current frame is completed by repeating the processes described above for all the MBs of the current frame.
As described above, the first and second register groups store a current MB and a reference MB. The reference MB window is vertically shifted in a search area for motion estimation. Overlapping pixels between a current reference MB and a next reference MB are reused. As a result, fewer instructions (Load/Store) are required when loading the next reference MB into the second register groups. This allows faster motion estimation with less power consumption.
FIGS. lOa through 10d show the advantages of the present invention over conventional motion estimation methods. FIG. lOa identifies the instruction count for a conventional motion estimation method in which decimation is not performed, i. e., full search algorithm It was determined that 26. 2% of the total instruction count for the conventional method of FIG. lOa is required for memory access instruction and the remaining 73.8% of the instruction counts are for non-memory accessing. FIG. lOa corresponds to FIG. 2 where a reference MB is horizontally shifted in a search area and motion estimation is carried out using SAD for each pixel. FIG. lOb shows total instruction count for a conventional motion estimation method where decimation is performed. FIG. lOc shows the total instruction count for conventional motion estimation in which decimation and SIMD are used.
FIG. lOd shows the total instruction count for the motion estimation using the present invention. For the three cases shown in FIGS. lOb through lOd, the percentages 27.0%, 1.6%, and 0.9%, respectively, are a relative ratio of the memory access instruction counts compared
<Desc/Clms Page number 14>
with the conventional motion estimation method of FIG. lOa. It is apparent that the orthogonal scanning method to access the non-overlapped portion is the most efficient technique for reducing the memory access count.
FIG. 11 shows the number of total clock cycles required for 2 frames having the Quarter Common Intermediate Format (QCIF) required to extract 99 minimum SADs. In FIG.
11, lla corresponds to FIG. lOa, llb corresponds to FIG. lOb, lle corresponds to FIG. lOc, and I ld corresponds to FIG. lOd. The performance of the orthogonal scanning scheme to access the non-overlapped portion is twice the improvement over the conventional motion estimation method using normal SIMD.
The scanning technique described above can be implemented with a Single Instruction Multiple Data (SIMD) device or a Very Long Instruction Word (VLIW) device for comparing the current-macroblock with the reference macroblock. The scheme used for matching macroblocks can include a Mean of the Absolute Difference (MAD), Mean of the Absolute Error (MAE), or Sum of the Absolute Difference (SAD) scheme. The method for selecting the next reference macroblock can include a fast algorithm or full search algorithm Of course, other single instruction/multi-data devices, matching schemes, and searching algorithms can also be used.
The invention may be embodied in a general purpose digital computer by running a program from a computer usable medium, including but not limited to storage media such as magnetic storage media (e. g., ROM's, floppy disks, hard disks, etc. ), optically readable media (e. g. , CD-ROMs, DVDs, etc. ) and carrier waves (e. g. , transmissions over the Internet). The
<Desc/Clms Page number 15>
computer usable medium can be stored and executed ni distributed computer systems connected by a network.
The system described above can use dedicated processor systems, micro controllers, programmable logic devices, or microprocessors that perform some or all of the operations.
Some of the operations described above may be implemented in software and other operations may be implemented in hardware.
For the sake of convenience, the operations are described as various interconnected functional blocks or distinct software modules. This is not necessary, however, and there may be cases where these functional blocks or modules are equivalently aggregated into a single logic device, program or operation with unclear boundaries. In any event, the functional blocks and software modules or features of the flexible interface can be implemented by themselves, or in combination with other operations in either hardware or software.
Having described and illustrated the principles of the invention in a preferred embodiment thereof, it should be apparent that the invention may be modified in arrangement and detail without departing from such principles. Claimed are all modifications and variations coming within the spirit and scope of the following claims.

Claims (29)

  1. CLAIMS An image processing apparatus, comprising : a first storage element adapted to store a current macroblock; a second storage element adapted to store a first reference macroblock; a computing unit to compute a difference between contents of the first storage element and the second storage element; and a controller adapted to load a second reference macroblock into the second storage element by replacing a nonoverlapping portion of the first reference macroblock with a nonoverlapping portion of the second reference macroblock.
  2. 2. An image processing apparatus of claim 1 wherein results of the computing unit are used far determining a motion vector.
  3. 3. An image processing circuit of claim 1 wherein the computing unit includes a Single Instruction Multiple Data (SIMD) device.
  4. 4. An image processing apparatus according to claim 1 wherein portions of the first reference macroblock that are overlapping with portions of the second reference macroblock are reused in the second storage element by the computing unit to compute the difference between the first storage element and the second storage element.
    <Desc/Clms Page number 17>
  5. 5. An Image processing apparatus according to claun 1 wherein the first storage element comprises multiple registers each storing a group of pixel values for the current macroblock and the second storage element comprises multiple registers storing a group of pixel values for the first reference macroblock.
  6. 6. An image processing apparatus according to claim 5 wherein the computing unit compares the group of pixel values stored in each register of the first storage element with the group of pixels values stored in each register of the second storage element at the same time.
  7. 7. An image processing apparatus according to claim 5 wherein each one of the multiple registers in the first storage element stores a row or a column of the current macroblock and each one of the multiple registers in the second storage element stores a row or a column of the first reference macroblock.
  8. 8. An image processing apparatus according to claim 1 wherein the nonoverlapping portion of the second reference macroblock is loaded from a memory into the second storage element.
  9. 9. An image processing apparatus according to claim 1 wherein the controller loads the second reference macroblock into the second storage element by moving a first register position storing nonoverlapping portion to a last register position in the second storage
    <Desc/Clms Page number 18>
    element and moving up m order other registers in the second storage element storIng overlapping portions of the first reference macroblock.
  10. 10. An image processing apparatus according to claim 1 including a preprocessor that decimates a current frame into multiple decimated current frames and decimates a reference frame into multiple decimated reference frames.
  11. 11. An image processing apparatus according to claim 1 wherein the controller and the computing unit are implemented in either software or hardware.
  12. 12. An image processing apparatus according to claim 5 wherein the computing unit includes: a third storage element adapted to store absolute differences between each pixel of each register of the first storage element and each pixel of each register of the second storage element; and a summation circuit for deriving a summation for the absolute difference values stored in the third storage element.
  13. 13. An image processing apparatus according to claim 12 wherein the summation circuit comprises only multiple adders.
    <Desc/Clms Page number 19>
  14. 14. An image processing apparatus according to claim 12 wherem a smgle umer sum instruction causes the summation circuit to generate the summation for all of the absolute difference values stored in the third storage element.
  15. 15. A motion estimation method, comprising: loading a current macroblock; loading a current reference macroblock; comparing the current macroblock with the current reference macroblock; and loading a next reference macroblock by replacing a nonoverlapping portion of the loaded current reference macroblock with a nonoverlapping portion of the next reference macroblock.
  16. 16. A method according to claim 15 including reusing an overlapping portion of the current reference macroblock for comparing the next reference macroblock with the current macroblock.
  17. 17. A method according to claim 15 including: loading in one instruction a nonoverlapping group of pixels from the next reference macroblock into an identified register that currently contains a nonoverlapping portion of pixels for the current reference macroblock ; and reusing pixels in other registers that overlap with the next reference macroblock.
    <Desc/Clms Page number 20>
  18. 18. A method according to claim 17 mcludiiig loading the tdentihed register from a memory storing a reference frame.
  19. 19. A method according to claim 17 including moving an order of the identified register storing the nonoverlapping protion of the next reference macroblock to a last register position and moving up the order of the other registers.
  20. 20. A method according to claim 15 including comparing each group of pixel values for the loaded current macroblock with each group of pixel values for the loaded current reference macroblock at the same time.
  21. 21. A method according to claim 20 wherein the group of pixel values each comprise a row or column of the current macroblock or a row or column of the current reference macroblock.
  22. 22. A method according to claim 15 including using a Single Instruction Multiple Data (SIMD) device or a Very Long Instruction Word (VLIW) device for comparing the current macroblock with the current reference macroblock.
  23. 23. A method according to claim 15 including comparing the current macroblock with the current reference macroblock using a matching macroblock scheme.
    <Desc/Clms Page number 21>
  24. 24. A method according to claim 23 wherein the matching macroblock scheme is Mean of the Absolute Difference (MAD), Mean of the Absolute Error (MAE), or the Sum of the Absolute Difference (SAD).
  25. 25. A method according to claim 15 including selecting the next reference macroblock using a fast algorithm or full search algorithm
  26. 26. A method according to claim 15 including: decimating a current frame into multiple decimated current frames; decimating a reference frame into multiple decimated reference frames; selecting the current macroblock from the decimated current frames; shifting the selected current macroblock over search areas of the decimated reference frames to identify a reference macroblock most similar to the current macroblock; and deriving a motion vector for the identified reference macroblock.
  27. 27. A method according to claim 20 including: storing absolute differences between each group of pixel values for the loaded current macroblock with each group of pixel values for the loaded current reference macroblock; and deriving a summation of the absolute difference values.
  28. 28. A method according to claim 27 including using only adders to derive the summation for the absolute difference values.
    <Desc/Clms Page number 22>
  29. 29. A method according to claim 28 including using a single inner sum instruction to generate the summation for all of the absolute difference values.
GB0213247A 2001-07-09 2002-06-10 Motion estimation apparatus and method for scanning a reference macroblock window in a search area Expired - Fee Related GB2378345B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR20010040904 2001-07-09
US10/112,011 US20030012281A1 (en) 2001-07-09 2002-03-29 Motion estimation apparatus and method for scanning an reference macroblock window in a search area

Publications (3)

Publication Number Publication Date
GB0213247D0 GB0213247D0 (en) 2002-07-17
GB2378345A true GB2378345A (en) 2003-02-05
GB2378345B GB2378345B (en) 2004-03-03

Family

ID=26639222

Family Applications (1)

Application Number Title Priority Date Filing Date
GB0213247A Expired - Fee Related GB2378345B (en) 2001-07-09 2002-06-10 Motion estimation apparatus and method for scanning a reference macroblock window in a search area

Country Status (4)

Country Link
JP (1) JP2003125415A (en)
KR (1) KR100486249B1 (en)
CN (1) CN1297134C (en)
GB (1) GB2378345B (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2400260B (en) * 2003-03-31 2006-08-23 Duma Video Inc Video compression method and apparatus
WO2007008357A1 (en) * 2005-07-08 2007-01-18 Tag Networks, Inc. Video game system using pre-generated motion vectors
US8118676B2 (en) 2005-07-08 2012-02-21 Activevideo Networks, Inc. Video game system using pre-encoded macro-blocks
US8194862B2 (en) 2009-07-31 2012-06-05 Activevideo Networks, Inc. Video game system with mixing of independent pre-encoded digital audio bitstreams
US8270439B2 (en) 2005-07-08 2012-09-18 Activevideo Networks, Inc. Video game system using pre-encoded digital audio mixing
US8284842B2 (en) 2005-07-08 2012-10-09 Activevideo Networks, Inc. Video game system using pre-encoded macro-blocks and a reference grid
US9788029B2 (en) 2014-04-25 2017-10-10 Activevideo Networks, Inc. Intelligent multiplexing using class-based, multi-dimensioned decision logic for managed networks
US9800945B2 (en) 2012-04-03 2017-10-24 Activevideo Networks, Inc. Class-based intelligent multiplexing over unmanaged networks
US9826197B2 (en) 2007-01-12 2017-11-21 Activevideo Networks, Inc. Providing television broadcasts over a managed network and interactive content over an unmanaged network to a client device
US10200744B2 (en) 2013-06-06 2019-02-05 Activevideo Networks, Inc. Overlay rendering of user interface onto source video
US10275128B2 (en) 2013-03-15 2019-04-30 Activevideo Networks, Inc. Multiple-mode system and method for providing user selectable video content
US10409445B2 (en) 2012-01-09 2019-09-10 Activevideo Networks, Inc. Rendering of an interactive lean-backward user interface on a television

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005025230A1 (en) * 2003-08-28 2005-03-17 Hitachi Ulsi Systems Co., Ltd. Image processing device
KR100621137B1 (en) * 2004-02-27 2006-09-13 세이코 엡슨 가부시키가이샤 Moving image encoding apparatus and moving image processing apparatus
US20060159170A1 (en) * 2005-01-19 2006-07-20 Ren-Wei Chiang Method and system for hierarchical search with cache
KR100677562B1 (en) 2005-02-03 2007-02-02 삼성전자주식회사 Motion estimation method and motion estimation apparatus
CN100370808C (en) * 2005-06-13 2008-02-20 北京中星微电子有限公司 Sports detecting method
KR100678911B1 (en) * 2005-07-21 2007-02-05 삼성전자주식회사 Method and apparatus for video signal encoding and decoding with extending directional intra prediction
US8074248B2 (en) 2005-07-26 2011-12-06 Activevideo Networks, Inc. System and method for providing video content associated with a source image to a television in a communication network
JP4182442B2 (en) * 2006-04-27 2008-11-19 ソニー株式会社 Image data processing apparatus, image data processing method, image data processing method program, and recording medium storing image data processing method program
KR101328931B1 (en) * 2006-11-28 2013-11-14 엘지전자 주식회사 Video decoder and decoding method
WO2008088772A2 (en) 2007-01-12 2008-07-24 Ictv, Inc. Mpeg objects and systems and methods for using mpeg objects
KR101520027B1 (en) 2007-06-21 2015-05-14 삼성전자주식회사 Method and apparatus for motion estimation
KR100909390B1 (en) * 2007-09-18 2009-07-24 한국과학기술원 High speed motion compensation device and method
CN101179724B (en) * 2007-12-11 2010-09-29 北京中星微电子有限公司 Frame storage method and apparatus for interframe compressed encoding
CN101800893B (en) * 2009-02-06 2013-01-16 宏碁股份有限公司 Low-power high-performance video coding method for implementing motion estimation
CN102340617B (en) * 2010-07-14 2014-06-11 奇景光电股份有限公司 Motion estimation (ME) and motion compensation (MC) circuit
WO2012051528A2 (en) 2010-10-14 2012-04-19 Activevideo Networks, Inc. Streaming digital video between video devices using a cable television system
EP2695388B1 (en) 2011-04-07 2017-06-07 ActiveVideo Networks, Inc. Reduction of latency in video distribution networks using adaptive bit rates
US9123084B2 (en) 2012-04-12 2015-09-01 Activevideo Networks, Inc. Graphical application integration with MPEG objects
US9219922B2 (en) 2013-06-06 2015-12-22 Activevideo Networks, Inc. System and method for exploiting scene graph information in construction of an encoded video sequence
US9294785B2 (en) 2013-06-06 2016-03-22 Activevideo Networks, Inc. System and method for exploiting scene graph information in construction of an encoded video sequence
CN109120941A (en) * 2018-09-03 2019-01-01 山东师范大学 A kind of video image data method for reusing, processor and system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2327827A (en) * 1996-11-29 1999-02-03 Sony Corp Motion vector detection image processing apparatus
EP0959626A2 (en) * 1998-05-19 1999-11-24 Nippon Telegraph And Telephone Corporation Motion vector search method and apparatus

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07264602A (en) * 1994-03-24 1995-10-13 Sony Corp Motion vector detecting method
KR100205146B1 (en) * 1996-09-12 1999-07-01 이득렬 Motion estimation method in digital video encoder
KR100397055B1 (en) * 2000-07-21 2003-09-06 (주)씨앤에스 테크놀로지 Motion estimator architecture for low bit rate image communication
KR100446235B1 (en) * 2001-05-07 2004-08-30 엘지전자 주식회사 Merging search method of motion vector using multi-candidates

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2327827A (en) * 1996-11-29 1999-02-03 Sony Corp Motion vector detection image processing apparatus
EP0959626A2 (en) * 1998-05-19 1999-11-24 Nippon Telegraph And Telephone Corporation Motion vector search method and apparatus

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7519115B2 (en) 2003-03-31 2009-04-14 Duma Video, Inc. Video compression method and apparatus
GB2400260B (en) * 2003-03-31 2006-08-23 Duma Video Inc Video compression method and apparatus
WO2007008357A1 (en) * 2005-07-08 2007-01-18 Tag Networks, Inc. Video game system using pre-generated motion vectors
US8118676B2 (en) 2005-07-08 2012-02-21 Activevideo Networks, Inc. Video game system using pre-encoded macro-blocks
US8270439B2 (en) 2005-07-08 2012-09-18 Activevideo Networks, Inc. Video game system using pre-encoded digital audio mixing
US8284842B2 (en) 2005-07-08 2012-10-09 Activevideo Networks, Inc. Video game system using pre-encoded macro-blocks and a reference grid
US8619867B2 (en) 2005-07-08 2013-12-31 Activevideo Networks, Inc. Video game system using pre-encoded macro-blocks and a reference grid
US9061206B2 (en) 2005-07-08 2015-06-23 Activevideo Networks, Inc. Video game system using pre-generated motion vectors
US9826197B2 (en) 2007-01-12 2017-11-21 Activevideo Networks, Inc. Providing television broadcasts over a managed network and interactive content over an unmanaged network to a client device
US8194862B2 (en) 2009-07-31 2012-06-05 Activevideo Networks, Inc. Video game system with mixing of independent pre-encoded digital audio bitstreams
US10409445B2 (en) 2012-01-09 2019-09-10 Activevideo Networks, Inc. Rendering of an interactive lean-backward user interface on a television
US9800945B2 (en) 2012-04-03 2017-10-24 Activevideo Networks, Inc. Class-based intelligent multiplexing over unmanaged networks
US10506298B2 (en) 2012-04-03 2019-12-10 Activevideo Networks, Inc. Class-based intelligent multiplexing over unmanaged networks
US10757481B2 (en) 2012-04-03 2020-08-25 Activevideo Networks, Inc. Class-based intelligent multiplexing over unmanaged networks
US10275128B2 (en) 2013-03-15 2019-04-30 Activevideo Networks, Inc. Multiple-mode system and method for providing user selectable video content
US11073969B2 (en) 2013-03-15 2021-07-27 Activevideo Networks, Inc. Multiple-mode system and method for providing user selectable video content
US10200744B2 (en) 2013-06-06 2019-02-05 Activevideo Networks, Inc. Overlay rendering of user interface onto source video
US9788029B2 (en) 2014-04-25 2017-10-10 Activevideo Networks, Inc. Intelligent multiplexing using class-based, multi-dimensioned decision logic for managed networks

Also Published As

Publication number Publication date
CN1297134C (en) 2007-01-24
KR100486249B1 (en) 2005-05-03
GB0213247D0 (en) 2002-07-17
GB2378345B (en) 2004-03-03
KR20030007087A (en) 2003-01-23
CN1396762A (en) 2003-02-12
JP2003125415A (en) 2003-04-25

Similar Documents

Publication Publication Date Title
GB2378345A (en) Method for scanning a reference macroblock window in a search area
EP1262073B1 (en) Methods and apparatus for motion estimation using neighboring macroblocks
US6292589B1 (en) Method for choosing rate control parameters in motion-compensated transform-based picture coding scheme using non-parametric technique
US20070071101A1 (en) Systolic-array based systems and methods for performing block matching in motion compensation
US5995080A (en) Method and apparatus for interleaving and de-interleaving YUV pixel data
US20050238102A1 (en) Hierarchical motion estimation apparatus and method
US6430317B1 (en) Method and apparatus for estimating motion using block features obtained from an M-ary pyramid
US20090110077A1 (en) Image coding device, image coding method, and image coding integrated circuit
US8345764B2 (en) Motion estimation device having motion estimation processing elements with adder tree arrays
WO2001049039A1 (en) Methods and apparatus for reduction of prediction modes in motion estimation
US20030012281A1 (en) Motion estimation apparatus and method for scanning an reference macroblock window in a search area
US20080037647A1 (en) Methods and Apparatus For Quarter-Pel Refinement In A SIMD Array Processor
WO2001049029A1 (en) Methods and apparatus for motion estimation in compressed domain
EP1086590A1 (en) Motion estimation
US6016163A (en) Methods and apparatus for comparing blocks of pixels
Lee et al. Multi-pass and frame parallel algorithms of motion estimation in H. 264/AVC for generic GPU
JPH04314290A (en) Motion vector detector
Baglietto et al. Parallel implementation of the full search block matching algorithm for motion estimation
Li et al. A VLSI architecture design of an edge based fast intra prediction mode decision algorithm for H. 264/AVC
ONOYE et al. Single chip implementation of motion estimator dedicated to MPEG2 MP@ HL
Hanssens et al. Parallel processor for motion estimation
US20100220786A1 (en) Method and apparatus for multiple reference picture motion estimation
US6968011B2 (en) Motion vector detecting device improved in detection speed of motion vectors and system employing the same devices
US11622106B2 (en) Supporting multiple partition sizes using a unified pixel input data interface for fetching reference pixels in video encoders
US11909993B1 (en) Fractional motion estimation engine with parallel code unit pipelines

Legal Events

Date Code Title Description
PCNP Patent ceased through non-payment of renewal fee

Effective date: 20100610