KR20010052624A - Motion estimation - Google Patents

Abstract

A method of determining a best match between a first pixel array of a currently encoded picture and a plurality of second pixel arrays of a search area of a reference picture, the first and second pixel arrays each comprising a plurality of rows and columns Pixel values. The method is designed to be performed in a motion estimation search engine of a digital video encoder and comprises a set of horizontal sums (S _1H to S _16H ) representing the sum of the pixel values of the individual rows of the first pixel array, Providing a first orthogonal sum type of the first pixel array (M1) consisting of a set of vertical sums (S _1V to S _16V ) representing the sum of the individual pixel values of the columns of the pixel array, A set of horizontal sums (S _1H to S _16H ) representing the sum of the individual pixel values of the rows of each second pixel array for each of at least selected ones of the second pixel arrays Providing a plurality of second pixel orthogonal sum types each consisting of a set of vertical sums (S _1V to S _16V ) representing the sum of the individual pixel values of the columns of the pixel array.

Description

Motion estimation {Motion estimation}

In general, encoding an MPEG video data stream requires many steps. The first of these steps consists of dividing each picture into macroblocks. Next, theoretically, each macroblock of each " non-intra " picture in the MPEG video data stream has a specified vertical (vertical) of the corresponding position of the current macroblock in the anchor picture And all possible 16x16 pixel arrays located within the horizontal search range. This theoretical " full search algorithm " (i.e., searching all possible blocks in the search area for best match) produces the best match, e.g., NxN block size and N + 2w) x (N + 2w) because of the large amount of computation required for the search area. The distortion function MAE must be calculated (2w + 1) ² times for each block, which is a large amount of computation. Rather, the algorithm is used only as a benchmark or benchmark that allows comparison of different, more realistic motion estimation algorithms that can be implemented much faster and with less computation. These more realistic motion estimation algorithms are generally referred to as " fast search algorithms ".

In the predetermined prediction mode, the result of the above-described searching or " motion estimation " process is a motion vector corresponding to the position of the macroblock (according to the specified matching criterion) closest to the anchorperson within the specified search range . Once the prediction mode and motion vector (s) have been determined, the pixel values of the closest matching macroblock are subtracted from the corresponding pixels of the current macroblock and the resulting differential pixel of the 16x16 array is subtracted from the 8x8 "quot; blocks ", and a discrete cosine transform (DCT) is performed for each block, and the resulting coefficients are each quantized and transformed into Huffman coding (prediction type, motion vector, Other information) and generates an MPEG bit stream. If no appropriate macroblock matches are detected in another picture or if the current picture is an intra or " I- " picture, the process is performed on actual pixels of the current macroblock (i.e., No difference is taken with respect to the pixels), and the macroblock is designated as the " intra " macroblock.

In all MPEG-2 prediction modes, the basic technique of motion estimation is to compare a given 16x16 pixel array in an anchor image with the current macroblock, estimate the matched quality according to the specified metric, Lt; RTI ID = 0.0 > 16x16 < / RTI > The hardware or software that performs such searches is commonly referred to as a " search engine ", and there are many well-known criteria for determining the matched quality. Among the best known standards, a minimum absolute error (MAE) in which a metric is composed of a sum of absolute difference values of 256 pixels in a macroblock and corresponding pixels in a matching anchor picture macro block, And a minimum square error (MSE) in which the metric is constructed by summing the squares of the pixel differences. Either way, the horizontal and vertical position of the match for the current macroblock constitutes a motion vector by selecting the match with the smallest value as the best match in the specified search position. Nevertheless, if the resulting minimum sum appears to be too large, then there is no suitable match in the current macroblock, which is coded as an intra macroblock. For the purposes of the present invention, either of the two criteria, or any other suitable technique may be used.

Various fast search algorithms reduce the overall effort by evaluating a distortion function (e.g., a MAE function) only on a certain subset of candidate motion vector locations within the search area. These algorithms are based on the assumption that the distortion measure monotonically decreases to the best match prediction direction. Although this assumption is not always true, it is much less computable to find the optimal motion vector of the lane.

The most commonly used method for motion estimation is a hybrid approach, generally divided into several processing steps. First, the image is decimated by pixel averaging. Next, a fast search algorithm that operates on a smaller number of pixels is performed, yielding the result of approaching the best match. Then, a full search algorithm is performed in a small search area around the obtained motion vector. If a half-pel vector is required (as in MPEG-2), half-pel search is performed as a separate step or combined into a limited full search.

Despite significant reductions that can be achieved in the hybrid approach to motion estimation, a large amount of computation must still be performed for each iteration of computing the MAE. For a desired application such as an MPEG-2 HDTV with a motion block size of 16x16, it is desirable to calculate a distortion function every clock cycle for each block offset. The distortion function computational unit (DFCU) (8-bit luminance data is used for motion estimation) starting from 8 to produce a < / RTI > This number is equal to the sum of 255 subtracting circuits, 256 absolute summing circuits, and 255 summing circuits of increasing bit width, for a total of 757 circuits starting at 8 per DFCU and increasing in bit width will be.

Depending on the image resolution, many of these extremely complex devices will be required in an actual system. It is possible to use a small number of circuits in the DFCU to reuse the hardware of the DFCU, but it may increase processing time and may not be acceptable in the required applications such as HDTV. In this case, the number of DFCUs simply needs to be increased to compensate for the improved parallelism.

In the hybrid approach to motion estimation, the first step (roughly search) is usually the most demanding step in terms of hardware usage, since it has to cover the largest search area to produce a fairly accurate match.

The present invention relates to a motion estimation method.

Figure 1a shows a 32-orthogonal type for a 16x16 macroblock that is not decimated.

Figure IB shows a 16-orthogonal type for 8x8 macroblocks representing 16x16 macroblocks that are 2: 1 decimated.

Figure 2 is a flow chart and graph illustrating the best match estimations together in accordance with a preferred embodiment of the present invention.

3 illustrates a basic method of a preferred implementation of the present invention in the context of orthogonal sum updating in a horizontal motion estimation search;

4 is a block diagram of an orthogonal sum generator that constitutes an embodiment of the present invention;

Figure 5 illustrates a RAM operation sequence in an exemplary horizontal motion estimation search using the method of the present invention.

6 is a block diagram of a motion estimation search engine constituting an embodiment of the present invention;

Based on the foregoing, there is presently a need in the art to provide a method and apparatus that reduces the DFCU hardware required to perform motion estimation and complexity or motion estimation, and provides significant picture quality improvements at a reasonable cost, There is a need for estimation methods. The invention, as defined in the independent claims, fulfills this need in the art. Dependent claims define advantageous embodiments.

In summary, the method of the present invention finds the best match by comparing the unique types of macroblocks rather than comparing the individual brightness values of the pixels arranged in the search area with the current macroblock. The method is based on the assumption that the same hypothesis is based on all the fast search algorithms, i.e., the distortion measure monotonically decreases to the best match prediction direction.

The present invention relates to an image processing apparatus having a first pixel array having individual pixel values of a plurality of rows and columns and having pixel values of a plurality of rows and columns in an image to be currently encoded, Quot; is < / RTI > The method is designed to be performed in a motion estimation search engine of a digital video encoder and comprises a set of horizontal sum representing the sum of pixel values of each of the rows of the first pixel array, Providing a first orthogonal sum type of the first pixel array consisting of a set of vertical sums representing a sum of values for each of the plurality of second pixel arrays, A plurality of first and second pixel arrays each comprising a set of horizontal summations representing the sum of the pixel values of the rows of the pixel array and a set of vertical summations representing the sum of pixel values of the columns of each second pixel array, Pixel orthogonal-sum type, and to determine a best match between the first pixel array and the second pixel array, And comparing the first orthogonal sum type with each of the orthogonal sum types. In the disclosed embodiment, the first and second pixel arrays are decimated or non-decimated macroblocks having a structure defined by, for example, the MPEG standard, the MPEG-2 standard.

The present invention includes an apparatus, such as a motion estimation search engine of a digital video encoder, for implementing the method of the present invention.

In a preferred embodiment, the method and apparatus of the present invention significantly reduces the required computation and significantly accelerates motion estimation search by storing and widely reusing previously computed (usable) sums in local memory to produce orthogonal sum Thereby significantly reducing the required hardware of the motion estimation search engine. In addition, the local memory may be effectively a RAM, for example a DRAM or an SRAM, as opposed to being implemented as a shift register matrix required in presently available technology. However, although this is novel and presently preferred in one aspect of the features of the present invention, this is not necessarily an essential feature of the present invention in its broadest sense, as will be apparent from the following description.

These and other objects, features, and advantages of the present invention will be readily understood from the following detailed description taken in conjunction with the accompanying drawings.

In summary, the motion estimation method of the present invention generally comprises the following steps. First, the individual pixel values of each row and column of the current macroblock are summed to produce a set of orthogonal sums representing a unique pattern or " signature " of the content of the macroblock. Next, the resulting orthogonal sum type of the macroblock is compared with the corresponding orthogonal sum types of pixel arrays of each macroblock size in the specified search area of the reference or anchor picture (s) And finds the best match according to the specified match criterion or search metric, e.g., a minimum absolute error (MAE) distortion function. Statistically, since macroblocks with different content rarely occur when they have the same type, the probability of matching is low. In addition, the small step increments of the macroblock origin in the search area will not cause a large jump in magnitude for the bandwidth limited filtered video, since orthogonals represent the average luminance magnitude per line (row) or column. For this reason, it can be concluded that the distortion measure calculated based on the agreement of the orthogonal sum sets monotonously decreases not only in the search method of the prior art but also in the best match prediction direction.

A specific example of the motion estimation method of the present invention will be described with reference to Figs. 1A and 1B. In particular, in Fig. 1A, the individual pixel (luminance) values are summed for each row (1H-16H) and each column (1V-16V) of the decimated 16x16 macroblocks M1, To produce a set of orthogonal sums S _1H to S _16H (horizontal sum) and S _1V to S _16V (vertical sum) that constitute the orthogonal sum type of the macroblock M ₁ . In Fig. 1B, the individual pixel (luminance) values are summed for each row 1H-8H and each column 1V-8V of the 8x8 macroblock M1 ' To produce a set of orthogonal sums S _1H to S _8H (horizontal sum) and S _1V to S _8V (vertical sum) that make up the sum type. The 8x8 macroblock M1 'constitutes a macroblock M1 decimated horizontally and vertically by 2: 1.

2, the motion estimation method of the present invention is performed as follows. In particular, the best match estimation process may be performed by comparing the orthogonal sum type of each macroblock in the specified search area (search area macroblock (SAM)) of the reference or anchor picture with the orthogonal sum type of the current coded macroblock (CM) (The ME), and then the maximum correlation with the orthogonal-sum set of the current macroblock according to the specified matching criterion (search metric), such as MAE, MSE, or any other suitable metric, (Search area) macroblock having the best match (BM). The graph shown in the lower portion of Fig. 2 shows the magnitude (M) of the elements of the orthogonal sum set.

Due to the large complexity of the distortion function computation unit (DFCU), the motion estimation search is typically performed for at least initially decimated video (i.e., decimated macroblock). For example, when generating an orthogonal sum set for the non-decimated macroblocks shown in FIG. 1A, the number of sums representing the orthogonal sum type of 16x16 macroblocks is 32 (2x16) When generating an orthogonal sum for a 2: 1 decimated macroblock, the number of sums representing the orthogonal sum type of the 8x8 macroblock is reduced to 16 (2x8). Evaluating the distortion function for 2N numbers will reduce the required computation of the DFCU compared to the prior art that requires the distortion function to be evaluated for the number of N2. For example, for the non-decimated 16x16 macroblock shown in FIG. 1A, the distortion function should be evaluated for a number less than 8 (256/32), and the decimated 8x8 macroblock shown in FIG. , The distortion function should be evaluated for a number less than four times (64/16).

As described above, the computational complexity of the DFCU is a major factor in the cost of the motion estimation circuit (search engine). However, since the motion estimation method of the present invention provides a dramatic reduction in the cost and complexity of the DFCU, it becomes much more practical to start with a decimated video that is not decimated or low-level in the motion estimation search, Estimated search accuracy, and eventually image quality. In this regard, the motion estimation method of the present invention not only significantly reduces the number of stages of motion estimation, but also eliminates all of the decimation stages to eliminate the special video filtering circuitry required. With this hardware reduction, the search process may begin with a video that is not decimated, which can significantly improve image quality at reasonable cost.

Another advantage realized by the motion estimation method of the present invention is greatly improved motion speed. Usually, a plurality of logical stages are required to compare the arranged luminance magnitudes, except in practice the possibility of obtaining results within a single clock cycle. For this reason, the system clock frequency must be reduced or the system must be pipelined using logical resources. The motion estimation method of the present invention dramatically reduces the MAE computation by allowing simultaneous computation of the orthogonal sum to be achieved within a single clock cycle.

In addition to these advantages, the present invention greatly reduces cross-communication between calculations performed on data originating from different memories. This allows pre-computation and storage of intermediate results (orthogonal sum) that can be very beneficial in some hardware structures prior to motion estimation.

The basic principle of a preferred embodiment of the present invention will be described with reference to Fig. In particular, the horizontal (perpendicular to) the sum horizontal (perpendicular to) the sum (OS _NEW) for the 8 pixel macroblocks shifted to right by one pixel with respect to (OS _OLD) is calculated before 8 pixel macroblocks during a previous iteration of the horizontal navigation For calculation, the following equation (1) is used.

OS _NEW = OS _OLD - a ₀ + a _n0 (1)

Where a ₀ is the pixel value of the pixel that was the horizontal origin of the previous macroblock and a _n0 is the pixel value of the pixel that is the horizontal origin of the macroblock moved by one pixel to the right with respect to the " .

For example, if the horizontal origin of the previous macroblock is a _n-1 pixel and the horizontal origin of the macroblock moved by one pixel to the right is represented by a _n , _NEW = OS _OLD - a _n-1 + a _{n + 7} .

That is, since the only the pixels included in the new macroblocks that were not included in the previous macroblock pixel a _{n + 7,} due to the one pixel moves to the right, and its value is calculated before calculating the OS _NEW quadrature-sum because be added to the OS _oLD and, although not included in the new macroblock is the only pixels the pixel a _n-1 that were part of the previous macro-block, due to the one pixel moves to the right, and its value prior to calculating the OS _{nEW Must} be subtracted from the calculated orthogonal OS _OLD .

Similarly, the pixel search advances to the right by one pixel, the horizontal origin of the previous macroblock becomes a pixel denoted by a _n , and the horizontal origin of the " new " macroblock shifted by one pixel to the right is a _{n + 1} (1), OS _NEW = OS _OLD - a _{n - 1} + a _{n + 7} is obtained.

That is, since the only the pixels included in the new macroblocks that were not included in the previous macroblock pixel a _{n + 7,} due to the one pixel further movement to the right, its value is the sum of orthogonal calculated previously in calculating the OS _NEW because be added to the OS _oLD and, although not included in the new macroblock is the only pixels the pixel a _n-1 that were part of the previous macro-block, due to the one pixel further movement to the right, and its value prior to calculating the OS _nEW Lt; RTI ID = 0.0 > _OLD . ≪ / RTI >

This process of updating the orthogonal OS _NEW value is repeated for each additional one pixel shift during the horizontal search until it reaches the boundary of the horizontal search range in the search area of the reference picture, The horizontal search is completed.

4, a block diagram of an orthogonal-sum generator 20 that constitutes an embodiment of the present invention can be seen. First, although the present invention is described using the example of 8x8 macroblocks, it should be understood that the present invention is not limited to macroblocks or pixel arrays of any particular size or structure. Although other hardware implementations of the method of the present invention will be readily apparent to those of ordinary skill in the art and are broadly encompassed by the present invention, the motion estimation method of the present invention may be applied to the orthogonal-sum generator 20 ).

First, a set of horizontal sums representing the sum of the individual pixel (luminance) values of the rows of the macroblock and a set of horizontal sums representing the sum of the individual (luminance) values of the rows of the macroblock in the manner described above with reference to Figs. 1A, 1B and 2 All orthogonal sum types of the currently encoded macroblocks are calculated by calculating a set of vertical sums representing the sum of pixel values.

Second, the pixel values for the macroblocked initial reference picture array (macroblock) having the specified origin in the specified search area of the reference picture stored in the reference picture (anchor) memory (not shown) are stored in the local memory 22 For example, a DRAM, an SRAM, or a shift register matrix). The anchor memory is preferably configured such that its output is always vertically adjacent. For example, if the outputs of the anchor memory output pixels from lines 1, 2, 3, and 4, the anchor memory may be moved from the lines (rows) 2, 3, 4, and 5 Pixels. This can be accomplished, for example, by suitably dividing this memory without increasing the anchor memory size using the method disclosed in the undisclosed international application PCT / IB99 / 00989 which includes the disclosure herein.

During the initialization procedure, the entire set of horizontal sums for the reference pixel array of the initial macroblock size are each stored in a set of parallel horizontal summation registers, each of which is coupled to the respective data outputs of the local memory 22 with a subtraction (- The vertical sums for each column of the initial reference pixel array are output by the four input vertical sum adder circuit 27 and the vertical sums calculated in this manner are successively stored in the shift register 29).

After this initialization process is completed, the motion estimation search method of the present invention operates as follows. In particular, as the motion estimation search progresses on a pixel-by-pixel basis in the horizontal direction (hereinafter referred to as " horizontal search ") through the specified search region of the reference pixel, the resulting reference pixel array is shifted by one pixel Will be moved accordingly.

After each pixel shift in the search area, the pixel values stored in the respective rows of the i-th column of the local memory 22 are read from the local memory 22 and applied to the subtraction inputs of the respective horizontal summation circuits 25 , The pixel value corresponding to the (N + i) -th column of the search area of the reference picture is written into each row of the i-th column of the local memory 22 so as to replace the pixel values just read from it, where i = N, and N is the horizontal size of the initial reference pixel array (i.e., the horizontal size of the coded macroblock). Preferably, after reaching N, i will again be the count value 1 for memory addressing purposes and will be incremented by one until it reaches N again, and the boundary of the horizontal search range (measured from the horizontal origin of the initial reference pixel array The cycle is repeated until the horizontal search is ended in this way. In this regard, a modulo-8 address counter (not shown) or any other suitable mechanism may be used for the purpose of performing this function.

Pixel values (hereinafter simply referred to as " new pixel values ") corresponding to the (N + i) -th column of the search area of the reference image are also added (+) of each horizontal summation circuit 25, Input and the vertical summation adder circuit 27, respectively. By way of example, if the local memory 22 is a DRAM, the above-described memory read and write operations may be performed during a single memory clock cycle through a read-modify-write operation.

When receiving the read pixel value and the new pixel value, each horizontal summing circuit 25 adds the new pixel value received at its summation input to the previously accumulated horizontal sum, Subtracts the value from the previously accumulated horizontal sum, and outputs the resulting sum as a " new " horizontal sum. That is, the set of horizontal sums output at the output of the horizontal summing circuit 25 will constitute a set of horizontal sums for the " new " reference pixel array shifted by one pixel from the reference pixel array of the previous iteration.

Further, after each pixel shift, the shift register 29 is horizontally shifted by one word to the right, so that the vertical sum stored at the last stage of the register is discarded, and the remaining vertical sums are shifted by one stage to the right. When receiving the new pixel value, the vertical sum adder circuit 27 generates at the output a " new " vertical sum that is loaded into the first stage of the shift register 29 (which is a N-word shift register) Replace the previous vertical sum. The resulting set of vertical sums appearing at the outputs of the shift register 29 constitute a set of vertical summations for the " new " reference pixel array shifted by one pixel from the reference pixel array of the previous iteration.

The above-described process is repeated after each pixel shift during the horizontal search through the search area of the reference picture until the horizontal search is terminated.

Referring to FIG. 5, a memory read / write operation sequence for horizontal search according to an embodiment of the present invention will be described. The circle represents the active address, and the rectangle represents the inactive address. A column with a (+) sign indicates RAM input (pixel number), and a column with a minus sign indicates RAM output (pixel number). In particular, for each row of the searched reference image search area, the first eight horizontally adjacent pixels 1-8 are stored in each of the four respective rows (sections) of the local memory 22, and at the same time the corresponding horizontal sum And is stored in the modifying circuit 25. At this time, the horizontal sum output by the horizontal summing circuit 25 is a valid horizontal sum for the initial reference pixel array (macroblock). Then, for each single pixel shift, as the horizontal search through the search area of the reference picture progresses, the address counter is incremented by one to designate pixel i, where i = 1 to 8 and the count value is the count end value (8). Thus, the previous pixel value for each row of the local memory 22 is read from the currently addressed position in the local memory 22 and applied to the subtract (-) input of each horizontal summation circuit 25, At the same time that a new pixel value for each row of the local memory 22 is written to the currently addressed location in the local memory 22, the sum (+) input of each horizontal summation circuit 25 and the 4- And applied to each input of the vertical sum adder circuit 27. Thus, after each single pixel shift, the updated total set of horizontal sums will be output by the horizontal summation circuit 25, and the updated vertical sum of the entire set will be output by the shift register 29. [

5, after the first eight pixels are written to the appropriate row of the local memory 22, the reference pixel array will be shifted by one pixel to the right, and the pixel number < RTI ID = 0.0 > 1 will be read from local memory 22 and replaced with pixel number 9 and then the reference pixel array will be shifted by one pixel to the right and pixel number 2 will be read from local memory 22 and replaced with pixel number 10 The reference pixel array will be shifted by one pixel to the right and pixel number 3 will be read from local memory 22 and replaced by pixel number 11 and then the reference pixel array will be shifted to the right by one pixel Pixel number 4 will be read from local memory 22 and replaced by pixel number 12 and then the reference pixel array will be shifted by one pixel to the right and pixel number 5 Will be read from local memory 22 and replaced with pixel number 13 and then the reference pixel array will be shifted by one pixel to the right and pixel number 6 will be read from local memory 22 and replaced by pixel number 14 , Then the reference pixel array will be moved by one pixel to the right and pixel number 7 will be read from local memory 22 and replaced by pixel number 15 and then the reference pixel array will be shifted by one pixel to the right Pixel number 8 will be read from local memory 22 and replaced by pixel number 16 and then the reference pixel array will be shifted by one pixel to the right and pixel number 9 is read from local memory 22 and pixel number 17 The reference pixel array will then be shifted by one pixel to the right, pixel number 10 is read from the local memory 22, 18, and finally the reference pixel array will be shifted by one pixel to the right and pixel number 11 will be read from local memory 22 and replaced with pixel number 19, and so on.

In FIG. 6, a block diagram of a field-based motion estimation search engine 40 that constitutes an embodiment of the present invention can be seen. As shown, the search engine 40 includes a field 1 orthogonal-sum generator 20a (as shown in Figure 2) and a parallel field 2 orthogonal-sum generator 20b (as shown in Figure 2) . The field 1 orthogonal sum generator 20a receives four new pixels from the field one anchor memory 45 on the parallel line 44 during each one pixel shift of the field one reference pixel array during the horizontal search operation and the field one The orthogonal generator 20b receives four new pixels from the field 2 anchor memory 47 via the parallel line 46 during each one pixel shift of the field 2 reference pixel array during the horizontal search operation. The field 1 orthogonal sum generator 50a receives the pixels of the currently field 1 macroblock to be coded (i.e., coded macroblock) from the field 1 coded picture memory 52, and the field 2 orthogonal-sum generator 50b Receives the pixels of the field 2 coded macro block from the field 2 coded image memory 54. [ The field 1 orthogonal sum generator 50a generates at its output the entire orthogonal sum set (both horizontal and vertical) representing the orthogonal sum type of the field 1 coded macroblocks, and the field 2 orthogonal sum generator 50b generates The entire set of orthonormal expressions representing the orthogonal sum type of the field 2 coded macroblock is generated at its output.

Continuing with Fig. 6, the search engine 40 receives the orthogonal sum type of the current reference pixel array in one set of inputs and the orthogonal sum type of the field 1 coded macroblock in the other set of inputs , Then determines which of the reference pixel arrays constitutes the best match for the coded macroblock from the specified search area of the field 1 anchor memory 45, according to the prescribed search metric (e.g., MAE) Field 1 best match estimator 60 that outputs the result as " Field 1 Motion Vector ". Similarly, the search engine 40 inputs the orthogonal sum type of the current reference pixel array in one set of inputs and the orthogonal sum type of the field 2 coded macroblocks in the other set of inputs, According to the search metric (e.g., MAE), it is determined which of the reference pixel arrays from the specified search area of the field 2 anchor memory 47 constitutes the best match for the coded macroblock, Field 2 best match estimator 60 that outputs the result as a " Field 2 Motion Vector ". For a more efficient design implementation, it will be readily seen that these RAMs can be combined to store data for both fields since the search engine RAMs are controlled in the same way for both fields.

As described above, the computational complexity of the DFCU is a major factor in the cost of the motion estimation circuit (search engine). However, since the motion estimation method of the present invention dramatically reduces the cost and complexity of the DFCU, it becomes much more practical to start with decimated video that is not decimated or low-level in the motion estimation search, Accuracy, and eventually dramatically improves picture quality. In this regard, the motion estimation method of the present invention not only significantly reduces the number of stages of motion estimation, but also removes all of the decimation stages to eliminate the special video filtering circuitry required. With this hardware reduction, the search process may begin with a video that is not decimated, which can significantly improve image quality at reasonable cost.

Another advantage realized by the motion estimation method of the present invention is greatly improved motion speed. Usually, a plurality of logical stages are required to compare the arranged luminance magnitudes, except in practice the possibility of obtaining results within a single clock cycle. For this reason, the system clock frequency must be reduced or the system must be pipelined using logical resources.

In addition to these advantages, the preferred embodiment of the present invention greatly accelerates the motion estimation method using the orthogonal-sum block matching described above. Furthermore, the present invention achieves the following three significant advantages over currently available techniques.

(1) substantial hardware reduction in orthogonal summing. As the macroblock moves within the anchor picture using the available sums, the orthogates are updated to produce new (updated) orthogonals, thus allowing much less computational effort that requires significantly less computational hardware.

(2) the operating speed is significantly accelerated by removing the long chain of adder circuits that produce orthogonal sum.

(3) The present invention requires that the output of all engine memories be immediately available for comparison, and that rather than using a large-scaled register matrix to store the search data as required by presently available techniques, It provides significant cost savings by allowing the use of RAM to store data.

(4) Due to the novel structure, the motion estimation search engine according to the present invention can be implemented in logic and memory integrated in a single silicon device using the latest memory technology, .

A preferred embodiment of the present invention is characterized in that during a motion estimation search in which the reference pixel array is shifted by one pixel in the horizontal search direction during each of a plurality of motion estimation search iterations, the values of N pixels included in the horizontal row of the reference pixel array And a RAM-based search engine that updates the horizontal sum representing the sum. The RAM-based search engine accumulates the values of the N pixels included in the horizontal row of the reference pixel array prior to any movement of the reference pixel array to generate a horizontal sum,

OS _NEW = OS _OLD - a ₀ + a _n0

Here, OS _NEW is a new horizontal sum after the final movement of the reference pixel array by one pixel in the horizontal direction, OS _OLD is the horizontal sum of the reference pixel array before the final movement by one pixel in the horizontal direction, and a ₀ is 1 Pixel is the pixel value of the pixel that was the horizontal origin of the reference pixel ery before the last movement of the reference pixel array by pixels and a _n0 is the pixel value of the right pixel of the reference pixel array as a result of the final movement of the reference pixel array by one pixel in the horizontal direction Which is a pixel value of a pixel which is the horizontal origin of the reference pixel array after the reference pixel array is shifted by one pixel, by updating the horizontal sum by calculating a new horizontal sum using the above equation.

Although the preferred embodiments of the present invention have been described in detail, many changes and / or modifications of the basic inventive concept as taught by those skilled in the art may be made by those skilled in the art, It will be within the scope of the present invention. For example, although the present invention has been described as being applicable to digital video encoders, the present invention is not limited to any particular application and may include, for example, receiving a received image to accommodate the requirements of a television or other image display system It will be apparent that when coding is needed, it can be used in the decoder portion of a television or other image display system. In the claims, any reference signs in parentheses shall not be construed as limiting the claim. The word " comprising " does not exclude the presence of elements or steps other than those listed in a claim. The ancestor " a or an " preceding an element does not exclude the presence of such plurality of elements. The present invention may be implemented by means of hardware, including some distinctive components, and by means of a suitably programmed computer. In a device claim enumerating some means, some of these means may be realized in one and the same hardware.

Claims (14)

A method of comparing a first pixel array having individual pixel values of a plurality of rows and columns and a second pixel array having individual pixel values of a plurality of rows and columns, Summing individual pixel values of individual pixel values of each row of the first pixel array to produce a first set of horizontal sums, Summing individual pixel values of individual pixel values of each column of the first pixel array to produce a first set of vertical sums, Summing individual pixel values of individual pixel values of each row of the second pixel array to produce a second set of horizontal sums, Summing individual pixel values of individual pixel values of each column of the second pixel array to produce a second set of vertical summations, Wherein the first set of horizontal sums and the first set of vertical sums comprise a first set of orthogonal sums, Summing individual pixel values of individual pixel values of each column of the second pixel array, wherein the second set of horizontal sums and the second set of vertical sums comprise a second set of orthogonal sums, And comparing the first and second sets of orthogonals. ≪ RTI ID = 0.0 > 11. < / RTI > 2. The apparatus of claim 1, wherein the first pixel array comprises undecimated macroblocks of an image to be currently coded, and the second pixel array comprises macroblocks that are not decimated in a search region of a reference image The first and second pixel array comparison methods. 2. The apparatus of claim 1, wherein the first pixel array comprises decimated macroblocks of the currently encoded picture, and the second pixel array comprises macroblocks decimated in a search region of a reference picture. A second pixel array comparison method. 2. The method of claim 1, wherein the first summing step includes updating a horizontal sum indicating a sum of values of N pixels included in a horizontal row of a reference pixel array during a motion estimation search, Calculating the horizontal sum, Moving the reference pixel array horizontally by one pixel, and To add a new pixel value to the previously calculated horizontal sum and to subtract the previous pixel value that is no longer included in the horizontal line of the reference pixel array after the shifting step from the previously calculated horizontal sum to generate a new horizontal sum And updating the horizontal sum. ≪ RTI ID = 0.0 > 11. < / RTI > 5. The method of claim 4, further comprising repeating the moving and updating steps until a boundary of the horizontal search range is reached. 5. The method of claim 4, The calculating step is performed by using a horizontal summing circuit 25 for accumulating the values of the N pixels included in the horizontal line of the reference pixel array before performing the shifting step, The step of updating the horizontal sum may comprise: OS _NEW = OS _OLD - a ₀ + a _n0 Where OS _NEW is a new horizontal sum, OS _OLD is the horizontal sum before the last iteration of the shifting step, and a ₀ is the pixel value of the pixel that was the horizontal origin of the reference pixel array before the final iteration of the shifting step and a _n0 is the pixel value of the pixel which is the horizontal origin of the reference pixel array after the reference pixel array is shifted by one pixel to the right with respect to the previous position of the reference pixel array as a result of the final iteration of the shifting step, Is performed by using the horizontal summation circuit (25) to calculate the new horizontal sum using the above equation. 2. The method of claim 1 further comprising: generating a horizontal sum for each N rows of the reference pixel array and for each iteration of a horizontal motion estimation search of a defined search area of a reference picture, And generating a vertical sum for < RTI ID = 0.0 > (a) storing M individual pixel values in each of the N rows of the memory 22 and storing N individual pixel values in each of the M columns of the memory 22, Storing initial pixel values corresponding to the pixel values, (b) calculating the horizontal sum for each of the N rows of the initial position of the reference pixel array and storing each of the calculated horizontal sums, (c) calculating the vertical sum for each of the M columns of the initial position of the reference pixel array, and storing the calculated vertical sums in a shift register (29) (d) moving the reference pixel array by one pixel in the horizontal direction, (e) in response to the moving step, i) providing N new pixel values, one for each of the N rows of each of the reference pixel arrays corresponding to the last column of the reference pixel array after being shifted by one pixel in the horizontal direction, ii) summing the N new pixel values to generate a new vertical sum, applying the new vertical sum to the shift register (29), adding the previously stored vertical sum to one word in the horizontal direction of the motion estimation search So that the first stored vertical sum is discarded and the new vertical sum stored at a previous storage location of the last stored vertical sum, iii) outputting a set of M new vertical summations from the shift register 29, iv) adding each of the N new pixel values to the previously calculated horizontal sum, and adding each previous pixel value that is no longer included in M columns of the reference pixel array after shifting by one pixel in the horizontal direction Updating each of the horizontal sums to produce a set of N new horizontal sums by subtracting the horizontal sum from the previously calculated horizontal sum for each N rows, v) outputting the set of N new horizontal sums. < RTI ID = 0.0 > 11. < / RTI > 8. The method of claim 7, wherein step (b) is performed using N horizontal summing circuits (25) corresponding to each of the N rows of the memory (22) 25) accumulates the values of the M individual pixel values stored in each row of the memory (22). A method for determining a best match between a first pixel array of a currently encoded image and a plurality of second pixel arrays of a search region of a reference image, the first and second pixel arrays comprising a plurality of pixels Values, the method comprising: And a set of first vertical sums representing a set of horizontal sums representing sums of individual pixel values of the rows of the first pixel array and a sum of individual pixel values of the columns of the first pixel array. Providing a first orthogonal sum signature of the first pixel array, A plurality of second orthogonal sum types, for each of at least selected ones of the plurality of second pixel arrays, a set of horizontal sums representing the sum of the pixel values of the respective rows of the respective second pixel arrays Providing a respective one of said plurality of second pixel orthogonality types consisting of a set of vertical sums representing the sum of the pixel values of the respective columns of pixels and the respective pixel values of the columns of each second pixel array, Sum type and each of the second orthogonal sum types to determine a best match between the first and second pixel arrays. A motion estimation apparatus for determining a best match between a first pixel array of an image to be currently encoded and a plurality of second pixel arrays of a search region of a reference picture, the first and second pixel arrays comprising a plurality of rows and a plurality The motion estimation apparatus comprising pixel values of individual columns, And a set of first vertical sums representing a set of horizontal sums representing sums of individual pixel values of the rows of the first pixel array and a sum of individual pixel values of the columns of the first pixel array. A plurality of second orthogonal sum types for each of the at least selected arrays of the plurality of second pixel arrays, each of the plurality of second orthogonal sum types providing a first orthogonal sum signature of the first pixel array, Each of said plurality comprising a set of horizontal sums representing sums of individual pixel values of rows of arrays and a set of vertical sums representing sums of individual pixel values of columns of each second pixel array, Means for providing second pixel orthogonal sum types of pixels, and Means for comparing each of said second orthogonal sum types with said first orthogonal sum type to determine a best match between said first and said second pixel arrays. 11. The apparatus of claim 10, wherein the first and second pixel arrays are respective macroblocks having a structure defined by an MPEG standard. 11. The apparatus of claim 10, further comprising a circuit (20) for updating a horizontal sum indicating a sum of values of N pixels included in a horizontal row of a reference pixel array during a motion estimation search, Means for calculating the horizontal sum, Means for moving the reference pixel array by one pixel in the horizontal direction, and Adding a new pixel value to a previously calculated horizontal sum, moving a reference pixel by one pixel in the horizontal direction, and then adding a previous pixel value that is no longer contained in the horizontal line of the reference pixel array to the previously calculated horizontal And means (25) for updating the horizontal sum to produce a new horizontal sum by subtracting from the sum. 11. The method of claim 10, further comprising generating a horizontal sum for each N rows of the reference pixel array, and for each iteration of the horizontal motion estimation search of the pre- Further comprising a circuit (20) for generating a vertical sum for the plurality of columns, the update circuit (20) Storing the M individual pixel values in each of the N rows of the memory 22 and storing N individual pixel values in each of the M columns of the memory 22 so that an initial The memory (22) for storing pixel values, Means (25) for calculating a horizontal sum for each of the N rows of the initial position of the reference pixel array and for storing each of the calculated horizontal sums, Means (27) for calculating the vertical sum for each of the M columns of the initial position of the reference pixel array, A shift register 29 for storing the calculated vertical sums, Means for moving the reference pixel array by one pixel in the horizontal direction, In response to each movement of the reference pixel array by one pixel in the horizontal direction, i) providing N new pixel values, one for each of the N rows of each of the reference pixel arrays corresponding to the last column of the reference pixel array, after shifting by one pixel in the horizontal direction, ii) summing the N new pixel values to generate a new vertical sum, applying the new vertical sum to the shift register (29), adding the previously stored vertical sum to one word in the horizontal direction of the motion estimation search So that the first stored vertical sum is discarded and the new vertical sum is stored in the previous storage position of the last stored vertical sum, iii) outputting a set of M new vertical summations from the shift register 29, iv) adding each of the N new pixel values to the previously calculated horizontal sum, and adding each previous pixel value that is no longer included in M columns of the reference pixel array after shifting by one pixel in the horizontal direction For each of the N rows, from the previously calculated horizontal sum, thereby generating a set of N new horizontal sums, v) means (25, 27, 29) for outputting the set of N new horizontal sums. 11. The method of claim 10, further comprising: during a motion estimation search in which the reference pixel array is shifted by one pixel in the horizontal search direction while repeating a plurality of the motion estimation searches, (20) for updating the horizontal sum indicating the sum of the values of the reference pixel array, wherein the updating circuit (20) updates the values of N pixels included in the horizontal row of the reference pixel array prior to any movement of the reference pixel array Accumulate to generate the horizontal sum, OS _NEW = OS _OLD - a ₀ + a _n0 Wherein OS _NEW is the new horizontal sum after the last movement of the reference pixel array by one pixel in the horizontal direction, OS _OLD is the horizontal sum before the final movement of the reference pixel array by one pixel in the horizontal direction, and a ₀ is the pixel value of the pixel which was the horizontal origin of the reference pixel array before the final movement of the reference pixel array by one pixel in the horizontal direction, and a _n0 is the pixel value of the last movement of the reference pixel array by one pixel in the horizontal direction. Which is the pixel value of the pixel that is the horizontal origin of the reference pixel array after the reference pixel array is shifted by one pixel to the right with respect to the previous position of the reference pixel array as a result And a horizontal summation circuit (25) for updating the horizontal sum.