KR100195694B1 - Motion estimation method and apparatus - Google Patents

Abstract

The present invention relates to a method and apparatus for estimating motion, using a motion estimation chip configured to calculate motion vectors and distortions after processing in a predetermined search window area of a 4 x 8 or 8 x 16 pixel reference block. A method for processing a 4 x 16 or 8 x 8 pixel sized reference block in pipeline mode, wherein the 8 x 4 or 16 x 8 reference blocks form a slice, and the slices are arranged in a planar manner from a stored current frame memory. The reference blocks are read in the vertical direction, sequentially, and then rotated to form a 4 x 8 or 8 x 16 sized reference block so that the rotated reference blocks have continuity and are thus processed in pipeline mode in the motion estimation chip. To be able.

Description

Method and device for estimating whirlwind

(A) of FIG. 1 shows a search window of an MxN block;

(B) of FIG. 1 shows an MxN block;

2 is a block diagram of a motion estimation commercial chip used in the present invention;

(C) of FIG. 3 shows a search window of a 4n x 8 reference block by a commercial chip of FIG.

(B) of FIG. 3 shows a 4n x 8 reference block,

(A) to (f) of FIG. 4 are timing charts showing the timing of the initialization sequence by the commercial chip of FIG.

(A) to (f) of FIG. 5 are timing charts showing the timing of the block sequence by the commercial chip of FIG.

(A) to (g) of FIG. 6 are timing charts showing the output timing by the commercial chip of FIG.

7 is a schematic diagram showing an input step in a pipeline mode according to a commercial chip of FIG.

(A) of FIG. 8 shows a block of the vertical 1/2 mode used in a general field frame;

(B) of FIG. 8 shows a conventional block arrangement for performing a general vertical 1/2 mode;

(A) of FIG. 9 shows a block arrangement for performing a vertical 1/2 mode according to the present invention;

(B) of FIG. 9 shows a block arrangement formed by rotating the block shown in (a) of FIG.

10 is a block diagram illustrating a motion estimator according to the present invention.

* Explanation of symbols for main parts of the drawings

1: current frame memory 2: previous frame memory

3: first data generator 4: second data generator

5: motion estimator 6: controller

The present invention relates to a technique for estimating motion of an image signal, and more particularly, to a motion estimation method and apparatus for implementing 'vertical half mode' using a commercial chipset capable of supporting 'horizontal half mode'. will be.

In general, 'motionestimation' in a video signal processing technique is a vector representation of how much the pixels of a current frame move compared to a previous frame in a continuous video signal. Instead of estimating a motion vector and transmitting the entire image, it refers to a technique (ie, image compression) that reduces transmission information by transmitting these motion vectors.

The Block Matching Algorithm (BMA), which is widely used in such motion estimation, assumes that the screen motion is moved horizontally or vertically in parallel to the block image at any position of the frame (ie, the previous frame) where the motion occurred. It is a method of estimating the best match and estimating a motion vector through the position. In this case, 8x8, 16x16 (the number of horizontal pixels x the number of vertical pixels) are mainly used as the size of the block.

Here, in order to find the previous block that is most similar to the reference block of the current frame, a search is performed within a predetermined range around the position of the reference block in the previous frame. Such a range is called a search window. The difference between each candidate block within the block is called distortion, and indicates the degree of similarity between the two blocks.

In addition, comparing all candidate blocks and reference blocks in the search window is called a full search block matching algorithm, and in order to find the best matching block, the previous frame is searched in the image of two consecutive frames. f₁ (x, y), assuming that the current frame is f₂ (x, y), f₁ (xa, yb) and f₂ (by changing a, b in f₂ (x, y) and f₁ (xa, yb) The difference of x, y) is obtained, and (a, b) for which the difference is minimum is predicted by the motion vector. In this way, the method of finding the minimum error between blocks is called Mean Absolute Error (MAE) method.

In the above equation, E _abs is the mean absolute error value, B is the block size, so if you find (a, b) that minimizes E _abs , this is the motion vector.

On the other hand, the motion estimation technique as described above is implemented as a semiconductor chip is STI3220 manufactured by SGS-THOMSON, the STI3220 is M x N (M) as shown in (b) of FIG. The reference block of size = 8 degrees (16, N-4n) has a search window of -8 to +7 in the horizontal direction and -8 to +7 in the vertical direction as shown in (a) of FIG.

As shown in FIG. 2, the commercial chip includes data consisting of an X port for receiving a reference block in 8-bit pixel units and an A, B, and C port for receiving search window data in 8-bit pixel units, respectively. A formatter; 256 processor arrays; Distortion (RAM) for storing 16-bit distortion; A distortion comparator for comparing the distortions; A distortion comparison result, the motion vector having the minimum distortion, an address for accessing the distortion RAM, and an input / output bus 10B for delivering the distortion; And / RESET, OE / BSYNC, EC [3: 0], H8, SQ signals to control various operations and / MVSYNC, / DSYNCO. It is composed of a controller that generates timing of the / DSYNC1 signal. Here, the RESET signal is a signal for resetting a commercial chip, OE is an output enable signal, / BSYNC is a timing signal requiring the start of a sequence, and EC [0: 3] is 4 for edge control. Bit is a data signal, / H8 means that the vertical size of the block is 8 pixels (ie 4n x 8), 1 means that the vertical size of the block is 16 pixels (ie 4n x 16), and the / SQ signal 0 is a signal for indicating that the block size is square (8 x 8 or 16 x 16), and 1 is rectangular.

The 4n x 8 reference block of the commercial chip constructed as described above has a rectangular shape consisting of 4n pixels in the horizontal direction and 8 pixels in the vertical direction, as shown in FIG. 3 (b). As shown in (a) of FIG. 3, it is divided into SWA1, SWC1, SWA2, SWB2, and SWC2 sub-windows formed in eight rows and fifteen columns. In the initialization sequence, SWA1 is input to A port, SWB1 to B port, SWC1 to C port, SWA2 to A port, SWB2 to B port, and so on. SWC2 is input to C port and reference block is input to X port, respectively.

That is, the SI3220 motion estimation chip has an operation mode consisting of an initialization sequence and a block sequence. The initialization sequence includes 15 columns of SWA1, 128 during 128 clock cycles as shown in (a) to (g) of FIG. Loads SWB1 and SWC1 blocks. At this time, the sequence starts when the BSYNC signal goes low for about one clock, and the chip is initialized internally without data loading for the first eight clocks, and then 120 pixels (8 x 15 = 120) per port for 120 clocks. Load the data of). The order in which data is loaded is from the first column to the 15th column sequentially while loading in the vertical direction. At this time, SWA1 and SWB1 are 8 pixels in the vertical direction, but SWC1 and SWC2 are 7 pixels, but 8 pixels are input to match the timing. Thus, as shown by a dotted line in FIG. And excluded from motion vector calculations.

The block sequence also loads SWA2, SWB2, SWC2 and the reference block during 32n (4n x 8 = 32n) clocks, as shown in Figures 5 (a) to (f), where the / SQ signal is referenced as low. If the block is square (eg 8x8, 16x16), it is not essential that the / BSYNC signal goes low at the beginning of the sequence, but if the reference block is not square (/ SQ = high), the BSYNC signal goes low at the beginning of the sequence. It is essential to be. After the reference block is loaded as described above, distortion calculation is performed on each candidate block, and the motion vector and the distortion of the candidate block having the minimum value are compared with each other to obtain the timing shown in FIGS. 6A to 6G. It is output through I / O bus (IOB).

That is, in the case where the output enable signal OE is high as shown in (c) of FIG. 6 in the M + 30th clock cycle of the next sequence after the block sequence is performed, FIG. When the / MVSYNC signal goes low, an 8-bit motion vector (MV) is loaded on the I / O bus, followed by the upper 8 bits (MSB) of the 16-bit distortion at the timing when / DSYNCO goes low. Immediately following, the lower 8 bits (LSB) of the distortion are output at the timing when / DSYNC1 goes low.

As such, during the block sequence, the motion vector and the distortion value of the previous block sequence can be read through the I / O bus, and a specific address is output to the I / O bus, and the distortion RAM of the corresponding address is accessed to generate the distortion data. You can read it.

As described above, in order to estimate the motion of one reference block in a search window having a size of -8 to +7 in each of the horizontal and vertical directions, an initializing sequence of 128 clocks and a block sequence of 32n clocks must be performed once. do. However, when the continuity between blocks is maintained, that is, when the right 15 column of the kth search window and the left 15 column of the k + 1 search window coincide with each other, a plurality of reference blocks are generated by only one initialization sequence and a plurality of block sequences. Since the motion estimation can be performed for, time can be saved, and this operation mode is called a pipeline mode.

When the motion estimation chip is operated in the pipeline mode, one search window is divided into three stages as shown in FIG. 7. The blocks of the first and second stages are already input blocks, and the third The blocks in the step are blocks entered together with the reference block in the current search window (k search window). As described above, when the motion vector is calculated, the blocks input in the third stage of the k search window are shifted by one block and become blocks of the second stage in the k + 1 search window.

Meanwhile, in MPEG2, motion estimation is possible not only for frame pictures but also for field pictures. When motion estimation is performed in 8 x 8 block units in a frame picture, it is necessary to perform motion estimation in 8 x 4 block units in a field picture. have. As described above, a method in which the pixel is reduced in half with respect to the frame picture in the vertical direction in the field picture is referred to as vertical 1/2 mode.

As described above, the STI3220 chip supports horizontal 1/2 mode when n = 1 at 4n x 8 or 4n x 16, but not vertical 1/2 mode. In other words, the STI3220 chip processes a 4 x 8 sized reference block with a default search window of -8 to +7 in the horizontal and vertical directions into an initialization sequence of 128 clock cycles and a block sequence of 32 clock cycles.

Therefore, in order to support the vertical 1/2 mode as a commercial chip supporting only the horizontal 1/2 mode, 8 x 4 blocks should be rotated and converted into a 4 x 8 block shape before being input to the chip.

For example, as shown in (a) of FIG. 8, the first slice is composed of blocks 1-1, blocks 1-2, blocks 1-3, blocks 1-4, ... of 8 x 4, The second slice consists of blocks 2-1, blocks 2-2, blocks 2-3, blocks 2-4, ... of 8 x 4, and the third slice is blocks 3-1, block 3 of 8 x 4 -2, block 3-3, block 3-4, ..., since the data of these blocks are sequentially taken in the horizontal direction and rotated, and then processed by the commercially available chip as shown in FIG. As shown in FIG. 2), there is a problem that the continuity between adjacent blocks is broken and thus the pipeline mode cannot be used.

That is, as shown in (b) of FIG. 8, if blocks 1-1, blocks 1-2, blocks 1-3, etc. are sequentially rotated, the blocks as shown in (b) of FIG. 8 are rotated. Since the successive parts (bold lines a, b) between 1-1 and blocks 1-2 are separated from each other, and the non-contiguous parts (c) are adjacent to each other, pipeline mode cannot be used. Because it has to repeat the block sequence with, it takes a lot of processing time.

Accordingly, the present invention has been made in order to solve the above-mentioned problems, and it is possible to operate in a pipelined mode while supporting a vertical 1/2 mode using a motion estimation chip that supports only a horizontal 1/2 mode. Its purpose is to provide a motion estimation method and apparatus.

In order to achieve the above object, the method of the present invention uses a motion estimation chip configured to calculate a motion vector and distortion after processing a 4 x 8 or 8 x 16 pixel reference block in a predetermined search window area. A method for processing 8 x 4 or 16 x 8 pixel sized reference blocks in pipeline mode,

The 8x4 or 16x8 reference blocks form a slice, and the slices are arranged in a plane and read the reference blocks in a vertical direction sequentially from the current frame memory stored and then rotated in a 4x8 or 8x16 size. Make it a reference block

The rotated reference blocks have continuity, and thus can be processed in pipeline mode in the motion estimation chip.

Hereinafter, with reference to the accompanying drawings will be described in detail the present invention.

In the motion estimation method according to the present invention, as shown in (a) of FIG. 8, 8 x 4 or 16 x 8 reference blocks form a slice, and the slices are arranged in a plane and stored in the current frame memory of FIG. As shown in Fig. 2), the reference blocks are sequentially read in a vertical direction, and then rotated to form a 4 x 8 or 8 x 16 sized reference block so that the rotated reference blocks have continuity, and thus the motion estimation chip To allow them to be processed in pipeline mode.

That is, the present invention uses a commercial chip that supports only the horizontal 1/2 mode (when the horizontal direction is 1/2 of the vertical direction, such as 4 x 8 and 8 x 16, as described in the prior art). To support the vertical 1/2 mode (when the size of the vertical direction is 1/2 of the horizontal direction such as 8 x 4 and 16 x 8), which is often used for motion estimation of field pictures. As shown in FIG. 8A, the first slice is followed by 8 × 4 blocks 1-1, block 1-2, block 1-3, block 1-4, ... 2 slices lead to 8 x 4, Block 2-1, Block 2-2, Block 2-3, Block 2-4, ..., and 3rd slice to 8 x 4, Block 3-1, Block 3- 2, block 3-3, block 3-4, ..., block 1-1, block 2-1, block 3-1 ..., block as shown in FIG. 1-2, block 2-2, block 3-3, ..., then read in the vertical direction and shown in FIG. Thus, the presented by rotating 90 ° 4 x 8 blocks described, and thus a continuous part (d) will occur between the rotated block and also trucks.

As shown in FIG. 10, the apparatus for implementing the method of the present invention comprises a current frame memory 1 which receives image data and stores data of the current frame picture; A previous frame memory (2) for storing data of a previous frame by delaying data of a picture input from the current frame memory (1); A first data generator (3) for inputting and rotating a reference block of a predetermined size from the current frame memory (1); A second data generator (4) for inputting and rotating search window data of a predetermined size from the previous frame memory (2); A motion estimator (5) which receives pixel data of a reference block from the first data generator (3), receives pixel data of a search window from the second data generator (4), and calculates a motion vector and a distortion; And controlling the first and second data generators 3 and 4 to convert the data format of the vertical 1/2 mode to the data format of the horizontal 1/2 mode and to estimate the motion for a predetermined reference block. It consists of a controller 6 that controls the estimator 5.

In the apparatus of the present invention configured as described above, the motion estimator 5 is implemented by STI's STI3220 chip, and the operation of the chip receives an 8-bit reference pixel through the X port, as described in the related art. After inputting pixel data corresponding to the search window area of the reference block to the previous frame picture with port A, port B, and port C, performing initialization sequence and block sequence to calculate a motion vector and distortion, and then input / output (IOB). Output through the port.

At this time, in the current frame memory 1, 8 X 4 blocks and slices formed in succession on a horizontal line are arranged and stored as shown in (a) of FIG. 8, and the first data generator 3 Reads blocks from the current frame memory 1 in the vertical direction and rotates them to form 4 x 8 blocks. Here, since the continuous blocks are formed between the rotated blocks as shown in FIG. 9B, the pipelined mode supported by the motion estimator 5 can be used.

In addition, the second data generator 4 receives pixel data corresponding to a search window according to a specific reference block in the previous picture from the previous frame memory 2 and supplies the pixel data to the motion estimator 5, similarly to the first data generator. Convert the data format of the vertical 1/2 mode to the data format of the horizontal 1/2 mode. As such, converting the data format in the first data generator and the second data generator is possible by changing the order of reading the pixel data stored in the memory.

As described above, the motion estimation method and apparatus according to the present invention can support the vertical 1/2 mode which is frequently used in the motion estimation of the field picture using a commercial chip that supports only the horizontal 1/2 mode. In particular, since it can be processed in pipeline mode without breaking the continuity of data, it can improve the processing speed of motion estimation.

Claims (2)

A reference block of 8 x 4 or 16 x 8 pixels using a motion estimation chip that processes a 4 x 8 or 8 x 16 pixel reference block in a given search window area and then calculates motion vectors and distortion. A method for processing them in a pipelined mode, wherein the 8 x 4 or 16 x 8 reference blocks form a slice, and the slices are arranged in a planar manner to sequentially read the reference blocks in a vertical direction from a stored current frame memory. And turning on and then rotating the reference block to a size of 4 x 8 or 8 x 16 so that the rotated reference blocks can be processed in pipeline mode in the motion estimation chip. A current frame memory 1 which receives image data and stores data of a current frame picture; A previous frame memory (2) for storing data of a previous frame by delaying data of a picture input from the current frame memory (1); A first data generator (3) for inputting and rotating a reference block of a predetermined size from the current frame memory (1); A second data generator (4) for inputting and rotating search window data having a predetermined size from the previous frame memory (2); A motion estimator (5) which receives pixel data of a reference block from the first data generator (3), receives pixel data of a search window from the second data generator (4), and calculates a motion vector and a distortion; And controlling the first and second data generators 3 and 4 to convert the data format of the vertical 1/2 mode into the data format of the horizontal 1/2 mode, and to estimate the motion with respect to a predetermined reference block. A motion estimating device comprising a controller (6) for controlling an estimator (5).