KR101253917B1 - Apparatus and method for estimating of motion in a moving picture - Google Patents

Abstract

The present invention relates to a motion estimation apparatus and a method thereof, wherein a motion estimation apparatus according to an embodiment of the present invention blocks a plurality of image frames, and includes a macroblock included in a first image frame as a reference image frame and the first image frame. A probability density function estimator for estimating a probability density function of a motion vector representing a difference between macroblocks included in a second image frame that is an image frame after one image frame, and the probability density function is preset in the first image frame And a search position setting unit for setting a position having a value greater than a threshold as a search position, and a motion vector estimator for estimating the motion vector at the set search position. According to the present invention, after estimating a probability density function indicating which value the motion vector has a high probability, the motion vector can be estimated more quickly by searching only a high probability position.

Description

A motion estimation device and its method {APPARATUS AND METHOD FOR ESTIMATING OF MOTION IN A MOVING PICTURE}

The present invention relates to a motion estimation apparatus and a method thereof, and more particularly, to a motion estimation apparatus and estimation method for estimating a probability density function of a motion vector from a given image, and quickly finding the motion vector using the same.

Video data used in video conferencing, high-definition television, Video On Demand (VOD) receivers, personal computers that support Moving Picture Experts Group (MPEG) video, game consoles, terrestrial digital broadcast receivers, digital satellite broadcast receivers, and cable television Since the amount of data is greatly increased in the process of digitizing the analog signal, it is compressed through an appropriate compression method. A video is composed of a plurality of frames, and similarities exist between neighboring frames or between neighboring blocks or pixels within the same frame due to characteristics of an image signal. The video signal compression is performed with high efficiency by predictively encoding the video using the similarity of the video signals.

In general, there are three types of compression methods for image data, namely, a method of reducing temporal redundancy, a method of reducing spatial redundancy, and a method of reducing data using statistical characteristics of generated codes. It belongs to this. Among the methods of reducing temporal redundancy, motion estimation and compensation methods are adopted in most video compression standards such as MPEG and H.263. The motion estimation method plays a big role in reducing bit rate by eliminating temporal redundancy in video encoding, which includes pixel matching and block matching, and because motion is expressed in large blocks Unit estimation methods are widely used.

The block-based estimation method divides an image into blocks of a certain size and finds a block that best matches the block of the current image in the search region of the previous image. The motion estimation is done by calculating and coding it. Various matching functions may be used to calculate a match between two blocks. The most commonly used function is sum of absolute difference (SAD), which is a sum of absolute values of differences between pixels between two blocks. An H.264 video encoding apparatus uses a cost function based on rate-distortion optimization instead of the conventional SAD-oriented search method.

In particular, in order to simultaneously obtain extrusion efficiency and high image quality in the H.264 method, in contrast to the conventional video encoding, encoding is performed in units of 16 × 16 large blocks or 8 × 8 large blocks. It is configured to select a mode having the minimum value among each block. Therefore, a large gain in hatching efficiency is provided through variable block-based motion estimation of various sizes. In addition, the H.264 method may improve the coding efficiency by performing motion vector prediction on a quarter pixel basis with more accurate motion estimation.

However, when using 1/4 pixel search compared to other coding schemes, coding efficiency can be improved, but at least 16 times as many motion estimation operations must be performed, which is an important factor of speed reduction in H.264 video coding. . Therefore, efforts have been made to reduce the motion estimation time, which takes the longest encoding time.

In addition to motion picture compression, motion estimation is also used to convert the frame rate of a video. After estimating the motion between two consecutive images, the estimated motion can be compensated to produce an image between the two images. For example, when receiving an image composed of 60 images per second as an input, inserting the created image may output a video composed of 120 images per second. In order to apply the motion estimation and compensation technique to the frame rate conversion, the estimated motion vector must represent the motion of the real object. Since the motion estimation method used for image compression is intended to minimize the error remaining after motion compensation, the estimated motion vector often does not represent the motion of an actual object. Thus, efforts have been made to estimate motion vectors representing the motion of real objects.

It is an object of the present invention to provide a motion estimation apparatus and method for rapidly estimating a motion vector representing a motion of a real object.

According to an embodiment of the present invention, a motion estimation apparatus blocks a plurality of image frames to include a macroblock included in a first image frame, which is a reference image frame, and a second image frame, which is an image frame after the first image frame. A probability density function estimator for estimating a probability density function of a motion vector indicating a difference between included macroblocks, and setting a position having a value greater than a predetermined threshold in the first image frame as a search position A search position setting unit and a motion vector estimating unit estimating the motion vector at the set search position.

In addition, the probability density function estimating unit uses the motion vector of the second image frame as a state variable, and has a difference between the macroblock included in the first image frame and the macroblock included in the second image frame as an observation value. A particle filter may be used to estimate the probability density function of the state variable according to the observation value.

The probability density function estimator may estimate the probability density function based on the macroblocks adjacent to the macroblocks when the macroblocks are smaller than the moving object.

The probability density function estimator may further include a particle initializer for initializing particles including the state variable and a weight, and update the state variable and include the macroblock included in the first image frame and the second image frame. A particle filtering unit for calculating the distance between the macroblocks to be used as the observation value, calculating the similarity of the distances between the macroblocks, and increasing the weight to filter the particles, and a reference value in which the similarity level of the filtered particles is preset If higher, it may include a particle resampling unit for resampling the particles.

In addition, the search position setting unit may calculate a state variable having a value greater than the threshold value of the probability density function and set the search position.

The motion vector estimating unit may determine a macro block that is determined to move according to the state variable among macro blocks of the first image frame, and determines a difference between pixels between the determined macro block and the macro block of the second image frame. The motion vector can be estimated using the sum of the absolute values.

According to another exemplary embodiment of the present invention, a motion estimation method blocks a plurality of image frames and includes them in a macro block included in a first image frame, which is a reference image frame, and in a second image frame, which is an image frame after the first image frame. Estimating a probability density function of a motion vector indicating a difference between macroblocks to be used, setting a position having a value greater than a predetermined threshold value in the first image frame as a search position, and setting the search position; Estimating the motion vector at the search position.

According to the present invention, after estimating a probability density function indicating which value the motion vector has a high probability, the motion vector can be estimated more quickly by searching only a high probability position. In addition, since the probability density function represents a probability that the motion vector has a certain value, the motion vector closer to the actual motion can be estimated than the motion estimation method of minimizing the error after the motion compensation.

1 is a block diagram of a motion estimation apparatus according to an embodiment of the present invention;
2 is a flowchart of a motion estimation method according to FIG. 1;
3 is a detailed flowchart of a probability density function estimating step of the motion estimation method according to FIG. 2;
4 is an exemplary diagram for describing a probability density function estimation in the motion estimation method according to FIG. 2.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The terms used are terms selected in consideration of the functions in the embodiments, and the meaning of the terms may vary depending on the user, the intention or the precedent of the operator, and the like. Therefore, the meaning of the terms used in the following embodiments is defined according to the definition when specifically defined in this specification, and unless otherwise defined, it should be interpreted in a sense generally recognized by those skilled in the art.

1 is a block diagram of a motion estimation apparatus according to an embodiment of the present invention, and FIG. 2 is a flowchart of a motion estimation method according to FIG. 1.

Referring to FIG. 1, the motion estimation apparatus 100 includes a probability density function estimator 110, a search position setting unit 120, and a motion vector estimator 130. In addition, the probability density function estimator 110 specifically includes a particle initializer 111, a particle filter 112, and a particle resampling unit 113. The probability density function estimator 110 blocks a plurality of image frames and includes them in a macro block included in the first image frame, which is a reference image frame, and in a second image frame, which is an image frame after the first image frame. The probability density function of the motion vector representing the difference between the macroblocks is estimated (S210). Each image frame is divided into several macro blocks, a first image frame is a reference image frame according to a user setting, and a second image frame is a current image frame.

The image sequence may be defined as f (i, j, t), i is a vertical index, j is a horizontal index, and t is a time index. The size of an image frame at a specific time may be represented by [I × J]. If the size of a macro block used for motion estimation is [b _i × b _j ], the image is converted into [M × N] macro blocks. Can be divided. In this case, when M = 1 / b _i and N = 1 / b _j , and the index of the macro block for which the current motion is to be estimated is k, a relationship is formed between the space-time index and the block index as shown in Equation 1 below. do.

In addition, the probability density function estimator 110 uses the motion vector of the second image frame as a state variable, and measures the difference between the macroblock included in the first image frame and the macroblock included in the second image frame. We can estimate the probability density function of the state variable according to the observation value by using the particle filter with. If the state variable representing the movement of the macroblock is x _k , the state variable x _k can be represented by (v _i , v _j ) when the moving motion model is used, in which case v _i is the vertical movement and v _j is Indicates the movement in the horizontal direction. In addition, when using the affine model, the state variable x _k can be expressed as (v _i , v _j , r _i , r _j , θ), and the affine model is located after the transformation. (i ', j') and the position before conversion (i, j) are formed as shown in the following equation (2).

The distance between the (m, n) macroblock in the second image frame f (i, j, t) that is the current image frame and the macroblock of the first image frame f (i, j, t-1) that is the reference image frame The state variable can be expressed as in Equation 3 below.

In Equation 3, i _k = (m−1) b _i , j _k = (n−1) b _j , and i _k , j _k represent indexes of macro blocks of the second image frame. Also, (i ', j') and (i, j) are related by a motion model such as Equation (2).

In Equation 4, B is a set of indices in one macro block. When (i ', j') is not an integer, the pixel value is found by interpolation using a bilinear equation. The state function x _k including the motion vector is evolved as shown in Equation 5, and thus the observation z _k is expressed as shown in Equation 6.

In Equations 5 and 6, G and H are nonlinear functions, and v _k and n _k are process noise and observation noise, respectively. Given the observation z _{1 :} k up to the macroblock index k, the conditional probability of the value of the state variable is p (x _{k |} z _{1 : k} ).

According to another embodiment of the present invention, when the size of the macro block is smaller than the moving object, the probability density function estimator 110 may estimate the probability density function based on neighboring macro blocks. have. It is assumed that the movement of the macro block in the second image frame is similar to the movement of adjacent macro blocks, and the state of the target macro block is modeled depending on the adjacent macro blocks. In this case, the probability density function may be expressed as Equation 7 below.

In Equation 7, π _c is a mixing probability, and N (x _k ; x _c , σ _c ² ) represents a normal distribution function of mean x _c and variance σ _c ² . The mixing probabilities π _c and variance σ _c ² are set constant for all mixing densities. C (k) is a set composed of indexes of macro blocks around space-time in the k-th macro block, which can be expressed by Equation 8 below.

The state variable model may be expressed as Equation 9 below.

In Equation 9, γ is a result of using a coin toss algorithm having the same number of planes as the number of elements of the set C. Also, the nonlinear function g (X _{1 : k-1} ; γ) selects one of the neighboring states due to the result of the coin throwing γ as shown in Equation 10 below.

Meanwhile, the particle filter extracts a plurality of samples having prediction values according to the position and the direction angle of the macro block from the image frame, and uses the probability that each sample has the position and the direction angle of the macro block, thereby optimizing the position of the macro block. Estimate In this case, the conditional probability p (x _{k |} z _{1 : k} ) can be estimated by using the following Equations 11 and 12 repeatedly.

Hereinafter, the probability density function estimation unit 110 will be described with reference to FIG. 3 with respect to estimating the probability density function of the state variable according to the observation value using the particle filter.

3 is a detailed flowchart of a probability density function estimating step of the motion estimation method according to FIG. 2.

Referring to FIG. 3, first, the particle initialization unit 111 initializes a particle including a state variable and a weight (S310). The particle initializer 111 initializes the initial weight w ₀ ² and the state variable x ₀ ² associated with the particle S. The initial value of each element of the state variable is evenly distributed at the search position. Thus, particles are initially uniformly distributed in the search area. The initial weight w ₀ ² is set to 1 / S, and the initial state variable x ₀ ² is [w _1, w _{2 ,... ,} w _L ] ^T. In this case, x _i is set from R _l × U (-1, 1), U (a, b) represents the uniform distribution of [a, b], and R _l represents the search region of the l-th state variable. .

)를 추정하게 된다.Next, the particle filtering unit 112 updates the state variable initialized by the particle initialization unit 111 and sets the distance between the macroblock included in the first image frame and the macroblock included in the second image frame as an observation value. The particle is calculated by calculating the similarity of the distance between the macroblocks and increasing the weight (S320). For example, when particle filtering is performed on the k-th macro block, the state variable and weight of the k-th macro block are updated, and the parameter of the k-th macro block (

) Is estimated.

의 식을 이용하여 업데이트한다. 다음으로,

에 의해 지정된 제1 영상 프레임의 마크로 블록을 가져온다. 다음으로, 제1 영상 프레임에서 가져온 마크로 블록과 제2 영상 프레임의 k 번째 마크로 블록 간의 거리를 계산한다. 다음으로, 마크로 블록 간의 거리p(z_k│x_k ^s)의 유사성을 계산한다. 다음으로, 가중치를

을 이용하여 업데이트한다.The particle filtering unit 112 may use, for example, the following algorithm. First, a set C consisting of indices of macro blocks around space-time in the k-th macro block is determined using a coin toss algorithm. Next, particle S

Update using the equation. to the next,

The macro block of the first video frame designated by the fetch is obtained. Next, the distance between the macroblock taken from the first image frame and the k-th macroblock of the second image frame is calculated. Next, the similarity of the distance p (z _{k |} x _k ^s ) between macroblocks is calculated. Next, the weight

Use to update.

을 이용하여 파티클 S_E(K)의 값을 계산하고, S_E(K)가 기준 파티클 값인 S_T보다 작으면, 가중치(w_k ^s)의 누적분포함수(CDF)인 W_k ^s을 계산한다. 다음으로, [0, 1]의 균일 분포인 U(0, 1)로부터 구한 r을 이용하여, 가중치(w_k ^j)가 r보다 같거나 크도록 하는 j 값을 구한다. 다음으로, 리샘플링된 상태변수 x'_k ^s를 X_k ^j로 설정하고, 리샘플링된 가중치 w'_k ^s를 1/S로 설정한다.
Next, the particle resampling unit 113 determines whether the similarity level of the filtered particles is higher than a preset reference value (S330). If the similarity level is lower than the reference value, the weight w _k ^s is normalized, the parameter of the k-th macroblock is estimated, and if the similarity level is high, the particles are resampled (S340). For example, the resampling algorithm can be implemented as follows. first

Calculate the value of particle S _E (K), and if S _E (K) is less than the reference particle value S _T , calculate W _k ^s , which is the cumulative distribution function (CDF) of the weight (w _k ^s ). . Next, using r obtained from U (0, 1), which is a uniform distribution of [0, 1], j is obtained so that the weight w _k ^j is equal to or greater than r. Next, the resampled state variable x ' _k ^s is set to X _k ^j , and the resampled weight w' _k ^s is set to 1 / S.

Referring back to FIG. 2, the search position setting unit 120 next sets a position having a probability density function greater than a preset threshold value in the first image frame as the search position (S220). In this case, the threshold may be set differently by the user's setting. The search position setting unit 120 calculates a state variable having a probability density function estimated by the probability density function estimator 110 having a value greater than a threshold value, and sets a search position from the calculated state variable. The search position setting unit 120 outputs the set search position information to the motion vector estimator 130.

Next, the motion vector estimator 130 estimates the motion vector at the search position set by the search position setting unit 120. In this case, a macro block that is determined to move according to a state variable among macro blocks of the first image frame is determined, and a sum of all absolute values of the differences between pixels between the determined macro block and the macro block of the second image frame ( SAD) can be used to estimate the motion vector. Information about the motion vector may be obtained as an expected value by the following equation (13).

4 is an exemplary diagram for describing a probability density function estimation in the motion estimation method according to FIG. 2.

Referring to FIG. 4, each image frame is divided into a plurality of macro blocks, f (t-1) is a first image frame which is a reference image frame, and f (t) is a frame after the first image frame. 2 video frames. In the case of the macro block index 'k-1', the state variable X _{k -1} relates to the movement of the (k-1) -th macroblock of the second image frame relative to the first image frame, which is the previous image frame. In addition, in the case of the index 'k' which is the next macro block, the state variable X _k is related to the movement of the k-th macro block of the second image frame with respect to the first image frame which is the previous image frame.

According to the present invention, after estimating a probability density function indicating which value the motion vector has a high probability, the motion vector can be estimated more quickly by searching only a high probability position. In addition, since the probability density function represents a probability that the motion vector has a certain value, the motion vector closer to the actual motion can be estimated than the motion estimation method of minimizing the error after the motion compensation.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, Therefore, the present invention should be construed as a description of the claims which are intended to cover obvious variations that can be derived from the described embodiments.

100: motion estimation device 110: probability density function estimation unit
111: particle initialization unit 112: particle filtering unit
113: particle resampling unit 120: search position setting unit
130: motion vector estimation unit

Claims (12)

A motion vector indicating a difference in pixel values between a macroblock included in a first image frame as a reference image frame and a macroblock included in a second image frame after the first image frame by blocking a plurality of image frames. A probability density function estimator for estimating a probability density function of a;
A search position setting unit for setting at least one position having a value greater than a predetermined threshold value in the first image frame as a search position; And
And a motion vector estimator for estimating the motion vector at the set search position. The method of claim 1, wherein the probability density function estimator,
By using a particle filter having a motion vector of the second image frame as a state variable and a difference between pixel values included in the first image frame and a pixel block included in the second image frame as observation values. And estimating a probability density function of the state variable according to the observation value. The method of claim 1, wherein the probability density function estimator,
And when the size of the macro block is smaller than a moving object, the motion of the macro block is estimated based on the macro blocks neighboring the macro block. The method of claim 2, wherein the probability density function estimator,
A particle initializer which initializes a particle including the state variable and a weight;
Update the state variable, calculate the distance between the macroblock included in the first image frame and the macroblock included in the second image frame as the observation value, calculate the similarity of the distance between the macroblocks, and Particle filtering unit for filtering the particles by increasing the weight; And
And a particle resampling unit to resample the particles when the similarity level of the filtered particles is higher than a preset reference value. The method of claim 2, wherein the search position setting unit,
And estimating a state variable having a value greater than the threshold value and setting the probability variable as the search position. The apparatus of claim 2, wherein the motion vector estimating unit comprises:
A macro block that is determined to be moved according to the state variable among the macro blocks of the first image frame is determined, and the sum of all absolute values of the differences between pixels of the determined macro block and the macro block of the second image frame is added together. Motion estimating apparatus for estimating a motion vector using. A motion vector indicating a difference in pixel values between a macroblock included in a first image frame as a reference image frame and a macroblock included in a second image frame after the first image frame by blocking a plurality of image frames. Estimating a probability density function of;
Setting at least one position in which the probability density function has a value greater than a preset threshold in the first image frame as a search position; And
Estimating the motion vector at the set search position. 8. The method of claim 7, wherein estimating the probability density function comprises:
By using a particle filter having a motion vector of the second image frame as a state variable and a difference between pixel values included in the first image frame and a pixel block included in the second image frame as observation values. And a method for estimating a probability density function of the state variable according to the observation value. 8. The method of claim 7, wherein estimating the probability density function comprises:
And when the size of the macro block is smaller than a moving object, the motion of the macro block is estimated based on the macro blocks neighboring the macro block. The method of claim 8, wherein estimating the probability density function comprises:
Initializing a particle including the state variable and a weight;
Update the state variable, calculate the distance between the macroblock included in the first image frame and the macroblock included in the second image frame as the observation value, calculate the similarity of the distance between the macroblocks, and Increasing the weight to filter the particles; And
If the similarity level of the filtered particles is higher than a preset reference value, resampling the particles. The method of claim 8, wherein the setting to the search position comprises:
And calculating a state variable having a value of the probability density function greater than the threshold value and setting the state variable to the search position. The method of claim 8, wherein estimating the motion vector comprises:
A macro block that is determined to be moved according to the state variable among the macro blocks of the first image frame is determined, and the sum of all absolute values of the differences between pixels of the determined macro block and the macro block of the second image frame is added together. A motion estimation method for estimating a motion vector using.

Images