KR101777776B1 - Apparatus and method for sampled profile based motion estimation - Google Patents

Abstract

The present invention relates to an apparatus and method for estimating motion information based on a sample profile information, comprising: an edge image generation unit for generating an edge image of a current image by detecting an edge corresponding to an outline of an object in a current image; An image analyzer that receives the edge image and detects a horizontal maximum line HmaxLine and a vertical maximum line VmaxLine that most include edge information in a motion estimation target block; And a motion vector estimator for calculating a block matching error between the current image and the reference image based on the horizontal maximum line HmaxLine and the vertical maximum line VmaxLine to predict a coordinate difference of the block having the smallest difference as a motion vector .

Description

[0001] APPARATUS AND METHOD FOR SAMPLE PROFILE INFORMATION BASED MOTION ESTIMATION [0002]

The present invention relates to a sample profile information based motion estimation apparatus and method, and more particularly, to a motion vector estimation method and apparatus for detecting a motion vector of a current image and a reference image, The present invention relates to a sample profile information based motion estimation apparatus and method for estimating a motion vector by comparing a horizontal line and a vertical line,

A key element that supports multimedia is image compression technology along with digitalization. In recent years, digitalization of information is rapidly developing, and image compression technology is becoming very important.

In general, a digital video device such as a PC, an HDTV, or a mobile device converts an image signal compressed by an image compression method such as Moving Picture Experts Group (MPEG) -2, MPEG-4, or H.264 into an MPEG-based image codec And displays it by encoding.

This video CODEC eliminates temporal redundancy based on the block matching method. In the block matching method, the image is divided into small blocks, and a portion similar to the block of the current encoded / decoded image is temporally detected in the previous or previous encoded / decoded image, and the differential value is compressed .

To do this, it is necessary to find the most similar block between the previous frame and the current frame. This is referred to as Motion Estimation (ME), and a motion vector (MV) indicating a displacement, .

The concept of intermediate image frame Fi synthesis using the block matching method is shown in FIG.

1, in order to synthesize an intermediate image frame Fi between a previous frame Fn-1 and a current frame Fn, a motion vector MV between a block in a previous frame Fn-1 and a block Fn in the current frame Motion estimation is performed to synthesize blocks of the intermediate image frame Fi.

In motion estimation for video signal compression, the current image is divided into macroblocks, and error values such as a sum of absolute difference (SAD) and a sum of square difference (SSD) The motion vector is used as a motion vector.

In the case of H.264, a motion vector with the lowest cost is used for motion estimation with a cost considering the amount of data required to transmit the motion vector together with the SAD or SSD.

Here, in order to find a motion vector, it is necessary to calculate SAD or SSD for all the pixels in the motion search area, and find a position having the minimum value among them. For example, if the size of the block is NxN pixels and the motion search area is 32x32, addition / subtraction operations of 1024 * (2NxN-1) times are required. If the size of the image is 100Nx80N, 8192000 * (2NxN-1) * ). This causes a problem of causing a system load and delaying the processing speed due to a large amount of computation to occupy an operation up to 50% in the image encoding.

Korean Patent Publication No. 2004-0049214 (2004.06.11), Title of the Invention: High-speed motion vector estimation apparatus and method

SUMMARY OF THE INVENTION The present invention has been conceived to solve the above-described problems, and it is an object of the present invention to provide a method and apparatus for detecting edge of a current image and a reference image when estimating a motion vector for block matching, The present invention also provides a method and apparatus for estimating a motion vector based on a sample profile information based on a motion vector of a motion vector.

According to another aspect of the present invention, there is provided a sample profile information-based motion estimation apparatus comprising: an edge image generation unit detecting an edge corresponding to an outline of an object in a current image to generate an edge image of the current image; An image analyzer that receives the edge image and detects a horizontal maximum line HmaxLine and a vertical maximum line VmaxLine that most include edge information in a motion estimation target block; And a motion vector estimator for calculating a block matching error between the current image and the reference image based on the horizontal maximum line HmaxLine and the vertical maximum line VmaxLine to predict a coordinate difference of a block having the smallest difference as a motion vector.

Here, the image analyzing unit compares the edge image with a specific value T and performs threshold processing to generate a feature information image to be recorded as 0 and 1. The horizontal profile and the vertical profile of the feature information image are mapped to the following mathematical expression To calculate the horizontal maximum line HmaxLine and the vertical maximum line VmaxLine.

[Equation 1]

H (x) =? ^N I (x, y)

&Quot; (2) "

V (y) = 裡^M I (x, y)

&Quot; (3) "

X = arg _x Max (? ^N E (x, y))

&Quot; (4) "

Y = arg _x Max (? ^M E (x, y))

(X, y) represents the intensity value of the pixel at the (x, y) coordinates, and MxN is the size of the block. H (x) and V (y) are the horizontal profile and the vertical profile, respectively.

The motion vector estimator may calculate a block matching error based on the luminance values of the pixels corresponding to the horizontal maximum line HmaxLine and the vertical maximum line VmaxLine of the current image and the luminance values of the corresponding pixels of the reference image .

The motion vector estimation unit may calculate a difference between the current image and the reference image to at least one of SAD and SSD using the following equation.

&Quot; (5) "

SAD = {? | Iref (HmaxLine + X, y + Y) - Icur (HmaxLine, y) |

+ VmaxLine + VmaxLine + VmaxLine + VmaxLine + VmaxLine + VmaxLine +

&Quot; (6) "

SSD = {Σ | ref (HmaxLine, y) - Hcur (HmaxLine, y) | ²

+? ?? Vref (x, VmaxLine) - Vcur (x, VmaxLine) | ² } x?

(Where N is a scale constant and N is a natural number)

According to another aspect of the present invention, there is provided a sample profile information-based motion estimation method comprising: receiving an input image and generating an edge image of a current image by detecting an edge corresponding to an outline of an object in a current image; Detecting a horizontal max line (HmaxLine) and a vertical max line (VmaxLine) which receive the edge image and contain the edge information in the block in the largest amount; Calculating a block matching error between a current image block and a reference image block based on a horizontal maximum line HmaxLine and a vertical maximum line VmaxLine of the current image; And predicting a coordinate difference of blocks having the smallest block matching error as a motion vector.

Here, the step of detecting the horizontal maximum line HmaxLine and the vertical maximum line VmaxLine, which receive the edge image and contain the edge information the most, may include comparing the edge image with a specific value T, , Generating a feature information image recorded with 0 and 1; Calculating a horizontal profile and a vertical profile of the feature information image by applying the following equations (1) and (2);

[Equation 1]

H (x) =? ^N I (x, y)

&Quot; (2) "

V (y) = 裡^M I (x, y)

And detecting the horizontal maximum line HmaxLine and the vertical maximum line VmaxLine including the edge information of the horizontal profile and the vertical profile by applying the following Equations (3) and (4).

&Quot; (3) "

X = arg _x Max (? ^N E (x, y))

&Quot; (4) "

Y = arg _x Max (? ^M E (x, y))

The step of calculating a block matching error between the current image and the reference image based on the horizontal maximum line HmaxLine and the vertical maximum line VmaxLine of the current image may further include calculating a difference between the current image and the reference image by using the following equation And calculating a value of at least one of the SAD and the SSD.

&Quot; (5) "

SAD = {? | Iref (HmaxLine + X, y + Y) - Icur (HmaxLine, y) |

+ VmaxLine + VmaxLine + VmaxLine + VmaxLine + VmaxLine + VmaxLine +

&Quot; (6) "

SSD = {Σ | ref (HmaxLine, y) - Hcur (HmaxLine, y) | ²

+? ?? Vref (x, VmaxLine) - Vcur (x, VmaxLine) | ² } x?

(Where N is a scale constant and N is a natural number)

As described above, the apparatus and method for estimating motion information based on the sample profile information of the present invention detects edges of a current image and a reference image in motion vector estimation for block matching, By comparing the matching errors of the horizontal line and the vertical line, which are included, the motion vector is predicted, thereby reducing the amount of computation for motion prediction and enabling high-speed image encoding.

FIG. 1 is a diagram illustrating a motion vector prediction state using a block matching method,
FIG. 2 is a control block diagram of a sample profile information-based motion estimation apparatus according to the present invention;
3 is a flowchart of a sample profile information-based motion estimation method according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 is a control block diagram of a sample profile information-based motion estimation apparatus according to the present invention. In order to estimate the motion vector, a current image and a reference image are required. Here, the reference image refers to an image temporally prior to the current image, or a previously processed image in the encoding / decoding order.

2, the motion estimation apparatus of the present invention includes an edge image generation unit 110, an image analysis unit 120, a motion vector estimation unit 130, and a motion vector storage unit 140.

The edge image generation unit 110 generates an edge image by edge detection of an input image. The edge of the image is the discontinuity of the image brightness, which means the brightness of the image changes from a low value to a high value or vice versa. That is, since the edge corresponds to the outline of the object in the image, the pixel coordinates of the outline of the displayed object can be extracted by detecting the edge. As a method of generating an edge image, various techniques such as detecting a rate of change of brightness, that is, taking a slope as an edge when the slope is equal to or greater than a predetermined reference value, are used. Accordingly, the edge image generation unit 110 generates an edge image using the edge detection method that has been conventionally introduced.

The image analyzer 120 receives the edge image and detects a horizontal line and a vertical line that include most edge information.

The image analyzing unit 120 first thresholds the edge image to a specific value T, and if the edge value is larger than T, it records 1, and if it is smaller, it records 0 to generate a feature information image. Since the edge value is the rate of change of the image brightness, only the pixel having the possibility of the outline of the object is recorded as 1 in the feature information image in which the threshold process is completed.

The image analysis unit 120 generates a horizontal profile and a vertical profile for each block in the feature information image. The equations for calculating the horizontal profile and the vertical profile are as shown in the following equations (1) and (2).

[Equation 1]

H (x) =? ^N I (x, y)

&Quot; (2) "

V (y) = 裡^M I (x, y)

&Quot; (3) "

X = arg _x Max (? ^N E (x, y))

&Quot; (4) "

Y = arg _x Max (? ^M E (x, y))

(X, y) represents the intensity value of the pixel at the (x, y) coordinates, and MxN is the size of the block. In addition, H (X) is a horizontal profile, V (Y) is a vertical profile, and X, Y represent a horizontal line and a vertical line of a block selected by Equations (3) and (3).

As can be understood from Equations (1) and (2), the profile is a value obtained by adding the pixel values of the same vertical line or horizontal line in the two-dimensional image.

The image analyzer 120 generates a horizontal profile and a vertical profile of each block, and then extracts one horizontal line and one vertical line, each of which contains the most edge information, respectively. The following equations (3) and (4) are used to detect the horizontal maximum line HmaxLine and the vertical maximum line VmaxLine, which contain much edge information in each block.

&Quot; (3) "

X = arg _x Max (? ^N E (x, y))

&Quot; (4) "

Y = arg _x Max (? ^M E (x, y))

The motion vector estimator 130 calculates a matching error between a block in the reference image and a block in the current image using a horizontal maximum line HmaxLine and a vertical maximum line VmaxLine that most include edge information. A method of calculating a matching error corresponding to a Sum of Absolute Difference (SAD) or a Sum of Square Difference (SSD) in the motion vector estimator 130 using the horizontal maximum line HmaxLine and the vertical maximum line VmaxLine is as follows: (5) and (6).

&Quot; (5) "

SAD = {? | Iref (HmaxLine + X, y + Y) - Icur (HmaxLine, y) |

+ VmaxLine + VmaxLine + VmaxLine + VmaxLine + VmaxLine + VmaxLine +

&Quot; (6) "

SSD = {Σ | ref (HmaxLine, y) - Hcur (HmaxLine, y) | ²

+? ?? Vref (x, VmaxLine) - Vcur (x, VmaxLine) | ² } x?

Here, Iref (x, y) and Icur (x, y) represent intensity values at the (x, y) coordinates of the reference image and the current image, and (X, Y) represent candidate motion vectors. Also, a is a scale constant of N / 2 (N = natural number).

The motion vector estimator 130 calculates an error value such as a Sum of Absolute Difference (SAD) or a Sum of Square Difference (SSD) using a horizontal maximum line HmaxLine and a vertical maximum line VmaxLine, And the area where the error value with the minimum value is minimum is determined as a motion vector.

The motion vector storage unit 140 stores the estimated motion vectors.

As described above, according to the present invention, if a horizontal maximum line HmaxLine and a vertical maximum line VmaxLine, which have the largest amount of edge information for each block of the current image, are found, they can be used equally for all candidate motion vectors. (2NxN-1) times is reduced to 4N at the time of comparing the size of the blocks, thereby reducing the calculation time.

For example, if N is 8, 127 operations are required according to the prior art, but in the case of the present invention, the number of operations is reduced to 32, so that the proposed method achieves a speed improvement of about 4 times.

3 is a flowchart of a sample profile information-based motion estimation method according to the present invention.

First, the edge image generation unit 110 generates an edge image for the current image (S110). The edge of the image is the discontinuity of the image brightness, which means the brightness of the image changes from a low value to a high value or vice versa. That is, since the edge corresponds to the outline of the object in the image, the pixel coordinates of the outline of the displayed object can be extracted by detecting the edge.

The image analyzer 120 compares the edge image value with a specific value T and performs a threshold process to generate a feature information image to be recorded as 0 and 1 (S112). Since the edge value is the rate of change of the image brightness, only the pixel having the possibility of the outline of the object is recorded as 1 in the feature information image in which the threshold process is completed.

The image analysis unit 120 calculates a horizontal profile and a vertical profile for each block in the feature information image (S114). The image analyzer 120 may apply Equation (1) to generate the horizontal profile image of the feature information image, and apply Equation (2) to generate the vertical profile image. Here, the profile may be defined as a value obtained by summing the pixel values of the same vertical line or horizontal line in the two-dimensional image.

The image analyzing unit 120 detects a horizontal maximum line HmaxLine and a vertical maximum line VmaxLine that most include edge information based on the horizontal profile and the vertical profile value at step S116. The image analyzing unit 120 may detect the horizontal maximum line HmaxLine and the vertical maximum line VmaxLine that include the edge information most by applying Equations 3 and 4. [

Thereafter, the difference between the block of the reference image and the block of the current image is calculated using the horizontal maximum line HmaxLine and the vertical maximum line VmaxLine (S118). In calculating the error value, the motion vector estimation unit 130 may calculate the SAD using Equation (5) or calculate the SSD using Equation (6).

The motion vector estimator 130 calculates a matching error value and estimates a coordinate difference between the current block and the reference block having the minimum error value as a motion vector (S120).

Thus, those skilled in the art will appreciate that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. It is therefore to be understood that the embodiments described above are to be considered in all respects only as illustrative and not restrictive. The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

The present invention detects an edge of a current image and a reference image when estimating a motion vector for block matching and compares a matching error of a horizontal line and a vertical line, Thereby reducing the amount of computation for motion prediction and enabling high-speed encoding. The present invention can be applied to a sample profile information-based motion estimation apparatus and method.

110: edge image generating unit
120: Image analysis section
130: motion vector estimator
140: a motion vector storage unit

Claims (7)

An edge image generation unit for generating an edge image of a current image by detecting an edge corresponding to an outline of an object in a current image;
An image analyzer that receives the edge image and detects a horizontal maximum line HmaxLine and a vertical maximum line VmaxLine that most include edge information in a motion estimation target block; And
And a motion vector estimator for calculating a block matching error between the current image and the reference image based on the horizontal maximum line HmaxLine and the vertical maximum line VmaxLine to predict a coordinate difference of a block having the smallest difference as a motion vector and,
Wherein the motion vector estimator comprises:
Wherein a block matching error is calculated based on a luminance value of a pixel corresponding to a horizontal maximum line (HmaxLine) and a vertical maximum line (VmaxLine) of the current image and a luminance value of a corresponding pixel of the reference image.
The method according to claim 1,
Wherein the image analyzing unit comprises:
A feature information image to be recorded as 0 and 1 is generated by comparing the edge image with a specific value T, and a horizontal profile and a vertical profile of the feature information image are applied to the following equation, Wherein the horizontal maximum line (HmaxLine) and the vertical maximum line (VmaxLine) are detected.
[Equation 1]
H (x) =? ^N I (x, y)
&Quot; (2) "
V (y) = 裡^M I (x, y)
&Quot; (3) "
X = arg _x Max (? ^N E (x, y))
&Quot; (4) "
Y = arg _x Max (? ^M E (x, y))
Where (x, y) is the coordinates of the pixel and I (x, y) is the intensity value of the pixel at the (x, y) coordinates. Further, H (X) and V (Y) are the horizontal profile and the vertical profile, respectively, and represent the horizontal maximum line HmaxLine and the vertical maximum line VmaxLine in the edge image E. delete The method according to claim 1,
Wherein the motion vector estimator comprises:
Wherein the difference between the current image and the reference image is calculated as at least one of SAD and SSD using the following equation.
&Quot; (5) "
SAD = {? | Iref (HmaxLine + X, y + Y) - Icur (HmaxLine, y) |
+ VmaxLine + VmaxLine + VmaxLine + VmaxLine + VmaxLine + VmaxLine +
&Quot; (6) "
SSD = {Σ | ref (HmaxLine, y) - Hcur (HmaxLine, y) | ²
+? ?? Vref (x, VmaxLine) - Vcur (x, VmaxLine) | ² } x?
(Where N is a scale constant and N is a natural number) Generating an edge image of a current image by detecting an edge corresponding to an outline of an object in a current image by receiving a current image;
Detecting a horizontal max line (HmaxLine) and a vertical max line (VmaxLine) that receive the edge image and contain the edge information the most;
Calculating a block matching error between a current image and a reference image based on a luminance value of a pixel corresponding to a horizontal maximum line HmaxLine and a vertical maximum line VmaxLine of the current image and a luminance value of the corresponding pixel of the reference image; And
And estimating a coordinate difference of blocks having the smallest block matching error as a motion vector. 6. The method of claim 5,
The step of detecting a horizontal maximum line (HmaxLine) and a vertical maximum line (VmaxLine), which receives the edge image and contains edge information the most,
Generating a feature information image recorded with 0 and 1 by comparing the edge image with a specific value T and performing threshold processing;
Calculating a horizontal profile and a vertical profile of the feature information image by applying the following equations (1) and (2);
[Equation 1]
H (x) =? ^N I (x, y)
&Quot; (2) "
V (y) = 裡^M I (x, y)
Detecting a horizontal maximum line (HmaxLine) and a vertical maximum line (VmaxLine) which include the edge information most of the horizontal profile and the vertical profile by applying the following Equations (3) and (4) A method of motion estimation based on sample profile information.
&Quot; (3) "
X = arg _x Max (? ^N E (x, y))
&Quot; (4) "
Y = arg _x Max (? ^M E (x, y)) 6. The method of claim 5,
The step of calculating a block matching error between the current image and the reference image based on the horizontal maximum line (HmaxLine) and the vertical maximum line (VmaxLine)
And calculating a difference between the current image and the reference image using at least one of SAD and SSD using the following equation.
&Quot; (5) "
SAD = {? | Iref (HmaxLine + X, y + Y) - Icur (HmaxLine, y) |
+ VmaxLine + VmaxLine + VmaxLine + VmaxLine + VmaxLine + VmaxLine +
&Quot; (6) "
SSD = {Σ | ref (HmaxLine, y) - Hcur (HmaxLine, y) | ²
+? ?? Vref (x, VmaxLine) - Vcur (x, VmaxLine) | ² } x?
(Where N is a scale constant and N is a natural number)

Images