JPH0767111A - Motion estimating device for image - Google Patents

Abstract

PURPOSE:To exactly and efficiently estimate not only move but also various motions such as rotation or stretch concerning a moving area in a moving image by successively estimating the value of a motion parameter, which is increased for expressing the complicated motions, from the gradient and motion estimation error of a pixel value concerning each parameter. CONSTITUTION:An image signal and a motion estimation target area designate signal are respectively inputted from input terminals 1 and 2. As such a motion estimation target area, the face area of a figure is estimated, for example. Based on the motion expression decided by a parameter update part 3, the motion parameter of the target area is estimated. As the expression of the motion, affine transform mapping is considered, for example. It is expressed by parameters a to f that the pixel values of respective points (x) and (y) inside the motion estimation target area of a present frame image are equal with the pixel values of points x' and y' in the image of a preceding frame. At a parameter value updating amount calculation part 4, the value of the parameter is reflectively updated so as to reduce the motion estimation error.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion estimation apparatus for an image when performing motion compensation frame-to-spacing coding on a moving image.

[0002]

2. Description of the Related Art A high-efficiency coding technique for moving images is used to transmit images in a situation where the usable bandwidth is expected to be extremely limited, such as in a wireless videophone, or in comparison with an analog line. Development has been made to meet the demands such as transmission of high-quality images using a narrow communication path, and storage of images for a longer time than in the storage medium.

It is generally known that there is a very large correlation between successive frames in a moving image, and motion-compensated inter-frame prediction coding is a high-efficiency coding method that effectively utilizes this property. FIG. 6 shows a block diagram of conventional motion compensation interframe prediction coding. Here, first, the block motion estimation unit 11 obtains a motion vector for a moving region in the input image. The motion compensation unit 12 performs motion compensation of the previous frame according to the obtained motion vector. The transmission information is a prediction error and a motion vector of the obtained prediction image. Therefore, in the motion-compensated interframe coding, improving the coding efficiency is nothing but improving the accuracy of the predicted image, and improving the accuracy of the predicted image is equivalent to improving the accuracy of the motion estimation.

The conventional motion estimation methods are roughly classified into a block matching method and a gradient method. In the block matching method, a motion vector is obtained for each divided block. An error sum of pixel values between pixels located at the same position in a block of interest and a block in the previous frame located at a position displaced by a certain transition is obtained. The error sum is calculated for various transitions, and the transition having the smallest error sum is regarded as the motion vector of the block. This method guarantees the validity of the motion vector within the search range.

However, in motion estimation by block matching, only parallel movement is considered. Therefore, it is not possible to estimate motions that are not parallel movements such as rotation, expansion, contraction, and deformation. In addition, it takes a long time to perform the calculation because all the motion vector candidates within the designated range are searched.

On the other hand, the gradient method has been proposed as a method for obtaining a motion vector for each pixel. It is known that the movement vector (Vx, Vy) of the point (x, y) on the screen is approximately bound by the following equation.

Ex · Vx + Ey · Vy + Et = 0 where Ex and Ey are spatial gradients in the x-direction and y-direction, and Et is the gradient in the time-axis direction, both of which can be easily obtained by difference calculation. For example, as shown in FIG. 7, the pixel of interest in the current frame 13 is represented by a black circle (x,
y), the density difference from the pixel in the x direction (white circle) is E
The density difference from the pixel (white circle) in the x and Y directions is represented by Ey. Further, when a pixel at the same position as the target pixel in the previous frame 14 is indicated by a black circle (x ', y'), the density difference between this pixel value and the target pixel value in the current frame 13 is represented by Et. Thus, in the above equation, one Et is determined for one pixel of interest, and Ex and Ey are set in the X direction and Y, respectively.
By changing the direction, the above equation becomes 0 (V
x, Vy) is calculated.

Since the gradient method estimates the movement for each pixel, it is possible to deal with not only parallel movement but also rotation, expansion and contraction, but since the density value of each pixel is used, it is easily affected by noise. Also, there is a problem that large movements cannot be dealt with because only local gradients are considered. Therefore, there is an example in which the gradient method is applied to a large area such as a block instead of each pixel, but in this case, the motion estimation is limited to parallel movement.

[0009]

As described above, the conventional block matching method has a drawback that it can deal with only parallel movement and takes a long calculation time, and the gradient method is affected by noise and cannot deal with large movement. There was a drawback.

It is an object of the present invention to provide an image motion estimation device capable of accurately and efficiently estimating not only parallel movement but also various movements such as rotation and expansion / contraction for a moving area in a moving image. .

[0011]

According to the present invention, for a specific area of a temporally continuous frame image, a parameter value expressing the movement between the adjacent frame images of the specific area is recursively updated. Parameter value updating means, the change amount of the updating of the parameter value, the motion estimation error of a specific region between the current frame and the previous frame image by the parameter value before update and the gradient of the pixel value for each parameter in the current frame image And a parameter value update amount calculating means for determining the parameter value, and an update end determining means for terminating the update of the parameter value when the result of the parameter value updated by the parameter value updating means satisfies a predetermined condition. It is characterized by that.

[0012]

The image motion estimating apparatus according to the present invention sequentially estimates the value of the motion parameter increased for expressing a complicated motion from the gradient of the pixel value and the motion estimation error for each parameter. , Efficient and accurate motion estimation becomes possible. In addition, the initial value of motion estimation is set to a value having a minimum motion estimation error from among some predetermined candidates, and the parameter to be subject to motion estimation is stepwise according to the characteristics of each parameter. By changing it, it is possible to estimate a wide range of motion.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of a motion estimation apparatus according to an embodiment of the present invention. An image signal and a motion estimation target area designation signal are input from the input terminals 1 and 2, respectively. As the motion estimation target area, for example, a human face area is designated. The motion parameter of the target area is estimated based on the motion expression determined by the parameter value updating unit 3. An affine transformation map such as the expression (1) can be considered as a motion expression. Accordingly, it is expressed by the parameters a to f that the pixel value of each point (x, y) in the motion estimation target area of the current frame image is equal to the pixel value of the point (x ′, y ′) of the previous frame image. It However, a to d are parameters expressing rotation, enlargement, reduction and deformation, and e and f are parameters expressing parallel movement.

[0014]

[Equation 1] The parameter value update amount calculation unit 4 updates the parameter value so that the motion estimation error becomes smaller. As the updating method, for example, a recursive method based on the equation (2) can be considered.

[0015]

[Equation 2] In Expression (2), P _i is a parameter after i times of update, and g
_n (x, y) is the pixel value of the point (x, y) of the current frame image, _x'pi , _y'pi are the coordinate values converted by the parameter P _i for the position (x, y), g _{n- 1} (x ' _pi ,
y ′ _pi ) is the pixel value of the point (x ′ _pi , y ′ _pi ) of the previous frame image, ▽ pg _n is the gradient of the pixel value relating to the parameter P in the current frame image g _n , and Σ of the second term on the right side is the motion. The total sum over all points in the estimation target area is shown. That is, the denominator of the second term on the right side indicates the density difference between the pixel of interest on the current frame image and the converted pixel when the current six parameters are changed in basic units such as +1 (for example, e is incremented by +1). , The numerator indicates the density difference between the pixel of interest of the current frame image and the pixel on the previous frame of the coordinate value converted by the current parameter. Therefore, the whole second term on the right side is
It is the sum of the ratios of the density differences in the parameter unit of the current frame image to the density differences between the current frame image and the previous frame image by the current parameter. When a new parameter P _{i + 1} is obtained in this way, this is substituted into the second term on the right side to recursively obtain a new parameter. Then, after the update end determination unit 5 determines the end of the update of the parameter value, the motion parameter is output from the output terminal 6. As a method of determining the update end, a case where the value of the second term on the right side of the equation (2) becomes less than or equal to the threshold, a case where the motion estimation error after updating the parameter value increases, or a combination thereof is considered.

In equation (2), the sum is calculated for the entire second term on the right side, but the same parameter can be obtained by calculating the sum for each of the denominator and the numerator. FIG. 2 is a block diagram showing a second embodiment of the present invention.

The difference from FIG. 1 is that there is an initial value setting unit 7. In the initial value setting unit 7, the parameter value updating unit 3
The initial value of the parameter value used in is selected from some initial value candidates. For example, in the above motion expression,
fixed a, b, c, d at 1, 0, 0, 1 respectively, e,
f is (0,0), (-8,8), (8, -8),
A method is conceivable in which (8, 8) and (-8, -8) are used as initial value candidates, and a parameter value having the smallest motion estimation error is selected as an initial value.

FIG. 3 is a block diagram showing a third embodiment of the present invention. The difference from FIG. 1 is that there is a parameter value update execution determination unit 8. The parameter value update execution determination unit 8 determines whether or not to update the parameter value for a particular parameter in each update process. As a method, for example, when the motion estimation error after updating the parameter value increases, the parameter is not updated.

FIG. 4 is a block diagram showing a fourth embodiment of the present invention. The difference from FIG. 1 is that there is an estimation parameter updating unit 9. The estimated parameter updating unit 9 determines a parameter whose value is to be updated. The estimation parameter update is usually repeated until the values for all parameters are updated. A more specific description will be given using the above motion expression. In the present invention, the fact that the parameter values can be estimated in multiple stages is used, and when it is difficult to suddenly find all the parameter values a to f, first, for example, as in equation (3), there is a limitation that does not consider deformation. It is also possible to obtain it from the representation of the movement by the affine transformation map. Here, θ is a rotation, and s is an enlargement / reduction parameter.

[0020]

[Equation 3] In this case, as an example of the process of updating the estimated parameters

[0021]

[Equation 4] Can be considered.

FIG. 5 is a block diagram showing a fifth embodiment of the present invention. The difference from FIG. 1 is that the parameter value correction unit 1
There is 0. The parameter value correction unit 10 searches for a parameter value with a smaller motion estimation error by slightly changing the parameter value obtained by updating the parameter. For example, the following method can be considered. A motion estimation error is obtained when the parameter value is slightly changed in the + direction and the − direction. If the + direction is small, the parameter value is further changed in the + direction, and the motion estimation error at each time is obtained. When the increase of the motion estimation error is recognized, the parameter value having the smallest error until then is output as the correction value.

[0023]

According to the present invention, the value of a parameter to be updated is sequentially obtained from the gradient of the pixel value and the motion estimation error for each current parameter, so that various values of a specific area in a frame image can be obtained. It has an effect that motion can be estimated accurately and efficiently.

[Brief description of drawings]

FIG. 1 is a block diagram of a motion estimation device according to a first embodiment of the present invention.

FIG. 2 is a block diagram showing a second embodiment of the present invention.

FIG. 3 is a block diagram showing a third embodiment of the present invention.

FIG. 4 is a block diagram showing a fourth embodiment of the present invention.

FIG. 5 is a block diagram showing a fifth embodiment of the present invention.

FIG. 6 is a block diagram showing an example of conventional motion compensation interframe predictive coding.

FIG. 7 is a diagram for explaining a conventional gradient method.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Input terminal 2 ... Input terminal 3 ... Parameter value update part 4 ... Parameter value update amount calculation part 5 ... Update end determination part 6 ... Output terminal 7 ... Initial value setting part 8 ... Parameter value update execution determination part 9 ... Estimated parameter Update unit 10 ... Parameter value correction unit

Claims (1)

[Claims] 1. A parameter value updating unit for recursively updating a parameter value expressing a movement between adjacent frame images of a specific area in a specific area of temporally continuous frame images, and the parameter. Parameter value update that determines the amount of change in value update based on the motion estimation error of a specific region between the current frame image and the previous frame image based on the parameter value before update and the gradient of the pixel value for each parameter in the current frame image A motion of an image, comprising an amount calculation means and an update end determination means for ending the update of the parameter value when the result of the parameter value updated by the parameter value updating means satisfies a predetermined condition. Estimator.