JPH0998424A - Method and device for extracting affine transformation parameter - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately extract a value with a small calculation amount by extracting the affine transformation parameter of rotation angles in respective horizontal and vertical directions, expansion magnifications and a translation amount. SOLUTION: In a coordinate system 101 on a picture plane, when it is assumed that transformation supplied by the affine transformation parameter is added to the area 103 of a previous frame and deformation to the area 102 is performed on a present frame, it is considered that the affine transformation parameter added to the area 103 is the same for all the inner part of the area 103. Then, so as to solve obtained simultaneous equations, when the respective horizontal and vertical components of a vector are placed as equations I-IV by using already known D1x, D1a and D1b and Dx, Da and Db obtained by a parallel motion vector searched for the respective three points, the rotation angles θx and θy in the horizontal and vertical directions of the affine transformation parameter are obtained in the equations V and VI, the expansion magnifications Sx and Sy are obtained in the equations VII and VIII and a translation parameter (d) is obtained as well.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an affine transformation parameter extraction method and apparatus used for moving image compression processing in the field of video technology.

[0002]

2. Description of the Related Art In recent years, in order to realize a higher compression rate, a moving image compression method has been adopted which uses motion compensation utilizing the correlation of images in the time axis direction. Among them, as a method of expressing motion, a method of expressing parallel movement by a motion vector obtained by block matching is generally used. Affine transformation is known as a method of expressing not only parallel movement but also rotation and expansion / contraction with pseudo parameters.

A conventional method for obtaining an affine transformation formula and affine transformation parameters will be described below. The affine transformation formula is generally represented by (Equation 5). M in (Equation 5) is represented by (Equation 6) in the 2 × 2 determinant, d is a translation component represented by (Equation 7), and D1 is in the current image frame (hereinafter referred to as the current frame). D is an arbitrary region component in an image frame (hereinafter referred to as a previous frame) that is referred to when the motion is represented by parallel movement or rotational expansion / contraction, and D is an arbitrary region component that represents the motion.

[0004]

(Equation 5)

[0005]

(Equation 6)

[0006]

(Equation 7)

In Equation 6, Sx is a horizontal expansion / contraction ratio, Sy is a vertical expansion / contraction ratio, θx is a horizontal rotation angle, and θy is a vertical rotation angle. In (Equation 7), dx is a horizontal translation amount, and dy is a vertical translation amount. Using this affine transformation formula, any 2
In the arbitrary regions D and D1 that follow the motion existing in one image frame, the image signal component D already obtained without transmitting the image signal component included in D1 and 6 included in M and d Two parameters Sx, S
Image compression is realized by reproducing D1 using y, θx, θy, dx, and dy. Conventionally, the method used to find the affine transformation parameters used here is to detect the motion by gradually rotating or stretching the entire previous frame to transform it and then performing block matching between the current frame and the previous frame. The method of extracting the parameter of the target area is taken.

[0008]

However, in the above method for obtaining the affine transformation parameters, enormous calculation is required when gradually changing the rotational expansion / contraction parameters and deforming the previous frame, and other rotation angles are also required. Since the precision of the expansion / contraction ratio depends on the changing unit, the accurate rotation angle and the expansion / contraction ratio cannot be expressed. The present invention solves the above-mentioned problem, and in order to represent the motion of an image using an affine transformation, based on translation vectors of three different arbitrary points on an arbitrary region to be represented by the affine transformation. Extract affine transformation parameters,
The purpose is to enable reduction of calculation amount and extraction of accurate parameter values.

[0009]

In order to achieve this object, the affine transformation parameter extraction method of the present invention obtains a translation vector representing the movements of arbitrary three different points on the same image plane, and obtains the translation vector. From the three translation vectors, the rotation angle in each of the horizontal and vertical directions, the expansion / contraction ratio, and the translation amount are calculated from the position information of six points in the frame to be represented by the affine transformation and the frame to be referred to in the affine transformation. One affine transformation parameter is extracted.

[0010]

With this configuration, the positions of these three points in the previous frame are detected from the translation vectors obtained for three different arbitrary points in the current frame, and the relationship between the position in the current frame and the position in the previous frame is determined. By substituting into the affine transformation parameter formula, the affine transformation parameter of the area including these three points can be extracted.

[0011]

An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing an image plane in an xy coordinate system for explaining the affine transformation parameter extraction method. FIG. 2 is a block diagram of the affine transformation extraction device. In FIG. 1, 101 is a coordinate system for representing a position on an image plane, 102 is a region after deformation by affine transformation, which is a target for representing motion in the current frame, and 103 is used for representing motion of 102 in the previous frame. Areas before transformation, 104, 105 and 106 are arbitrary three different points which are the vertices of the area 102 in the current frame, 1
07, 108, and 109 are translation vectors of the three points 104, 105, and 106 detected from the current frame to the previous frame, and 110, 111, and 112 are the three vertices of the region 103 in the previous frame. 10,
Vectors 107, 108, 1 from the positions indicated by 5, 106
The position obtained by tracing 09 in reverse is shown.

As described above, the coordinate system 101 on the image plane
In, the deformation based on the affine transformation model is added to the region 103 of the previous frame, and the region 102 on the current frame is added.
Suppose it has been transformed into. As a method of analogizing the affine transformation parameter added at this time from the current frame image and the previous frame image, in the conventional example, the affine transformation parameter that is gradually changed is given to the previous frame image,
The created image is compared with the current frame image, and the parameter value given when the two images are the same or the one with the smallest difference is regarded as the affine transformation parameter of the current frame image. The method was taken.

In this case, for example, the rotation angle is changed by 1 degree from -10 degrees to 10 degrees both in the horizontal and vertical directions, and the expansion / contraction ratio is changed by 0.02 times in the horizontal and vertical directions from 0.8 times to 1.2 times. If an affine transformation parameter is to be detected by using the obtained one as an investigation target, D1 = MD +
The calculation given by d is 21 for all previous frame images.
4 must be performed, and the number of calculations is enormous. Also, only quantized parameter values such as 1 degree unit for the rotation angle and 0.02 times unit for the expansion / contraction ratio are given, and the affine transformation parameter values obtained by this method are also quantized. In this embodiment, the affine transformation calculation performed on the previous frame is not performed, and the calculation for searching the translation vector at three points and the three points in the current frame and the previous frame obtained from the searched vector are calculated. It is intended to extract the affine transformation parameters only from the positional relationship.

The method will be described below. First, in the coordinate system 101 on the image plane, if it can be assumed that the transformation given by the affine transformation parameter is added to the area 103 of the previous frame and transformed into the area 102 on the current frame, the affine transformation parameter added to the area 103 is the area. It is considered to be the same for all the insides of 103, and (Equation 8), (Equation 9), and (Equation 10) hold. M in (Equation 8), (Equation 9), (Equation 10) is (Equation 6), d
Is represented by (Equation 7). Here, (Equation 8), (Equation 9),
The simultaneous equations (Equation 11) and (Equation 1) obtained from (Equation 10)
To solve 2), known D1x, D1a, D1b
And the horizontal and vertical components of the vector by using Dx, Da, and Db obtained from the translation vector searched for these three points, (Equation 13), (Equation 14), (Equation 1)
5) and (Equation 16), the rotation angles θx and θy in the horizontal and vertical directions of the affine transformation parameters are calculated by
7) and (Equation 18), the scaling factor S in the horizontal and vertical directions
x and Sy are obtained as in (Equation 19) and (Equation 20),
By substituting this into any of (Equation 8), (Equation 9), and (Equation 10), the translation parameter d consisting of the translation amounts dx and dy in the horizontal and vertical directions can also be obtained.

[0015]

(Equation 8)

[0016]

[Equation 9]

[0017]

(Equation 10)

[0018]

[Equation 11]

[0019]

(Equation 12)

[0020]

(Equation 13)

[0021]

[Equation 14]

[0022]

(Equation 15)

[0023]

(Equation 16)

[0024]

[Equation 17]

[0025]

(Equation 18)

[0026]

[Equation 19]

[0027]

(Equation 20)

In this way, the region 102 can be processed without enormous calculation by gradually adding rotation or expansion / contraction to the previous frame.
It is possible to obtain the affine transformation parameter that represents, and it is possible to obtain the parameter having an accurate value, not the quantized one. However, when encoding in image compression, it may be necessary to quantize due to the limitation of the number of information bits. Comparing the number of calculations, even in the conventional method, calculation such as block matching is performed when comparing the image after the affine transformation to the previous frame image and the current frame image. In this example, the number of calculations performed to search for the three translation vectors in 3 is almost equal, and the number of affine transformation calculations added to the previous frame in the conventional example is small.

However, the condition that the parameter value obtained by this method is accurate is that there is no error in the translation vector of the three points, and a method such as block matching used in actual translation vector detection. Then, since an error is included in the obtained translation vector, an error also appears in the affine transformation parameter.
This is not a problem that appears only in this embodiment because there is an error caused by giving a quantized parameter value even in the conventional method. In addition, by executing the extraction of the affine transformation parameters shown in the present embodiment, when modeling is performed using the affine transformation when performing image compression using the correlation in the time axis direction between the image planes in the moving image, The encoding method becomes easy. In addition, by using this method for interpolation processing in which image information is transmitted by skipping any image frame in the time axis direction and the image frame skipped on the receiving side is created from the preceding and following image frames. Calculation can be greatly reduced.

Next, a special case of the example shown in the embodiment of the present invention will be described. In general, as an area used for image compression, a block which is a rectangle or a square in which the image plane is finely equally divided in the horizontal and vertical directions is used. Therefore, also in this embodiment, assuming that the divided block is a square, this block is diagonally divided into two to form a right-angled isosceles triangle, and the method described in the first embodiment is executed for the area of this triangle. To do. In this case, for example, D1x, D1a, D1 in FIG.
The corner D1x of the triangular area represented by b becomes a right angle, and D1
b and D1x have the same vertical position, and D1a and D1x
The horizontal position of may be the same,
In this case, xa of (Equation 13) and yb of (Equation 14) shown in the first embodiment are 0. Also, xb and ya become equal to one side of the square block, and if this is given as bs, the parameter θx used in the affine transformation,
θy, Sx, and Sy are (Equation 21), (Equation 22), (Equation 2)
3) and (24) are given and are simplified from those shown in the first embodiment.

[0031]

(Equation 21)

[0032]

(Equation 22)

[0033]

(Equation 23)

[0034]

(Equation 24)

In this way, it is possible to realize a method in which the image compression unit itself is simplified by uniformly dividing the image plane into blocks so that the image compression unit is constant and the affine transformation parameter extraction is simplified. FIG. 2 shows a block diagram of an affine transformation parameter extraction device according to the present invention. In FIG. 2, 201 is a frame memory, 202 is a translation vector detection block, 203 is an affine transformation parameter extraction block, 204 is a video signal input, 20
Reference numeral 5 is an affine transformation parameter output. The video signal of the previous frame input from 204 is the frame memory 201.
Stored in the parallel movement vector detection block 202 and newly input to the translation vector detection block 202.
Called as needed to determine the point translation vector. In the translation vector detection block 202,
The video signal of the previous frame from the frame memory 201 and the video signal of the newly input current frame are compared, and for example, block matching is performed on a block in a fixed area whose center is a point for which a translation vector is to be obtained. By doing so, three parallel translation vectors corresponding to the vertices of the target triangular region represented by the affine transformation are detected. The coordinate positions of the three vertices of the triangular area and the translation vector values of each of the three vertices obtained here are transmitted to the affine transformation parameter extraction block 203. In the affine transformation parameter extraction block 203, the affine transformation parameter can be extracted by calculating the above-mentioned rotation angles in the horizontal and vertical directions, the expansion / contraction ratio, and the translation amount from the transmitted values.

[0036]

As described above, according to the present invention, the positions of three different points on another plane obtained by detecting arbitrary three different points on the same image plane and a translation vector representing the movement of those three points. It is an excellent affine transformation parameter that can extract 6 affine transformation parameters of rotation angle in each direction of horizontal and vertical directions, expansion / contraction ratio, and translation amount using information, and can obtain highly accurate values with a small amount of calculation. The extraction method can be realized.

[Brief description of drawings]

FIG. 1 is an affine transformation parameter of a first embodiment of the present invention.
Image plane including a coordinate system that represents the movement of the data extraction method

FIG. 2 is an affine transformation parameter of the first embodiment of the present invention.
Block diagram of data extraction device

[Explanation of symbols]

101 coordinate system for representing position on image plane 102 current frame region 103 previous frame region 104, 105, 106 three vertices of current frame region 102 107, 108, 109 translation vector 110, 111, 112 previous frame Three vertices of the area 103 201 frame memory 202 translation vector detection block 203 affine transformation parameter extraction block 204 video signal input 205 affine transformation parameter output

Claims (3)

[Claims] 1. When using an affine transformation as a method for expressing the correlation of two image planes in a moving image in the time axis direction, three different arbitrary points on the same image plane and each of these three points are used. , Affine for obtaining position information on a total of 6 points on the image plane of 3 points on another image plane which are presumed to follow the movement, and representing a region represented by the corresponding 3 points of these position information. An affine transformation parameter extraction method for calculating six parameter values of horizontal and vertical rotation angles, expansion / contraction magnification, and parallel movement amount used for transformation. 2. Arbitrary three different points on one image plane in a moving image are defined as Da, Db, Dx, and three points on an image plane different from the image plane including Da, Db, Dx are D1a, Dx.
1b and D1x, and by the deformation of the rotational expansion / contraction parallel movement, the point Da becomes the point D1a, the point Db becomes the point D1b, and the point Dx becomes the point D.
It is assumed that it has moved to 1x, and means for calculating the translation vector searched for these three moved points, means for obtaining the vectors DaDx, DbDx, D1aD1x, D1bD1x, and the horizontal and vertical directions included in their deformation. An affine transformation parameter extraction device comprising means for calculating a rotation angle, means for calculating a horizontal and vertical expansion / contraction ratio, and means for calculating a translation amount. 3. A frame memory for storing image data,
Using the image data from the frame memory, each of the three vertexes of the triangular area has a translation vector detecting unit, and the coordinate positions of the three vertices of the triangular area and the translation vector value d are used. , The affine transformation parameters are (Equation 1),
An affine transformation parameter extraction device that extracts using (Equation 2), (Equation 3), and (Equation 4). [Equation 1] [Equation 2] (Equation 3) [Equation 4]