JPH04180371A - Motion detector for image - Google Patents

Abstract

PURPOSE:To obtain the same conversion coefficient for the same motion even when motion vectors obtained by positions on a screen are provided with different direction and size by calculating the motion of the whole image from detected amount of motion >=2 as a conversion efficient. CONSTITUTION:A motion vector selection circuit 3 selects two motion vectors out of vectors V1-Vn, and a conversion coefficient calculation circuit 4 calculates the motion of an image as the conversion coefficient. By displaying the motion of the image by using the conversion coefficient, the same conversion coefficient can be obtained for the same motion even when the direction and size of the motion vectors obtained by the positions on the screen such as the motion of rotation, expansion, and compression, etc., are different.

Description

DETAILED DESCRIPTION OF THE INVENTION Field of the Invention The present invention relates to an image motion detection device for detecting screen flipping caused by shaking, rotation, etc. of a camera.

Conventional technology Conventionally, as a technology in this field, for example, the patent application
The one shown in Japanese Patent No. 1-269475 is known. FIG. 6 shows a block diagram of this conventional image motion detection apparatus. In FIG. 6, 22 to 29 calculate the correlation between the signal of the previous image frame and the signal of the current image frame using the so-called block matching method for each area obtained by dividing the screen into four. A correlation value with the highest correlation among the values is searched, and a candidate vector that gives that correlation value is detected. ``The validity/invalidity determination circuit 30 determines validity/invalidity of the candidate vector using about 11 correlation values, and outputs a determination signal to the vector determination circuit 31. Vector judgment circuit 3
1 calculates a motion vector indicating the amount of parallel movement of the entire image from candidate vectors determined to be valid. With this configuration, candidate vectors in areas where there is little change in the picture pattern are considered to be unreliable and are invalidated, and are excluded from motion vector calculation, thereby improving the accuracy of motion vector detection.

In addition, as a method for determining the movement of an object that has motion including rotation and zooming, for example, "Method for detecting motion vectors of objects that change size and direction" (IEICE Spring National Conference (1989) D-109, p. 7 -91).

Problems to be Solved by the Invention However, in the former configuration as shown in FIG. 6, the validity or invalidity of each candidate vector is determined only by the correlation value obtained for each divided screen corresponding to each candidate vector. , insufficient consideration was given to the movement of the entire image and the imprinting of different candidate vectors.

For example, even if a part of the screen with a highly variable pattern moves differently from the whole image, the candidate vector for this movement will be determined to be valid by the correlation value described above, and the This could lead to large errors in motion determination.

In addition, in the conventional example, in order to detect the zooming state of a camera, for example, the center point of the car camera's optical system and the center point of the signal processing must match, and rotation and translation must be closely matched. In some cases this could not be detected.

Also, the latter IEICE Spring National Conference (1989) D-109, p? In the method shown in 91, even if there is a region whose motion differs greatly from the motion of the entire image, it is not excluded and used to calculate the motion of the entire image, so it is not possible to Images are affected by local movement, and it is not always possible to obtain high accuracy.

The present invention solves these conventional problems, and even when a part of the screen moves differently from the whole screen, or when the camera shakes or rotates, or when the camera zooms, etc., the image remains unchanged. An object of the present invention is to provide an image motion detection device that can accurately detect the motion of an image.

Means for Solving the Problems The present invention provides means for detecting the amounts of motion in a plurality of areas of an image, means for calculating a transformation coefficient representing the motion of the image using a plurality of the detected amounts of motion, Means for calculating predicted motion amounts in multiple regions on the screen using the conversion coefficients, and means for extracting regions where the error between the predicted motion amounts and the detected motion amounts of the multiple regions is less than or equal to a predetermined value. The apparatus includes a region extracting means and a means for determining the movement of the entire image using the extracted amount of movement.

According to the present invention, the above-described configuration detects the amount of movement of a plurality of areas of an image, selects a plurality of amounts of movement of the detected plurality of areas, and performs parallel translation, rotation, enlargement, etc.
A transformation coefficient is calculated that represents the movement of the entire image caused by the combination of reductions. Therefore, even if the direction and magnitude of the motion vector obtained differs depending on the position of the screen, as in the case of movements such as rotation, enlargement, and reduction, the same conversion coefficient will be obtained by carving the same movement, and the same It becomes possible to identify areas belonging to movement. In addition, based on the calculated conversion coefficients, the amount of motion at other positions in the image is predicted, and regions where the difference between the predicted amount of motion and the amount of detected motion is smaller than a predetermined threshold are extracted. Only moving parts can be obtained as the same area, and movements of the entire image, including parallel translation, rotation, enlargement, and reduction, as well as movements of some areas that differ from the overall movement, can be detected with high accuracy. I can do it.

EXAMPLES Hereinafter, examples of the present invention will be described with reference to the drawings.

FIG. 1 shows a configuration diagram of an image motion detection device in an embodiment of the present invention. In FIG. 1, 1 is an image input terminal, and 2 is a motion vector detection circuit, which is means for detecting the amount of motion in n areas of the screen as motion vectors 1 to 2o.

3 is a motion vector selection circuit which selects any two motion vectors ■1, ■1 (1≠J) from the obtained motion vectors VI-Vo; The two motion vectors ■1, ■, (1≠
This is a conversion coefficient calculation circuit that calculates conversion coefficients based on J). 5 is a motion amount prediction circuit, and a conversion coefficient calculation circuit 4
This is a means of predicting the amount of movement at other positions on the screen using the conversion coefficients obtained in . 6 is an error detection circuit that calculates the error between the amount of motion detected by the motion vector detection circuit 2 and the amount of motion predicted by the motion amount prediction circuit 5; 7 is a threshold circuit that compares the error with a threshold; 8 is the result. 9 is a motion determining circuit that determines the motion of the entire image; and 10 is a motion amount output terminal that outputs the result.

The operation of the image motion detection apparatus of this embodiment configured as described above will be described below.

The motion vector detection circuit 2 uses a so-called pattern matching method to calculate the amount of motion in each area when the screen is divided into n parts into motion vectors 1 to V.

Detected as. The motion vector selection circuit 3 is ■1 to vn
Two motion vectors are selected from among them, and using these, the conversion coefficient calculation circuit 4 calculates the movement of the image as a conversion coefficient in the following manner.

If the point (x+y) in the image moves to (X, Y) due to movement such as parallel translation, enlargement/reduction, rotation, etc., then X, Y are x, y before movement and coefficients -a, b, c
, d can be used to express X=ax+by+c Y=-bx+ay+d. The transformation coefficients a - d can be determined as follows using motion vectors at two positions.

Letting the X and Y components of the motion vector at point (x, y) be 1 and 2, respectively, then = X - x v, = y - y, so the transformation coefficient and the point (x, y) are The relationship between motion vectors 8 and 2 is (a-1)x+by+c=V.

-bx+ (a-1)y+c=Vv. Coefficients a, b, c, d are 2
X components Vxl, VX2 of the motion vector at each position of the point (X+, y+>, (X2. V2)
, Y components ■y1, ■y2 can be expressed as follows.

(a 1)X++t)y++C:VX+(a-1)x
2+byp+c=Vx2 bX++(a I)5'++d=V5/'+bx2+
(a-1) y2+d=■y2 Therefore, coefficients a, b, c, and d can be found by solving this simultaneous equation.

By the method described above, the conversion coefficient calculation circuit 3 calculates a conversion coefficient using two of the n motion vectors 1 to vn. In this way, by expressing the movement of an image using conversion coefficients, even if the direction and magnitude of the motion vector obtained by the screen position differs, such as rotation, enlargement, reduction, etc., the same movement can be expressed. The same conversion coefficients are obtained for both.

Next, the motion amount prediction circuit 5 predicts motion vectors at other positions in the image based on the calculated conversion coefficients. The error detection circuit 6 calculates the error between the predicted motion vector and the motion vector detected by the motion vector detection circuit 2. Note that in a region having a motion similar to the motion of the two regions used to calculate the transformation coefficient, the error between the predicted motion vector and the detected motion vector is smaller than the predetermined darkness value. Therefore, such an area is detected by the equivalence circuit 7, and is extracted by the motion area extraction circuit 8 as an area having the same motion.

As described above, according to this embodiment, since the movement of an image is expressed using conversion coefficients, the direction and magnitude of the motion vector obtained from the position of the screen, such as movements such as rotation, enlargement, and reduction, are expressed using conversion coefficients. Even if the motions are different, the same transformation coefficients are obtained for the same motion, making it possible to identify regions belonging to the same motion. In addition, since the area in which the difference between the predicted amount of motion and the detected amount of motion is smaller than a predetermined threshold is extracted, it is possible to extract parts with similar motion, making it possible to detect the motion of the entire image and It becomes possible to detect parts that differ from the overall movement.

Therefore, the movement of the image can be detected with high accuracy.

FIG. 2 is a block diagram of an image motion detection device according to a second embodiment of the present invention. The same means as in the embodiment of FIG. 1 are denoted by the same reference numerals, and the explanation thereof will be omitted. The difference from FIG. 1 is that a coefficient evaluation circuit 41, a coefficient selection circuit 42, a candidate area coefficient circuit 81, and a maximum area selection circuit 82 are provided.

The operation of the image motion detection device of the second embodiment configured as described above will be described below with respect to the differences from that of FIG. 1.

The motion vector selection circuit 3 selects the detected motion vectors ■1~
■ Any three motion vectors from o ■1, ■4, V
Select k (i≠J, 1≠k1.1f−k).

Then, the transformation coefficient calculation circuit 4 calculates transformation coefficients that give the motion of the entire image from these motion vectors. That is, the conversion coefficients a-f can be obtained as follows using the motion vectors V1, 4, and Vb at the three positions.

If the point (xt' y) in the image moves to (X, Y) due to movements such as parallel translation, enlargement/reduction, and rotation, then x, Y are the x, y before the movement, and the coefficients a, b,
Using c, d, e, f, X=ax+b
It can be expressed as y+c Y=dx+ey+f. The coefficients a, b, c are 3 points (XI.

y+), (x2v 3' 2) (X3, V3) X component of the motion vector at each position Vx+,
It can be expressed as follows using VX2 and vx3.

(a 1) x++by++c=Vx+(a-
1) X2+b5/++C=VX2(a 1)x3+
by3+c=Vx3 Therefore coefficients a, b, c
can be found by solving this simultaneous equation. Also, the coefficients d, e, and f are 3 points (X I!'
/ ' )・(X21 V2) (xa・y3)
Y component Vy+ of the motion vector at each position of
, ■It can be expressed as follows using y2 and vy3.

dX++ (e 1)'y++f=VV+dX2+
(e 1) y2+f=Vy2dX3+ (e-1)
V3+f=Vy3 Therefore, the coefficients d, e, and f can be found by solving this simultaneous equation.

By the method described above, the conversion coefficient calculation circuit 4 can calculate the conversion coefficient using three of the n motion vectors 1 to 2o.

Here, if the movement of the entire image is a combination of translation, rotation, expansion, and reduction, and there is no shape distortion, there is a special relationship for the transformation coefficients a - d, that is, a = e = sxφ cos θ -(1)b=-d=-S
The relationship x-sin θ ---(2) should hold true. Here, Sx is the magnification rate in the X-axis direction, S, is the magnification rate in the Y-axis direction, and θ is the rotation angle of the image. In normal camera operations, when the subject is stationary, the movement of the entire image can be thought of as a combination of parallel translation, rotation, enlargement, and reduction.

Therefore, after calculating the conversion coefficients using the three motion vectors ■1, ■1, ■, the coefficient evaluation circuit 41 evaluates the conversion coefficients, and the coefficient selection circuit 42 selects the one that satisfies the above-mentioned relationship. By using it to predict the amount, it is possible to determine whether the movement of the image obtained from the three motion vectors v1, ■1, and Vb is a movement that can occur through normal camera operations such as parallel translation, rotation, and enlargement/reduction. can. For example, if a part of the screen moves differently from the whole, or if a foreign object enters the screen, the image may expand in the X-axis direction and contract in the Y-axis direction. Then, it can be seen that the transformation coefficient obtained from the motion vector of such a region does not satisfy the relationships of equations (1) and (2) above, and is inappropriate to be used for detecting the motion of the entire image. By using only the transformation coefficients that satisfy the relationships of equations (1) and (2) above for motion amount prediction, it is possible to eliminate motion in areas different from the motion of the entire image and accurately estimate the motion of the entire image. can be detected.

FIG. 3 is for illustrating a method for determining the motion of an image by evaluating coefficients obtained from three motion vectors. It is assumed that the motion vectors of four regions of the image have been obtained as shown in FIG. 3(a). This corresponds to, for example, a case where an object in the upper right area of the screen moves significantly to the left while the camera is rotating to the left about the center of the screen. When three of the four motion vectors are selected and the conversion coefficients are calculated, the results are as shown in Figure 3 (b), (c) and (d).
does not satisfy the relationships of equations (1) and (2) described above, and it can be determined that only the combination shown in (d) in the figure is valid. Therefore, by excluding the motion vector in the upper right area of the screen that has a motion different from that of the whole image, the motion of the entire image can be detected with high accuracy.

Transformation coefficients are obtained by the number of combinations of detected motion vectors, in this example, the number of combinations in which three motion vectors are selected from n motion vectors. Typically, such classification has a large number of dimensions and the determination of parameters,
Although it is difficult in terms of the number of operations, as in this example, the motion vector is predicted from the transformation coefficient, and the number of regions where the error between the predicted motion vector and the detected motion vector is smaller than a predetermined value is maximized. By finding a region and using only detected motion vectors belonging to this region, it is possible to easily classify regions belonging to representative motions.

FIG. 4 is a diagram for explaining the operation of the motion area extraction circuit (candidate area coefficient circuit 81 and maximum area selection circuit 82) in this embodiment. FIG. 4(a) is a diagram showing an outline of the detected 16@ motion vectors when the screen is divided into 16 areas, and most of the image is translated from left to right. Areas 9.13.14 and 16 behave differently. In such a case, for example, area 1.2
．． An example of an area (vc complement area) in which the error at each position predicted from the motion vector of 5 is smaller than a predetermined value is shown in FIG. 4(b). At this time, the number of candidate areas is 13, and it can be seen that the movement of area l, 2.5 represents the movement of most of the image. On the other hand, the candidate area when predicted from the motion vector of area 9.13.14 is as shown in Figure 4 (C), the number of candidate areas is 3, and the movement of area 9.13.14 is the same as the overall movement. It can be seen that this is an isolated movement. Therefore, the representative motion of the entire image can be determined more accurately by excluding the motion vectors of the isolated regions 9, 13, and 14 and using the motion vectors of other regions.

The overall operation of this embodiment will be explained below with reference to FIG.

(Step S1) The motion vector detection circuit 2 first obtains n motion vectors for each area.

(Step “S2) The maximum number of candidate areas is set to O.

(Step S3) The motion vector selection circuit 3 selects three combinations from among the n motion vectors.

(Step 54) The number of vc complement areas is set to 0.

(Step S5) The conversion coefficient calculation circuit 4 calculates the conversion coefficients a - f.

(Stella 7”S6) The coefficient evaluation mill VFI41 evaluates the effectiveness of the conversion coefficients by determining whether the movement of the image is parallel translation, rotation, enlargement, etc.
Evaluate and determine whether it is a reduction.

(Step ''57) The coefficient selection circuit 42 selects one region to be evaluated.

(i) S8) The motion amount prediction circuit 5 predicts a motion vector using only valid motion vectors. (Step S9) The M difference detection circuit 6 detects the error between the detected motion vector and the predicted motion vector.

(Step 510) The threshold circuit 7 determines whether the error is below a predetermined threshold.

(Step 5ll) The fPA complementary area counting circuit 81
If the error is less than or equal to the threshold, the number of candidate regions is increased by 1.

(Switch 512) Determine whether all areas to be evaluated have been completed.

(Sunfubu 513) If there are still areas to be evaluated, select the next area and return to (Stevebu S7).

(S7"514) Determine whether fP?c is larger than the maximum number of candidate regions in the complement region.

(Step S15) If the number of candidate regions is greater than the maximum number of candidate regions, the maximum number of candidate regions is updated.

(Step "316") It is determined whether all motion vector combinations have been completed.

(Step 517) If there are still combinations remaining, the combinations are updated and the process returns to step S4.

(Step 818) Among all the combinations, the maximum area selection circuit 82 selects only the motion vector of the area with the maximum candidate area, and using these motion vectors, the motion amount determination circuit 9 determines the image. The overall motion is determined and output to the motion amount output terminal 10.

In this way, a conversion coefficient representing the movement of both images is calculated using the three motion vectors, and based on the coefficient value of this conversion coefficient, it is determined whether the movement of the image is a combination of translation, rotation, reduction, and enlargement. By doing so, the movement of the entire image can be detected with high accuracy by excluding the movement of areas different from the movement of the entire image. In addition, by finding the region where the number of regions where the error between the predicted motion vector and the detected motion vector is smaller than a predetermined value is the maximum, it is possible to eliminate the influence of motion in regions that are significantly different from the predicted motion. It is easy to detect the overall representative movement with high accuracy.

Note that the present invention is not limited to the above embodiments, and various modifications can be made based on the spirit of the present invention. For example, the movement of the entire image may be calculated from the amount of movement of 4 or more.

As described in detail, the present invention provides the following effects.

(1) Since the movement of the entire image is calculated as a conversion coefficient from two or more detected amounts of movement, the direction and size of the motion vector obtained from the position of the screen, such as rotation, enlargement, reduction, etc. Even if the motions are different, the same conversion coefficients are obtained for the same motion, making it possible to identify regions belonging to the same motion.

(2) Based on the calculated conversion coefficients, the amount of motion at other positions in the image is predicted, and areas where the difference between the predicted amount of motion and the amount of detected motion is smaller than a predetermined threshold are extracted. Parts with similar movements are obtained as the same area,
It is possible to detect the movement of the entire image, including translation, rotation, enlargement, and reduction, without being affected by the movement of some areas that differ from the overall movement.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of an image motion detection device according to a first embodiment of the present invention, FIG. 2 is a block diagram of an image motion detection device according to a second embodiment of the present invention, and FIG. 3 is a block diagram of an image motion detection device according to a second embodiment of the present invention. FIG. 4 is a vector configuration diagram for explaining the coefficient evaluation in the second embodiment of the present invention, FIG. 4 is a vector configuration diagram for explaining the operation of motion area extraction in the second embodiment of the present invention, and FIG. A flowchart of the operation in the second embodiment, and FIG. 6 is a block diagram of a conventional image motion detection device. l... Image input terminal, 2... Motion vector detection circuit, 3... Motion vector selection circuit, 4... Conversion coefficient calculation circuit, 5... Motion amount prediction circuit, 6... Error detection Circuit, 7... Threshold circuit, 8... Motion area extraction circuit, 9
...Motion amount determination circuit, 10...Motion amount output terminal, 4
1... Coefficient evaluation circuit, 42... Coefficient selection circuit, 81
. . . Candidate region coefficient circuit, 82 . . . Maximum region selection circuit. Agent Patent Attorney Matsu 1) Tadashi Michi Figure 3 Figure 4

Claims (3)

[Claims] (1) motion vector detection means for detecting the amount of motion of a plurality of areas on the screen; transformation coefficient calculation means for calculating a transformation coefficient representing the movement of the image using a plurality of the detected amounts of motion; a motion quantum measuring means for calculating predicted motion amounts in a plurality of regions on the screen using conversion coefficients; and a region where an error between the predicted motion amount and the detected motion amount of the plurality of regions is less than or equal to a predetermined value. 1. An image motion detection device comprising: a motion region extracting means for extracting a motion region; and a motion amount determining means for determining a motion of the entire image using the motion amount of the extracted region. (2) The motion region extracting means calculates, for each of the predicted motion amounts calculated above, the number of regions in which the error between the predicted motion amount and the motion amounts detected in a plurality of regions on the screen is equal to or less than a predetermined threshold. 2. The image motion detection apparatus according to claim 1, wherein the motion amount of the region that gives the maximum number of regions is selected. (3) The conversion coefficient calculation means calculates a conversion coefficient using the amount of movement of three or more areas on the screen, and the motion quantum measurement means determines the effectiveness of the conversion coefficient when the movement of the image is parallel or Determining by evaluating whether it is rotation, expansion, reduction, or a combination thereof, and performing prediction in multiple areas on the screen using the remaining transformation coefficients by excluding the invalid transformation coefficients. 2. The image motion detection device according to claim 1, wherein the image motion detection device calculates a motion amount.