JPH01218190A - Moving vector detecting device - Google Patents

Abstract

PURPOSE:To accurately detect a moving vector without necessitating a highly accurate angular speed or acceleration detector and only with the use of small- sized image memory by providing a first moving detection device using a detector for angular speed or acceleration given on an image pickup device and a special second moving detector. CONSTITUTION:An image signal at a reference point A(x1, y1) is stored in a reference point image memory 3 prior one frame. When the current image signal in the range shown by the broken line which extends from a position B determined by a moving vector ss obtained by an angular speed detector 1 and the reference point A is swept, the absolute value of the difference between the current image signal and the content in the memory 3 is sequentially calculated by a subtraction circuit 5 and an absolute value circuit 6, and the result is written in an evaluation data memory 8 sequentially. After the completion of the write, the content in the memory 8 is evaluated by a minimum value address detection circuit 9, and the address of the center of an area where the value of the data in the memory becomes less. As a result, a moving vector arithmetic circuit 10 can calculate a corrected moving vector.

Description

DETAILED DESCRIPTION OF THE INVENTION Field of the Invention The present invention relates to a motion vector detection device that detects the direction and magnitude of screen blur caused by shaking of an imaging device.

2. Description of the Related Art Conventional motion vector detection devices include, for example, attaching an angular velocity sensor using a vibration or rotating gyro to an imaging device, mechanically detecting pitch and yaw motions, and converting these into electrical signals. There is one that uses a method of converting it into a motion vector. There is also a method of obtaining a motion vector by performing pattern matching or correlation calculation between the current captured screen and the captured screen one or more fields before.

A device that uses this motion vector indicating the magnitude and direction of screen shake caused by shaking of the imaging device to correct the shaking of the captured screen due to camera shake during image capture and obtains an easy-to-see moving image is disclosed in, for example, Japanese Patent Laid-Open No. 143333-1983. There are examples such as Publication No. This uses the aforementioned angular velocity sensor to detect the movement of the screen due to camera shake, and controls the imaging direction to eliminate this, thereby canceling out the shaking of the image capture screen due to camera shake.

Problems to be Solved by the Invention However, in the above motion vector detection device,
In the former case, the angular velocity sensor itself is required to have high resolution and low temperature drift, and has the problem of increasing the size and cost of the detection sensor. Furthermore, in the latter case, pattern matching and correlation calculation must be performed on all the image data in the range where the motion vector that is considered to be used exists, which requires a large image memory and a long calculation time.

In addition, when detecting motion vectors using such image processing, if the S/N of the input image is poor, or if the subject is flat, it may be impossible to make a determination, or it may result in an erroneous determination.
There was a problem with this.

In view of this, an object of the present invention is to provide a motion vector detection device that detects a motion vector based on the swinging of the imaging device itself.

Means for Solving the Problems The present invention provides a first motion detection device using an angular velocity or acceleration detector attached to an imaging device, and an image signal of a reference point one field or more earlier and the first motion detection device. The motion vector detecting device includes a second motion detecting device that obtains a motion vector by using a current image signal near the expected movement position of the image signal given as an output.

According to the above-described configuration, the present invention obtains an approximate motion vector using a simple acceleration or angular velocity detection sensor, and obtains an image signal in the vicinity of the position after the movement based on this, and an image signal at the position before the movement. By evaluating the similarity of image signals from more than one field ago, a highly accurate motion vector can be detected using simple image processing.

Embodiment FIG. 1 is a block diagram of a motion vector detection apparatus in a first embodiment of the present invention, and FIG. 2 is an explanatory diagram of a motion vector detection method. In FIG. 1, 1 is an angular velocity detector, 2 is an area discrimination circuit, 3 is a reference point image memory, 4 is a reference point timing generation/read/write control circuit, 6 is a subtraction circuit, 6
1 is an absolute value circuit, 7 is a read/write control circuit, 8 is an evaluation data memory, 9 is a minimum value address detection circuit, and 1o is a motion vector calculation circuit. The operation of the motion vector detection device of this embodiment configured as described above will be described below with reference to FIGS. 1 and 2.

First, in FIG. 2, assume that the subject Q moves to Q' on the screen in one frame due to the movement of the imaging device. At this time, the purpose is to detect blindness as a true motion vector. First, based on the output of the angular velocity detector 1 shown in FIG.
Assume that 5=(Xs, Ys) is obtained as a motion vector. If the motion vector S is larger than a certain value, it is assumed that there is movement, and subsequent processing is performed. Hereinafter, vector components will be represented by uppercase letters, and coordinate components will be represented by lowercase letters. Although the accuracy of the angular velocity detector 1 is not very high, S should be close to the true motion vector m. Therefore, the reference point M(xl, 7
+) is stored in the reference point image memory 3 one frame before, and the position B (X
When the current image signal in the range of ±Δx in the X direction and ±Δy in the y direction indicated by the broken line is swept around + +xs + “!1+”s), the current image signal and the reference point image memory described above are Subtract circuit 6 for the absolute value of the difference from the contents of 3.
, are sequentially calculated by the absolute value circuit 6. Then, the calculation results are sequentially written into the evaluation data memory 8. Here, this memory capacity may be 2ΔxX2Δy in address space. After writing is completed, the contents of the evaluation data memory 8 are evaluated by the minimum value address detection circuit 9, and the center address of the area where the value of the data in the memory becomes small is determined. The evaluation method is to perform low-frequency filtrate processing on the contents of the evaluation data memory 8 to remove noise components, and then find the addresses of parts where data with values smaller than a predetermined value are conspicuously concentrated. . For example, the average of the addresses of points having data with a value lower than a predetermined value, or the data with a lower value,
By calculating the weighted average of the addresses with large weights, it is possible to calculate the center address of the area in the evaluation data memory 8 where small values exist.

The address space of the evaluation data memory 8 is designated by x' in Figure 2.
, y' coordinate, and if the center address obtained by the above method is the point (jOCo, 7o), then the difference between the true motion vector m and the motion vector S obtained by the output of the angular velocity detector 1 is The vector l is 1=cXa−Δ”z
3'o-Δy). Based on this difference vector l, the motion vector calculation circuit 1° corrects the motion vector W"j'+7=(Xs +Xo-Δ"+Ys
+7o−Δy).

As described above, according to this embodiment, rough motion vector information obtained from the output of the angular velocity detector 1 is used to evaluate the image data in the vicinity and the image data before movement one frame before. It is possible to generate highly accurate motion vectors without using large-scale image memory or arithmetic processing, and
If the size and direction of the motion vector can be determined to a certain extent without using a high-precision angular velocity detector that is costly,
Obtainable.

In addition, even if the S/N of the image is poor or the subject has no notable features and a motion vector cannot be obtained by image processing alone, the motion vector is determined to be undefined and the motion vector S
This can be solved by substituting .

Here, the state where the motion vector cannot be determined is when the low frequency component of the contents of the evaluation data memory 8 is greater than or equal to a total value, or
This is when the area where data lower than a predetermined level exists, that is, the area where the center address is thought to be, is wider than a certain predetermined width.

FIG. 3 is an explanatory diagram of a motion vector detection method in a second embodiment of the present invention. The configuration of the motion vector detection device is the same as that shown in FIG. 1, but the area discrimination method of the area discrimination circuit 2 is different.

The operation of the motion vector detection device of the second embodiment configured as described above will be explained below. Similarly to the first embodiment, in FIG. Suppose that the frame moves to Q' on the screen. First, from the output of the angular velocity detector 1, as a motion vector, ! ! = (Xs+Ys) is obtained, and if this vector is larger than a certain value, it is considered that there is movement.

Next, the motion vector direction given by the motion vector S and the reference point M("113'+") is set as an indicator straight line;
'I7t)=Ys()C-:1C1) If the angle between the motion vector S and the The region in the direction of the vector S, that is, the region within the broken line, is determined by the region determining circuit 2. In addition, when the angle between the motion vector S and the X-axis is other than that, it is ±ΔX in the
The area between the reference points and the direction of the motion vector S is determined. FIG. 3 is an example of the former.

When this area is swept, the absolute value of the difference between the reference point image signal stored in the reference point image memory 3 one frame before and the current image signal is calculated by the subtraction circuit 6 and the absolute value circuit 6. Then, the result is written into the evaluation data memory 8 together with address information indicating the current sweep position. After writing is completed, the contents of the evaluation data memory 8 are evaluated by the minimum value address detection circuit 9, and the center address of the area where the data in the memory becomes small is determined. The coordinates of this position are D(
"m + 7m ), the motion vector m to be found is m = c , the motion vector S is substituted.

As described above, according to the present embodiment, the capacity of the evaluation data memory 8 becomes larger than the first embodiment due to the writing of address information, but unlike the conventional example, it takes up most of the screen area. There is no need to handle it. Further, when the angular velocity detector 1 independently measures angular velocities in two directions, horizontal and vertical, the direction of the motion vector can be determined as long as the gains of each detector can be matched. That is, the first
Unlike the angular velocity detector used in the embodiment, it is not necessary to detect the rough direction and magnitude of the motion vector, but only the rough direction needs to be known. This reduces the requirement for the accuracy of the angular velocity detector 1, making it possible to reduce costs.

FIG. 4 is an explanatory diagram of a detection method in a third embodiment of the present invention. Further, the configuration of the motion vector detection device is shown in FIG. In Fig. 5, 1 is an angular velocity detector, 2 is an area discrimination circuit, 3 is a reference point image memory, 4 is a reference point timing generation/read/write control circuit, 5 is a subtraction circuit, 6 is an absolute value circuit, and 7 is a read /write control circuit, 8 is a memory for evaluation data, 9 is a minimum value address detection circuit, and 10 is a motion vector calculation circuit, which has the same configuration as in FIG. The difference from the configuration shown in FIG. 1 is that the screen is divided into several areas, motion vectors are detected in each area, and a newly installed determination circuit 11 detects motion vectors other than areas where detection is not possible. By performing averaging processing using only motion vector data in areas where detection was successful, motion vectors can be obtained more reliably.

The operation of the motion vector detection device of the third embodiment configured as described above will be described below. In FIG. 4, the outermost frame indicates the entire screen. Split this into several screens. (In this example, 6 pieces ~
In each of the divided screens (divided into [F]), motion vectors are detected using the method of the first embodiment or the second embodiment. That is, the current image signal near the end point of the motion vector S obtained by the angular velocity detector 1 when the reference point (x mark) of each area ■ to [F] is the starting point, and the reference point of one frame before. By correlating the image signals of the points, the motion vectors in each of the divided regions ① to [F] are detected.

Here, as shown in FIG. 4, the subject is in area ■.

In area O1, the subject Q moves to Q' due to the shaking of the imaging device, in area O1 ■, the subject T moves to T' due to the shaking of the imaging device, and in area [F], the subject R moves to Q'.
Suppose that it also moves and moves to R'. In this case, the area ■
, 0, no correlation appears in the contents of the evaluation data memory 8 and a motion vector cannot be obtained using the above-described method of calculating correlation of image data. Also, in area [F],
If the subject R' falls outside the shaded area, there is no correlation between the contents of the evaluation data memory 8, and a motion vector cannot be obtained. Furthermore, in areas (2), (2), and (2), the subject is stationary, and motion vectors can be detected.

Therefore, the determination circuit 11 extracts only the motion vectors of the area in which the motion vector has been detected or the contents of the evaluation data memory 8 for that area, and takes the average value of these, thereby determining the final output of the motion vector. obtain. Also,
If the motion vector ' cannot be detected in all areas, the motion vector S is used instead.

As explained above, according to this embodiment, the screen is divided,
The motion vector is detected in each divided area, and the motion vector is detected using only the motion vector in the successfully detected area or the image data of that area, so if a part of the screen is flat or Even when the subject itself moves, the motion vector can be detected with high accuracy.

In the above embodiment, the image signal is considered as data for each pixel, but multiple pixels may be grouped into one block and processed using the average value, integrated value, or representative value. . In addition, one reference point is set in one area.
However, the motion vector may be calculated by using a plurality of reference points, performing the above-mentioned processing in accordance with each reference point, and cumulatively averaging the results on the evaluation data memory 8. Further, although a signal from one frame before was used as the reference point image signal, any signal from one field or more before may be used.

In addition, in the second embodiment, the area indicated by the area discrimination circuit 2 is linear in the y-axis direction ±Δy or in the x-axis direction ±Δy.
Although ΔX is used, it is not necessary to be limited to this, and for example, it may be a fan-shaped area including the motion vector S with the reference point M as the apex. The discrimination region in the first embodiment may have any shape as long as it is in the vicinity of point B.

Further, in the above embodiment, the first motion detection device uses an angular velocity detector, and outputs the output from the angular velocity detector as the moving distance on the screen.
The expected movement position of the reference point was obtained by pyrotechnically converting the direction, that is, the motion vector S. Using an acceleration detector, this output was multiplied by a coefficient equivalent to the square of time to calculate the distance (length). The motion vector S may be obtained by converting into dimensions.

Further, in the above embodiment, if the output of the first motion vector detector, that is, S is smaller than a certain predetermined value, the second motion detection device does not operate and it is not considered that there is a motion, but S The second motion detection device may be operated regardless of the size of the motion detection device. That is, if the magnitude of S is 0, the correlation between the image signal near the reference point and the image signal at the reference point one field or more before is checked.

In addition, motion vectors that indicate the direction and magnitude of screen shake due to shaking of the imaging device itself can be used to correct shaking of the imaging screen due to camera shake, and can also be used as data indicating the movement of the imaging device. can. For example, when there is movement, the band compression method of image data can be changed, or the position of the subject can be measured by measuring the movement of the imaging device.

As described in detail, according to the present invention, accurate motion vectors can be detected without requiring a highly accurate angular velocity or acceleration detector and by using only a small-scale image memory.

Further, even if the S/N of the image is poor, the motion vector can be detected with an accuracy comparable to that of an angular velocity or acceleration detector. Thereby, it becomes possible to correct the fluctuation of the image capture screen and to obtain an image capture screen without blur.

[Brief explanation of the drawing]

FIG. 1 is a configuration diagram of a motion vector detection device according to an embodiment of the present invention, FIGS. 2FA to 4 are first operation explanatory diagrams in different embodiments of the present invention, and FIG. 5 is a different embodiment of the present invention. FIG. 2 is a configuration diagram of a motion vector detection device in FIG. 1... Angular velocity detector, 2... Area discrimination circuit, 3... Reference point image tamori, 4...
・Reference point timing generation φ read/write control circuit, 6
...Subtraction circuit, 6 ...Absolute value circuit, 7
...Read/write control circuit, 8...
Evaluation data memory, 9... Minimum address detection circuit, 1o... Motion vector calculation circuit, 11
...Judgment circuit. Name of agent: Patent attorney Toshio Nakao and 1 other person
211! J Figure 3 Figure 4

Claims (5)

[Claims] (1) A first motion detection device using an angular velocity or acceleration detector attached to an imaging device, and the image signal given by the image signal of a reference point one field or more earlier and the output of the first motion detection device. A motion vector detection device comprising: a second motion detection device that obtains a motion vector using a current image signal in the vicinity of an expected movement position. (2) The motion vector detection device according to claim 1, wherein the area near the expected movement position is an area expanded in the horizontal and vertical directions around the expected movement position. (3) The motion vector detection device according to claim 1, wherein the vicinity of the predicted movement position is the vicinity of a straight line given by the reference point and the predicted movement position. (4) A claim characterized in that the second motion detection device operates only when the output of the first motion detection device exceeds a certain value, and sets the motion vector to 0 if it does not operate. The motion vector detection device according to item 1, 2 or 3. (5) The motion vector detection device according to any one of claims 1 to 4, characterized in that when the second motion detection device cannot determine the motion vector, the output of the first motion detection device is used to obtain the motion vector. . (6) A first motion detection device, a second motion detection device that divides the imaging screen into a plurality of screens and detects a motion vector in each divided screen, and a second motion detection device that divides the imaging screen into multiple screens and detects a motion vector in each divided screen; A motion vector detection device characterized by comprising a determination device that determines a motion vector of an entire screen using an image signal in a divided screen in which a motion vector can be detected.