JPH01256290A - Moving vector detector - Google Patents

Abstract

PURPOSE:To accurately obtain a fluctuating moving vector of an image pickup device itself by judging the presence or absence of rocking of the image pickup device by an angular velocity detector and obtaining the moving vector by picture processing. CONSTITUTION:The angular velocity detector 1 detects the rocking of the image pickup device in the yaw and pitch axis directions, and when a moving judging circuit 2 is larger than a prescribed level, it is regarded as the presence of the rocking of the image pickup device, and a Read/Write control circuit 7 is used to write an evaluation data into an evaluation data memory sequentially. When the write is finished, a minimum value address detection circuit 9 evaluates the content of the evaluation data memory 8 to obtain the center address of the area in which the data in the memory is small. The difference between the center address and the address of a reference point A represents the moving vector. As the method of evaluation, low pass filtering is applied to the content of the evaluation data memory 8 to eliminate the noise component and to obtain the area where data with smaller values than the prescribed level are collected. Thus, the moving vector due to rocking of the image pickup device itself is detected accurately.

Description

DETAILED DESCRIPTION OF THE INVENTION Field of the Invention The present invention relates to a motion vector detection device that detects the direction and magnitude of screen blur caused by shaking of an imaging device.

BACKGROUND OF THE INVENTION Conventional motion vector detection devices include, for example, an angular velocity sensor using a vibration or rotating gyro attached to an imaging device, which mechanically detects pitch and yaw movements and converts them into electrical signals. There is a method that uses conversion to measure the amount of screen blur and convert it into a motion vector. There is also a method of measuring the amount of screen blurring and obtaining a motion vector (zlz) by performing pattern matching or correlation calculation between the current captured screen and the captured screen one or more fields before. Then, using this motion vector that indicates the magnitude and direction of screen shake due to shaking of the imaging device, the shaking of the captured screen due to camera shake during image capture is corrected, and the viewing speed sensor is used.
By detecting the movement of the screen due to camera shake and controlling the imaging direction to eliminate this, the shaking of the image capture screen due to camera shake is canceled out.

Problems to be Solved by the Invention However, in the above-mentioned motion vector detection device, in the former case, the angular velocity sensor itself is required to have high resolution and low temperature drift, and the detection sensor becomes large and expensive.
It had problems such as: In the latter case, for example, if you take an image of a person walking in front of a wall like this, if the person moves within a considerable range of the screen using only image processing, the image capturing device will move due to camera shake and the subject will appear on the screen. This poses a problem in that it is impossible to distinguish whether the subject is moving upward or only the subject is moving.

In view of this, an object of the present invention is to provide a motion vector detection device that detects a motion vector based on the swinging of the imaging device itself.

Means for Solving the Problems The present invention provides a first motion detection device that detects the presence or absence of movement of an imaging device using an angular velocity or acceleration detector attached to the imaging device; Motion vector)/1 characterized by comprising a second motion detection device that obtains a motion vector using an image signal and a current image signal.
/Detection device.

According to the above-described configuration, when the first motion detection device detects the shaking of the imaging device, the second motion detection device uses the image signal of the reference point one field or more earlier and the current image signal. By performing pattern matching, a motion vector due to the shaking of the imaging device is detected.

Embodiment FIG. 1 shows a block diagram of a motion vector detection apparatus in a first embodiment of the present invention. In Figure 1, 1
2 is an angular velocity detector that detects the shaking of the imaging device; 2 is a motion determination circuit that uses the output of the angular velocity detector 1 to determine whether the imaging device has swung; 3 is a standard for storing image data of reference points; A point image memory, 4, a reference point timing generation Read/Write control circuit that performs control to obtain the reference point timing, write image data to the reference point image memory, and read it when calculating the difference with the image signal several fields later; is a subtraction circuit, 6 is an absolute value circuit, and 7 is a Read/W that controls writing and reading of the evaluation data memory 8.
rite control circuit, 8 is a memory for evaluation data, 9 is a minimum value address detection circuit, and 10 is an address/motion vector conversion circuit.

The operation of the motion vector)/1/detection device of this embodiment configured as described above will be explained below. The reference point timing generation Read/Write control circuit 4 writes the image signal at the reference point A of the input image signal into the reference point image memory 3, and outputs the value in the next frame. Then, the subtracter 6 calculates the difference between the current image signal and the image signal one frame before at the reference point A, and the absolute value circuit 6
Convert to absolute value. This data is used as evaluation data.

This value should be smaller the closer the image correlation is to the point, ie, the position after movement.

In FIG. 2, it is assumed that the subject Q moves to σ on the screen in one frame due to the swinging of the imaging device. The yaw axis of the imaging device is determined by the angular velocity detector 1.

A swing in the direction of the pitch axis is detected, and when this swing is larger than a predetermined level by the motion judgment circuit 2, it is determined that there is a swing of the imaging device, and the Read/Write control circuit 7 is used to read the above-mentioned evaluation data. are sequentially written into the evaluation data memory 8. When writing is completed, the minimum value address detection circuit 9 evaluates the contents of the evaluation data memory 8 and finds the center address of the area where the data in the memory becomes smaller.

The difference between this center address and the address of reference point A represents a motion vector. As an evaluation method, evaluation data memory 8
By performing low-pass filtering on the contents of the image, noise components are removed, and then the area where data having values smaller than a predetermined level are gathered (the shaded area in FIG. 2) is determined. Then, calculate the average of the addresses corresponding to this area, or the weighted average of the addresses with a large data weighing value that is -1 lower than the value of the evaluation data, and calculate the area in the evaluation data memory 8 where the small value exists. You can find the center address of

Using the center address B(xl, yl) and the reference point address (xo, yO) obtained as described above, the address/motion vector conversion circuit 10 obtains the motion vector m using the following equation.

m= (xl - xo, yl - yO) Furthermore, when the motion determination circuit 2 determines that there is no shaking of the imaging device, the Read/Write control circuit 7 does not need to write to the evaluation data memory 8; , the address/motion vector conversion circuit 1Q outputs information on the motion vector)/VO.

In addition, when a motion vector cannot be obtained in the above image processing, that is, when all the low-frequency components of the contents of the evaluation data memory 8 are greater than a certain value, or when the area where data lower than the predetermined level μ exists, that is, the When the area where the address seems to fall is wider than a certain predetermined width, it is assumed that the motion vector is unstable,
The motion vector obtained from the previous calculation is the current motion vector)
Output as /M. As described above, according to this embodiment, by detecting the shaking of the imaging device itself using the angular velocity detector 1, it is possible to determine whether the subject has moved or the imaging device has moved. It is possible to detect motion vectors based on motion.

Further, FIG. 3 shows a block diagram of a motion detection device according to a second embodiment of the present invention.

In FIG. 3, the difference from FIG. 1 is that the motion judgment circuit 2
The only difference is that the control based on the output of is applied only to the address/motion vector conversion circuit 1o, and since the other configurations are the same, common elements are given common numbers. With this configuration, the Read/'Write control circuit 7 writes evaluation data to the evaluation data memory 8 regardless of the judgment of the motion judgment circuit 2.

Similarly to the first embodiment, the minimum value address detection circuit 9
．． A motion vector is determined from the contents of the evaluation data memory 8 by the address/motion vector/l/conversion circuit 1o. Here, when the motion determination circuit 2 does not detect any shaking of the imaging device, the address/motion vector conversion circuit 10 outputs a motion vector 0, assuming that there is no shaking of the imaging device.

As described above, according to the present embodiment, the second motion detection device always operates, but unless the first motion detection device detects the shaking of the imaging device, it outputs the motion vector 0,
The same operation as in the first embodiment can be realized.

FIG. 4 is an explanatory diagram of a detection method in a third embodiment of the present invention. Further, the configuration of the motion vector detection device is shown in FIG. In FIG. 6, 1 is an angular velocity detector, 2 is an area discrimination circuit, 3 is a reference point image memory, 4 is a reference point timing generation, Read/Write control circuit, 6 is a subtraction circuit, and 6 is an absolute value circuit. , 7 is Read/Write
a control circuit; 8 is a memory for evaluation data; 9 is a minimum value address detection circuit; 1o is a motion vector calculation circuit;
Since the above configuration is similar to that in FIG. 1, common elements are given the same numbers. The difference from the configuration shown in Fig. 1 is that the screen is divided into several regions, motion vectors are detected in each divided region, and a newly installed determination circuit 11 detects motion vectors other than the undetectable regions. This is the point where motion vectors can be obtained more reliably by performing averaging processing using only motion vector data in divided regions where vectors have been successfully detected.

The operation of the motion vector detection device of the third embodiment configured as described above will be described below. In FIG. 4, the outermost frame represents the entire screen. This is divided into several screens (in this embodiment, it is divided into 6 screens A-F). In each of the divided screens, a motion vector is detected using the method of the first embodiment.

That is, when the motion determination circuit 2 determines that there is a shaking of the imaging device, it sequentially writes the absolute value of the difference between the current image data and the image data of the reference point one field or a plurality of fields ago into the evaluation data memory 8; The contents are evaluated by the minimum value address detection circuit 9 to obtain the center address.

Furthermore, when the motion determining circuit 2 determines that there is no rocking of the imaging device, the motion vector )/l/ is set to 0.

Here, as shown in Fig. 4, the subject is divided into divided areas ■, O
In the --like screen, in the divided area ■, the subject Q moves to α due to the shaking of the imaging device, in the divided areas ◎ and @, the subject T moves to T' due to the shaking of the imaging device, and in the divided area ■, the subject R moves. Suppose that it also moves and moves to R'. In this case, in the case of ◎ in the divided area, the above-described method of calculating the correlation between the image data does not show any correlation in the contents of the evaluation gator memory 8, making it impossible to obtain a motion vector. In the divided area (2), a motion vector can be obtained although there is an error due to the movement of the subject R itself.

Furthermore, in the divided areas ■, ◎, and ■, the subject is stationary, and the motion vector can be detected.

Therefore, the judgment circuit 11 extracts only the motion vectors of the divided regions ■, ◎, ■, ■, in which the motion vectors) /1/ were detected, or the evaluation data of the divided regions, and calculates the average value of them. The final output is the movement pattern) /1/
get.

Here, the motion vectors in the divided regions ■, ◎, and @ are almost the same, but the motion vector in the divided region [F] has an error because the subject R itself is also moving. Therefore, among the motion vectors obtained in each divided region, those with components greater than or equal to a certain reference value are excluded and averaged, and the final output, the motion vector, is obtained, which further improves the detection accuracy. . Further, if the motion vectors )/& detected in each divided region are slightly different from each other, it can be determined that the imaging device is rotating.

As explained above, according to this embodiment, the screen is divided,
The motion vector is detected in each divided area, and the motion vector in the successfully detected area is detected using only the evaluation data of that divided area. The motion vector can be detected with high accuracy even when the subject itself moves.

In the above embodiment, the image signal is considered as data for each pixel, but multiple pixels may be grouped into one block and processed using the average value, integrated value, or representative value. . In addition, one reference point is set in one area.
Although it has been described as one reference point, the motion vector may be calculated by using one or more reference points, performing the above-mentioned processing corresponding to each reference point, and cumulatively averaging the results on the evaluation data memory 8. Also, as the reference point image signal, we used a signal from one frame before, but if it is one field or more before,
You can use anything.

Further, in the above-described embodiment, the first motion detection device is
An angular velocity detector was used to determine the presence or absence of rocking of the imaging device based on the magnitude of the angular velocity, but it is also possible to use an acceleration detector to determine the presence or absence of rocking of the imaging device based on the magnitude of acceleration caused by the rocking of the imaging device. good. That is, there is no problem as long as the shaking of the imaging device is detected.

In addition, motion vectors that indicate the direction and magnitude of screen shake due to shaking of the imaging device itself can be used to correct shaking of the imaging screen due to camera shake, and can also be used as data indicating the movement of the imaging device. . For example, when there is movement, the method of band compression of image data can be changed, or the position of the subject can be measured by measuring the movement of the imaging device.

As described in detail, according to the present invention, a motion vector is obtained through image processing after determining the presence or absence of shaking of the imaging device using an angular velocity or acceleration detector. The motion of the subject can be determined, and the motion vector caused by the shaking of the imaging device itself can be accurately obtained. Furthermore, since the angular velocity or acceleration detector only needs to determine the presence or absence of rocking, high detection accuracy is not required, and the cost and size of the detector can be reduced.

Therefore, by using this, it is possible to cancel out the shaking of the imaging screen due to camera shake, and to obtain a blur-free imaging screen with certainty.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of a motion vector detection device according to a first embodiment of the present invention, FIG. 2 is an explanatory diagram of the operation of the same embodiment, and FIG.
FIG. 4 is a block diagram of a motion vector detection device according to a second embodiment of the present invention, FIG. 4 is an explanatory diagram of the operation of a third embodiment of the present invention, and FIG. 5 is a diagram of the structure of the same embodiment. 1...Angular velocity detector, 2...Movement judgment circuit, 3...Reference point image memory, 4...Pa...
・Reference point timing generation, Read/Write control circuit, 5... Subtraction circuit, 6... Absolute value circuit, 7... Read/Write control circuit, 8... ...Memory for evaluation data, 9...
- Minimum address detection circuit, 10...address/motion vector conversion circuit, 11...determination circuit. Name of agent: Patent attorney Toshio Nakao and 1 other person 2nd
Figure 4

Claims (4)

[Claims] (1) A first motion detection device that detects the presence or absence of movement of the imaging device using an angular velocity or acceleration detector attached to the imaging device, and an image signal of a reference point one field or more earlier and a current image. A motion vector detection device comprising a second motion detection device that obtains a motion vector using a signal. (2) The motion vector detection device according to claim 1, wherein the first motion detection device sets the motion vector to O when the first motion detection device determines that there is no movement of the imaging device. (3) Claim 1 characterized in that when the second motion detection device cannot make a determination, the immediately previous determination content is used as a motion vector.
Or the motion vector detection device according to 2. (4) A first motion detection device, a second motion detection device that divides the imaging screen into a plurality of screens and detects a motion vector in each divided screen, and detects a motion vector in each divided screen. A motion vector detection device comprising: a determination device that determines a motion vector of the entire screen using .