JPH0410780A - Image pickup device - Google Patents

Abstract

PURPOSE:To reduce the size, weight and cost of the image pickup device by using the information of variation corresponding to the time change of a moving vector detected from an image signal by image signal processing. CONSTITUTION:The image pickup device is provided with a detection means 30 for detecting the moving vector of an image from an input image signal, the 2nd detecting means 32 for detecting the time variation of the means 30 and a deciding means 38 for evaluating the time variation outputted from the means 32 and detecting panning. Thereby it is unnecessary to prepare active sensors such as an angular speed sensor, a rotary gyro and an accelerometer and the vibration of an image can be purely electronically detected from the image signal. Consequently, the size, weight and cost of the image pickup optical device can be reduced.

Description

Detailed Description of the Invention (Industrial Application Field) The present invention relates to imaging optical devices such as television cameras, electronic cameras, and industrial image measuring equipment, and particularly to imaging optical devices having functions of image stabilization and automatic subject tracking light. It is related to the device.

(Prior Art) Conventionally, the panning operation in the above-mentioned imaging optical device has been performed by, for example, F National Techn.
ical Report Vol. 34. No, 6.
Dec, 1988, Q, the imaging optical device is provided with an angular velocity sensor for determining the vertical angular velocity of the imaging optical device body and an angular velocity sensor for determining the horizontal angular velocity, and the imaging optical device Dynamically detect the angular velocity of the vertical and horizontal components of
It is determined whether panning is being performed based on the frequency, its duration in one direction, etc.

(Problems to be Solved by the Invention) However, in this type of conventional example, in order to detect the angular velocity of the imaging optical device main body, a sensor for determining the angular velocity in the vertical direction and a sensor for determining the angular velocity in the horizontal direction are used. Since at least two sensors are required independently, their installation requires space and circuit components for drive control of each sensor, making it possible to reduce the size, weight, and cost of the imaging optical device. However, it had problems such as damage.

(Means for Solving the Problems) The present invention has been made for the purpose of solving the above-mentioned problems, and is characterized by a method for detecting a motion vector of an image from a human-powered image signal. a second detection means for detecting a temporal variation of the motion detection means; and a discrimination means for detecting panning by evaluating the temporal variation output from the second detection means. It is located in the imaging device.

(Function) This eliminates the need to provide active sensors such as angular velocity sensors, rotary gyros, and accelerometers as in the past.
, image blur can be detected purely electronically from the image signal, which is advantageous in reducing the size, weight, and cost of the imaging optical device.

In addition, since the movement of the image is detected using the image signal, it is easy to combine with automatic focus detection and automatic exposure control using other image signals, and it is also effective for intelligent image processing required for shooting. It is.

(Embodiment) Hereinafter, an embodiment of the imaging apparatus according to the present invention will be described in detail with reference to the respective figures.

FIG. 1 shows a block diagram of a video camera incorporating an image blur correction device to which the present invention is applied.

In the figure, 10 is a subject, and 12 is a variable apex angle prism that can change the optical axis direction, for example, a prism whose periphery is sealed by filling silicon or the like between glass plates on flat plates arranged parallel to each other. By changing the angle of the flat glass plate, the optical axis direction can be changed and image blur can be corrected. 14 is a photographing lens system, 16 is an image sensor made of, for example, a two-dimensional COD, and 18 is a gamma correction for the output image signal of the image sensor 16;
This is a signal processing circuit that performs processing such as blanking processing and addition of synchronization signals.
A standard television signal of the C standard is output. Also, during the output, Y is a luminance signal, H, 5YNC is a horizontal synchronization signal,
V, 5YNC are vertical synchronization signals.

22 is a delay circuit that delays the luminance signal Y output from the signal processing circuit 18 by a predetermined time; for example, F I F
It consists of an O (first-in first-out) type field memory. 24 is a block division pulse generation circuit that generates gate pulses for dividing the video signal being scanned into predetermined blocks set on the screen;
6.28 is the pulse generation circuit 2 that divides the luminance signal Y into blocks.
This is a dividing circuit that divides the screen according to the output pulse No. 4 according to the divided areas set on the screen. That is, the dividing circuits 26 and 28 collectively output the input luminance signals for each block set on the screen, and specifically output the gate circuits whose opening and closing are controlled by the output pulses of the block dividing pulse generating circuit 24, and the gates. It consists of a memory that stores signals passing through the circuit.

30 is a motion vector detection circuit which compares the signal of the current screen with the signal of the screen a predetermined time ago outputted from the delay circuit 22, and calculates a motion vector for each block into which the screen is divided; An area setting circuit 38 determines the area to be subjected to image stabilization in units of minimum blocks set on the screen, and 38 is based on motion vectors existing within the area to be image stabilized set by the area setting circuit 32; A determination circuit that determines panning operations; 34 is an area setting circuit 3;
2 and a delay circuit for determining timing by delaying by the time consumed for signal processing in the determination circuit 38;
Reference numerals 6a and 36b are analog switches that turn signals ON and OFF in accordance with the outputs of the area setting circuit 32 and determination circuit 38, respectively. That is, the analog switch 36a turns on the signal line in order to pass only the motion vector information corresponding to the area on the screen set by the area setting circuit 32.
, the OFF switch 36b is turned ON and OFF according to the output of the determination circuit 38, and is turned OFF when it is determined that a panning operation is being performed, and is turned ON when it is determined that there is a camera shake, and the motion vector information is turned OFF. This is a switch that allows the signal to pass through to a drive circuit 40, which will be described later.

42 is an actuator that changes the angle of the two flat glasses of the variable apex angle prism 12 to change the apex angle; 40 is an actuator that drives the actuator 42 in accordance with an output representing the amount of camera shake detected by the motion vector detection circuit 30; This is a drive circuit that

Thereby, by changing the apex angle of the variable apex angle prism 12 based on the amount of camera shake detected from the motion vector of the image, it is possible to control the deviation angle of the output optical axis of the prism 12 with respect to the input optical axis.

Next, actual camera shake detection and panning detection operations in the imaging apparatus shown in FIG. 1 will be explained.

The subject image that has passed through the variable apex angle prism 12 and the photographing lens is incident on the image sensor 16, photoelectrically converted, and an image signal is output. The signal processing circuit 18 then performs the above signal processing on the image signal output from the image sensor 16,
Convert to a standardized standard television signal.

The luminance signal Y output from the signal processing circuit 18 is supplied to a video recorder or monitor display (not shown), is directly applied to the division circuit 28, and is also applied to the division circuit 28 for one field period (about 1 f3.
7m5ec) is delayed and applied to the dividing circuit 26.

In the division circuits 26 and 28, the block division pulse generation circuit 2
According to the gate pulse output from 4, the entire screen is mXn
The luminance signal Y can be extracted for each block.

The motion vector detection circuit 30 performs mx by image signal processing.
Find the motion vector for each of the n blocks.

Detection of motion vectors from image signals involves, for example, sampling the features of the image using multiple locations on the screen as representative points, comparing the features between temporally different fields, and calculating the displacement of the correlation between the feature points. The so-called representative point matching method, which calculates the motion vector representing the movement from the image, or the screen is divided into many blocks like a mesh, and for each block, the density difference between temporally different images and the spatial gradient within the block are calculated. The motion vector can be calculated using the spatiotemporal gradient method, etc., which calculates the motion vector from the relationship.

The spatio-temporal gradient method is described by B., P. Horn et al.
“Artificial Intelligence”
ical intelligence) 17,
p. 185-p. 203 (1981) Jl, and real-time processing is possible using dedicated hardware.

The motion vector for each block on the entire screen obtained in this way is called an optical flow.

The motion vector obtained by the motion vector detection circuit 30 is statistically processed by the image stabilization area setting circuit 32, which determines an area to be anti-shake (hereinafter referred to as an anti-shake area), and an anti-shake area is set.

Then, based on the displacement of the motion vector in the anti-shake area set by the anti-shake area setting circuit 32 according to the time change, the panning judgment circuit 38 determines whether the movement of the image is due to camera shake or due to a panning operation. A distinction is made.

This procedure will be described later using the flowchart shown in FIG.

When the panning determination circuit 38 determines that the image movement is due to camera shake, information on the amount of image movement output from the motion vector detection circuit 30 is applied to the drive circuit 40, and the actuator 42 The variable apex angle prism 12 is driven in a direction in which the output (amount of camera shake) of 30 is small (i.e., the movement of the screen due to camera shake is reduced), thereby correcting camera shake.

On the other hand, if the panning determination circuit 38 determines that the movement of the image is due to a panning operation, the camera itself is moving, so vibration-proofing should be performed to correct the movement of the image. Since this would cause an inconvenience, no voltage is applied to the drive circuit 40, and the operation of the anti-vibration function is stopped.

FIG. 2 is a flowchart showing the operation of the panning determination method from motion vectors (panning determination circuit 38) in FIG.

After the panning determination circuit starts operating, the initial setting of the time measurement counter (I=O) is performed in step 1, and step 5
If the counter value exceeds the predetermined threshold th+ in step 2, it is determined that the photographer is performing a panning operation, and the automatic image stabilization function (hereinafter referred to as AS: AutO5jabiliz) is activated.
(abbreviated as cation) is stopped at 5tep 3.

Specifically, the analog switch 36b is opened and the supply of control information to the drive circuit 40 is stopped in accordance with the information of the determination result.

If the counter value does not exceed the threshold th+ at 5tep 2, if the image movement is due to a cause other than panning, the process moves to 5tep 4, and the direction θ= of the motion vector v= (a, b) in the anti-shake area is determined. jan-'(' b
/a) A comparison is made between the time change Δθ and a predetermined threshold value tho.

Here, if Δθ≧tho, the main component of the motion vector is determined to be camera shake, and the AS function is activated in step 5. Specifically, the analog switch 36b
is turned ON, motion vector information is supplied to the drive circuit 40, the actuator 42 is driven according to the direction and magnitude of the motion, and the apex angle of the variable apex angle prism 12 is controlled in a direction that cancels the motion of the image. do.

Conversely, if Δθ< t h a, the following 5 steps 6
Proceed to, and the magnitude of the motion vector v = (a, b) τl =,
A comparison is made between the temporal change Δ1 of fa"+b" and a predetermined threshold value thv.

If Δ1 and 1≧thv, the AS function is activated at step 57.

In addition, in the case of Δ1l<thv, the unit time (16,7m
It was determined that a similar motion vector continued in 5ec), and
At step 8, the counter is incremented by 1, and the process returns to step 5, step 2, and this process is repeated thereafter.

That is, the counter ■ represents the duration of motion vectors that are in the same direction and have a substantially constant magnitude; 16.
It can be converted to actual time by multiplying by 7m5ec.

Note that even if the flowchart shown in Figure 2 is applied to each block created by dividing the input image, it will not be possible to calculate the total motion vector after statistical processing (for example, averaging processing) of the motion vectors detected in each block. May be applied.

According to the above-described embodiment, a case has been described in which the detection of a panning operation is determined based on the duration of a motion vector, but other methods may be used.

FIGS. 3 and 4 show other embodiments of the panning determination circuit in the imaging apparatus of the present invention, in which panning operation is detected based on the magnitude of the total vector obtained by adding motion vectors. It is something.

FIG. 3 shows a motion vector (code 1) detected when photographing due to camera shake or from a vehicle, and its direction exhibits a characteristic of randomly changing.

When multiple motion vectors are added in the time direction, since the directions are random, the total motion vector 2
The magnitude of is suppressed (code 2).

Next, FIG. 4 shows a motion vector (code 3) detected during a panning operation, and its direction is constant depending on the direction of movement of the camera.

Therefore, if a plurality of these are added in the time direction, the magnitude of the motion vector will be increased (reference 4).

Therefore, we set a threshold value for the size of a new motion vector that is created by adding multiple motion vectors in the time direction.
If it is determined that a panning operation is being performed when the threshold value is exceeded, it can be easily determined that a panning operation is being performed.

However, in this case, if an extremely large amount of camera shake occurs, the size of the new motion vector created by the addition will increase, and the camera may conclude that it was a panning operation.

Therefore, if a prominent motion vector is included in the fixed period of time during which motion vectors are added, it is desirable to simultaneously take measures such as excluding it from the addition target.

(Effects) As described above, according to the imaging device of the present invention,
Image blurring is achieved by using information on fluctuations in response to temporal changes in motion vectors detected from image signals through image signal processing.

It can accurately determine the state of the imaging device such as panning, and eliminates the need for external sensor mechanisms such as rotary gyros, accelerometers, and angular velocity sensors that were conventionally required for panning judgments, making it compact, lightweight, and low cost. It is possible to measure the

Furthermore, since the situation is detected from the image signal, it is possible to make the image signal processing more intelligent, including automatic focus detection, automatic exposure adjustment, etc. using other image signals, and its potential for development is great.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the configuration of an imaging device according to the present invention; FIG. 2 is a flowchart showing an algorithm for evaluating a motion vector detected from an image signal to determine the state of an image; FIG. FIG. 4 is a diagram for explaining another embodiment of the panning determination circuit according to the present invention. FIG. 3 shows how the motion vector changes over time during camera shake, and FIG. Each figure shows how the motion vector changes over time.

Claims (3)

[Claims] (1) a first detection means for detecting a motion vector of the image from an input image signal; a second detection means for detecting temporal fluctuations of the motion detection means; What is claimed is: 1. An imaging device comprising: determination means for detecting panning by evaluating temporal fluctuations. (2) In claim (1), the determining means is configured such that a motion vector variation difference in a corresponding region between temporally consecutive images is smaller than a predetermined value and the duration thereof is less than a predetermined value. An imaging device characterized in that the imaging device is configured to determine that a panning operation is being performed when the amount exceeds the limit. (3) In claim (1), the determining means adds motion vectors in corresponding regions of temporally continuous images for a certain period of time, and performs a panning operation when the motion vectors exceed a predetermined value. An imaging device characterized in that it is configured to determine that