JPH04207480A - Moving vector detector - Google Patents

Abstract

PURPOSE:To utilize effectively space gradient information provided in a picture and to obtain a high space resolution by setting a moving vector arithmetic area adaptively and automatically in response to a spatial frequency of an input picture. CONSTITUTION:Detection means 4, 9 detect a moving vector of a picture depending on the relation between a change for a prescribed time in a picture density corresponding to an optional position of a picture and a space gradient of a picture signal corresponding to an optional position, and setting means 5, 6, 10, 11, 13, 15 set adaptively the size and shape of a unit arithmetic area outputting a moving vector in response to the spatial frequency of the input picture. Thus, the size and shape of an optimum block suitable for the spatial frequency in each part of the input picture are set automatically based on a simple evaluation criterion representing the spatial frequency of the input picture. Thus, the spatial gradient information given in the picture is effectively utilized and a high spatial resolution is attained.

Description

Detailed Description of the Invention (Field of Industrial Application) The present invention relates to a motion control system using image signals, which is suitable for use in image stabilization and object tracking in imaging devices such as video cameras, electronic cameras, and electronic still cameras. This invention relates to a vector detection device.

(Prior Art) Conventionally, a method of detecting the motion of an image by detecting a motion vector from an image signal is known.

Various methods for determining motion vectors have been proposed; for example, USP 3890462, Japanese Patent Publication No. 60-46
Publication No. 87111, J, 0. Limb and J,A
,Murphy"Measuring the 5pe
ed of Moving 0bject from
Te1evision Signals”, IE
EE Trans, Com,, Com-23, pp, 4
74-478 (April 1975), etc., there is a spatiotemporal gradient method.

In this spatio-temporal gradient method, the amount of motion of each point is calculated according to the following basic formula. That is, α=Σs d-sign(g'x)/Σ+i Ig
'xlβ=Σs d-sign(g'y)/Σa
Ig'yl However, a and β indicate the amount of movement in the X direction and the
, g'y respectively indicate the spatial gradient in the X direction and the X direction when the image is expressed by g. Also Σ
, means a summation operation within a block that is a unit operation area consisting of multiple pixels, and sign() is g', , g
This is a function that outputs the sign of 'u.

Further, the size and shape of the block are fixed regardless of the input image.

(Problem to be Solved by the Invention) However, according to the conventional example described above, the size and shape of the block are fixed, so in the area where the spatial frequency of the image is low, there is no need for calculation within the block. There was a drawback that spatial gradient information was insufficient, resulting in a significant deterioration in accuracy.

Conversely, in areas where the spatial frequency of the image is high, the detection range is narrow due to the influence of the fine period of the image, and although large blocks are not necessarily required, the motion vector is The disadvantage is that the spatial resolution for detecting is limited.

(Means for Solving the Problems) Therefore, the present invention provides a motion vector detection device that can adaptively and automatically determine an optimal block according to the spatial frequency of an image using a simple method. It was developed for the purpose of A motion vector detection device comprising a detection means for detecting a motion vector of an image, and a setting means for adaptively setting the size and shape of a unit calculation area for outputting the motion vector according to the spatial frequency of an input image. be.

(Function) As a result, it is possible to automatically set the optimal block size and shape adapted to the spatial frequency of each part of the human image using a simple evaluation standard (evaluation function) that indicates the spatial frequency of the input image. It is possible to effectively utilize the spatial gradient information possessed by

(Example) Hereinafter, the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of an imaging device according to the present invention.

In the figure, 1 is an input terminal for an image signal, 2 is a preprocessing filter composed of a low-pass filter for preprocessing the input signal, 3 is a memory for storing the image signal for one field period or one frame period, and 4 is a memory for storing the image signal for one field period or one frame period. 5 is a multiplier; 6 and 13 are summation circuits that sequentially add input data until a reset signal is input; , 14 is a latch circuit that holds the input data input from the summation circuits 6 and 13 until an enable signal is input, 8 is a memory that stores the image signal for only the period necessary to calculate the spatial gradient of the image, and 9 is a A code output circuit outputs only the code data of the input signal; 10 is a comparator for comparison with the 0 value of the input data; 11 is an absolute value output circuit for determining the absolute value of the spatial gradient of the image; 12
1 is a delay circuit for creating a time difference between the reset signal and the enable signal, 15 is a divider, and 16 is a motion vector output terminal.

Next, the operation of the circuit shown in FIG. 1 will be explained in order.

The image signal input from the image signal input terminal 1 is first supplied to a preprocessing filter 2, which is usually composed of a low-pass filter, and is processed to reduce the influence of noise contained in the image signal and to smooth out steep spatial gradients. will be carried out.

After passing through the filter 2, the image signal is branched into two paths, one of which is supplied to a memory 3 and a subtracter 4, where the image output signal of the filter 2 and the image signal delayed by a predetermined time in the memory 3 are combined. is subtracted, and the density difference between two temporally consecutive images, that is, the temporal gradient d, is calculated.

The temporal gradient d of the image output from the subtracter 4 is input to the multiplier 5, multiplied by the sign output from the sign output circuit 9, and then input to the summation circuit 6.

The data (= d-sign(g')) output from the multiplier 5 and input to the summation circuit 6 is cumulatively added until a reset signal is input, and when the reset signal is input, the data is transferred to the next latch circuit 7. is input to the divider 15 as the numerator of the division.

The other output of the filter 2 is also supplied to a memory 8 and a subtracter 4, respectively, and the image space gradient g° is obtained by subtracting the filter output and the image signal delayed by the memory 8.
is calculated. This image spatial gradient g' is further branched into three paths, one of which is input to the sign output circuit 9 for multiplying the temporal gradient d by the sign of the spatial gradient, and one which is input to the summation circuit 6. 13 and the subsequent stage latch circuits 7 and 14, respectively, are inputted to comparator unit 0 for determining the timing of sending a reset signal and an enable signal. In addition, the absolute value of the other one is taken by the absolute value circuit 11,
The summation circuit 13 performs cumulative addition until the reset signal generated by the comparator 10 is input, passes through the latch circuit 14, and is input to the divider 15 as the denominator for the division.

Then, in the divider 15, the time gradient d latched in the latch circuit 7 is matched with the sign s output from the sign output circuit 9.
The motion vector of the image is obtained by dividing the sum of d-sign(g') multiplied by ign(g') within the detection block by the sum of the absolute values of the spatial gradients within the detection block, 1g'1.

Further, the comparator 10 compares the value of the spatial gradient g° of the image with the O value, and when the value of g° becomes 0 or approximately O, a reset signal and an enable signal are generated.

Note that the reset signal needs to be generated later than the enable signal by the time required for the data held in the latch circuit 7.14 to be input to the divider 15. After passing through the delay circuit 12 for causing the signal to rise, the signal is sent to the summation circuit 6.13 as a heliset signal.

The configuration and operation of the motion vector detection device of the present application are as described above, and the unique functions and effects thereof will be explained below.

FIG. 2 shows an arbitrary cross section of the image.

In the figure, the vertical axis shows the image density value, and the horizontal axis shows the position in the image.

X written on the horizontal axis 11 Xs r Xs +...
..., Xs is g' fx+) "tg" (xi)
This is the point where JHp++Jg' (xa) 4=rO. In other words, if the point where the spatial gradient of the image is 0 or almost 0 is determined as the block boundary, then the area between X1%Xm is block 1.
．． The optimal block size and two-dimensional shape are automatically set according to the spatial frequency of the image, such as block 2 between x2 and x1.

In FIG. 2, the block size is set to 1/2 of the image period T, but the image period T may be used as is as the block size.

By doing this, the size of the block is increased in the low-frequency region where the spatial gradient information of the image necessary for the calculation of the spatio-temporal gradient method is scarce, thereby eliminating the problem that occurs from the low-frequency region. Large error vectors can be removed.

Conversely, in high frequency regions where there is sufficient spatial gradient information, it is possible to reduce the block size, reducing the redundancy of spatial gradient information within the block that occurs in conventional methods where the block size is fixed. can,
As a result, high spatial resolution can be obtained.

In addition, in the present invention, when a block is bordered by a point where the spatial gradient magnitude is approximately 0, the range where the spatial gradient magnitude is approximately O is excluded from the calculation area of the spatiotemporal gradient method. Become.

In other words, the above range corresponds to an area in the image where spatial gradient information is extremely poor, or a region where spatial gradient information is abundant (but the sign of the spatial gradient is reversed), and in principle, the above range corresponds to a region where the spatial gradient information is reversed. Since this is the area where the algorithm is least effective, the area can be automatically removed from the calculation area as a gap between each block.

As described above, according to the present application, it is possible to optimize the block for detecting spatial gradient information in a direction that increases the accuracy according to the spatial frequency of the input image, and it is possible to improve the accuracy of motion vector detection.

According to the above embodiment, the spatial gradient of the image is used as a means for determining the boundary of the detection block, but according to the second embodiment described below, the density difference between consecutive images is used to determine the boundary of the detection block. As in the previous embodiment, the adaptive block size and shape are automatically set according to the spatial frequency of the input image signal.

FIG. 3 is for explaining the second embodiment.
The difference from the first embodiment shown in the figure is that the comparator 10
This is only the input signal to.

According to this embodiment, the density difference between successive images, that is, the time gradient d, calculated by the memory 3 and the subtractor 4 is supplied to the comparator 10, and this time gradient d is compared with a 0 value, and the time gradient d is compared with the 0 value. When the value of the gradient d becomes O or almost 0, a reset signal is sent to the summation circuit 6 via the delay circuit 12, and an enable signal is sent to the latch circuit 7.

FIG. 4 is a diagram showing the state of this embodiment.

In the figure, the vertical axis shows the image density value, and the horizontal axis shows the position in the image.

x, , xg , ..., Xa written on the horizontal axis is the density difference between temporally continuous image times, that is, the time gradient d
is the point where becomes 0. The above point OC+, X2゜...
.

Figure 4 shows how blocks are divided into points with a time gradient d=o, but the point d=o is skipped once and the block boundary is made so that one block is approximately equal to one period T of the image. You can also do this.

Furthermore, in the present invention, when a block boundary is a point where the magnitude of the temporal gradient d is approximately 0, the range where the magnitude of the temporal gradient d is approximately ○ is excluded from the calculation area of the spatiotemporal gradient method. It turns out.

In other words, the above range corresponds to an area in the image where spatial gradient information is extremely poor, or an area where the spatial gradient information is reversed even if there is a lot of spatial gradient information, and in principle, this area corresponds to a region where the spatial gradient information is reversed. Since this is the area where the algorithm is least effective, the area can be automatically removed from the calculation area as a gap between each block.

The motion vector detection device according to the present invention is configured as described above, and next, a case will be described in which the motion vector detection device according to the present invention is applied as a blur correction device for a video camera.

FIG. 5 is a block diagram showing a third embodiment in which the motion vector detection device of the present invention is used in a blur correction device for a video camera.

In the figure, 101 is a photographic lens optical system, 102 is an image sensor such as a CCD that photoelectrically converts the subject image formed on the imaging surface by the photographic lens optical system 101 and outputs an image signal, and 103 is an image sensor from the image sensor 103. A preamplifier 104 amplifies the output image signal to a predetermined level; 105 is a preprocessing circuit that applies AGC to the input image signal to keep the level constant; and performs processing such as gamma correction;
106 is an A/D converter that converts the input analog image signal into a digital image signal, 106 is an image memory that stores one field of the digital image signal output from the A/D converter 105, and 107 is an image memory. This is a memory control circuit that controls the address and write rate when reading an image into the image memory 106, and controls the read address and read rate of the image when reading the image from the image memory 106, and is operated by the system control circuit 109, which will be described later. controlled.

Reference numeral 108 denotes a motion vector detection circuit that detects a motion vector of an image from an image signal, and its internal configuration and operation are the same as those shown in FIGS. 1 and 3 above, but processing is performed using digital signals. Needless to say.

109 is a water bag! The system control circuit is configured by a microcomputer and calculates blur correction information from the motion vector information calculated in the motion vector detection circuit 108, and controls the memory control circuit 107 based on the calculation result. When reading an image from the memory 106, control is performed to offset the blurring by shifting the readout position, that is, the readout address, on the memory in the direction of the blurring.

110 is a camera signal processing circuit that performs predetermined signal processing on the image signal read from the memory 106 and converts it into a standardized video signal; 111 is controlled by a system control circuit; This is a view angle correction circuit that corrects the view angle of the read image. That is, since blur correction is performed by shifting the read position of the image in the memory 106, the angle of view of the read image is smaller by the shiftable range on the memory compared to the image read into the memory. Therefore, the view angle correction circuit 111 performs processing such as enlarging the view angle and interpolating the image so that the image read out from the memory 106 has the same size as the original view angle.

The field angle corrected signal is converted into an analog image signal by the D/A converter 112, and is supplied to a monitor such as a video recorder or an electronic viewfinder (not shown).

With the above configuration, a motion vector is calculated from the image signal output from the image sensor using the configuration shown in FIGS. , blur correction can be performed by shifting the read address of the memory.

Further, FIG. 6 is a block diagram showing another example of a video camera equipped with a blur correction device using the motion vector detection circuit of the present application. Note that the same reference numerals are used for the same components as in FIG. 5, and the explanation thereof will be omitted.

In the figure, numeral 201 is a variable apex angle prism that corrects blur by changing the apex angle of the photographing lens optical system, that is, the direction of the optical axis. The structure is, for example, a silicon-based liquid sealed between two parallel glass plates.
It is constructed so that the apex angle can be varied by changing the angle formed by the sheets of glass.

202 is a camera signal processing circuit that converts the image signal output from the preamplifier into a standardized video signal and outputs it;
203 detects the direction and amount of blur from the motion vector information supplied from the motion vector detection circuit 108;
This is a system control circuit made up of a microcomputer that calculates the drive amount of the variable apex angle prism to correct this. The correction information calculated by the system control circuit 203 is supplied to the drive circuit 204, and the actuator 20 for driving the variable apex angle prism 201
5 is driven in the direction of correcting the blur by an amount that cancels out the amount of blur.

In this way, the amount of blur is calculated by detecting a motion vector in the image signal, and the variable vertex angle prism is driven in a direction to cancel the amount of blur, thereby making it possible to perform blur correction.

In any of these embodiments, the motion vector calculation area is adaptively and automatically set according to the spatial frequency of the input image, so large error vectors can be removed even in areas with patterns with few features. High spatial resolution can be obtained by effectively utilizing the spatial gradient information of an image, so regions where the spatial gradient has a reversed sign, which the spatio-temporal gradient method is weak at, can be used as gaps between detection blocks from the calculation area. By removing this, detection accuracy can be improved, and blur correction can be performed with high precision and reliable operation.

(Effects of the Invention) As described above, according to the motion vector detection device 8 of the present invention, the motion vector calculation area is adaptively and automatically set according to the spatial frequency of the input image. Large error vectors in areas of patterns with few features can be reliably removed, and high spatial resolution can be obtained by effectively utilizing the spatial gradient information of the image.

Furthermore, since the boundaries of blocks are determined using a simple evaluation function, blocks can be determined quickly.

In addition, regions where the spatial gradient has a sign reversal, which the conventional spatio-temporal gradient method is not good at, can be easily excluded from the calculation region as gaps between blocks, and detection accuracy can be improved.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the configuration of a first embodiment of a motion vector detection device according to the present invention, FIG. 2 is a diagram illustrating a method for determining a block boundary using the spatial gradient of an image, and FIG. is a block diagram showing the configuration of a second embodiment of the motion vector detection device according to the present invention, FIG. 4 is a diagram for explaining a method for determining a block boundary using the temporal gradient of an image, and FIG. The third example shows the case where the motion vector detection device according to the present invention is applied to a blur correction device for a video camera.
FIG. 6 is a block diagram of a fourth embodiment in which the motion vector detection device of the present invention is applied to a blur correction device for a video camera. 9(z) 8(t) Release + figure

Claims (3)

[Claims] (1) Detection means for detecting a motion vector of an image from the relationship between the amount of change in the image density value corresponding to an arbitrary position of the image over a predetermined time and the spatial gradient of the image signal corresponding to the arbitrary position, and the motion vector 1. A motion vector detection device comprising: a setting means for adaptively setting the size and shape of a unit operation area for outputting the motion vector according to the spatial frequency of an input image. (2) In claim (1), the motion vector detection is characterized in that the setting means is configured to determine a boundary of the unit calculation area using a spatial gradient of the image. Device. (3) In claim (1), the setting means is configured to determine the boundary of the unit calculation area using a density difference between temporally consecutive images. Features a motion vector detection device.