JP3126998B2 - Motion vector detection device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion vector detecting device, and more particularly, to a device for detecting a motion vector from an image signal.

[0002]

2. Description of the Related Art As a motion vector detecting method by image signal processing, a cross-correlation method for calculating a cross-correlation coefficient h (ξ, η) according to a defining equation or a sum of absolute values of a difference between two images (hereinafter referred to as a residual value). There is a matching method for calculating the difference.

The formula for calculating the cross-correlation coefficient is represented by h (ｈ, η) = ∬g0 (x- （, y-η) · g1 (x, y) dxdy. Where g0 (x, y) · g1 (x, y
y) represents two images, and ξ and η represent the amount of deviation between the two images. The cross-correlation coefficient h (ξ, η) is
The maximum value is obtained when the two images completely match, and the value decreases exponentially when a deviation occurs between the two images.

In the matching method, the remaining amount e is calculated according to the following equation.
Calculate (ξ, η).

E (ξ, η) = ΣB | g 0 (x−ξ, y−η) · g 1 (x, y) | Here, the symbol B indicating the accumulation range is represented by a block and a spare, and two in a block unit. In order to calculate the degree of similarity and the degree of matching between images, the method is also called a block matching method or a template matching method. The degree of mismatch between the two images and the definition of the residual indicating the degree of mismatch may be defined not only as the sum of the absolute values of the differences between the pixels described above, but also as the sum of squares of the differences. The size and shape of the block are fixed irrespective of the input image.

[0006]

However, according to the above-mentioned method, there is a drawback that it is difficult to apply the method to large motions, for example, because a high frequency component of the input image causes mismatching.

As a countermeasure, there is a method in which a low-pass filter having a sufficiently large mask is used, and the image is divided by a large block including a characteristic pattern in order to expand the detection range. Increasing the block reduces the resolution.

Accordingly, an object of the present invention is to provide a motion vector detecting device capable of obtaining a wide detection range and high resolution.

[0009]

SUMMARY OF THE INVENTION The present invention has been made for the purpose of solving the above-mentioned problem, and is characterized by using a plurality of image signals passing through a first pre-processing filter means. The motion vector is calculated based on the correlation between the images, and the area is determined by an area determining unit that determines an image area in which a desired motion vector is to be obtained, and the area determining unit that has passed through a second preprocessing filter unit. And a motion vector calculating means for calculating a motion vector for the image area based on an image signal corresponding to the image area.

[0010]

In this way, when determining the motion vector detection area, the calculation is performed with priority given to the spatial resolution over the detection range, so that the motion vector detection area can be set with high accuracy, and the final motion can be set. The screen area for obtaining the vector can be determined accurately.

Further, for example, in the case of a screen shake, a motion vector having a uniform size in a uniform direction is obtained. Therefore, the motion vector calculating means performs the calculation by giving priority to the detection range over the spatial resolution. Thereby, the calculation accuracy can be improved.

As described above, since the detection range and the spatial resolution can be individually selected, it is possible to achieve both a wide detection range and a high resolution.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of a motion vector detecting device according to the present invention will be described with reference to the drawings.

FIG. 1 shows the configuration of an embodiment in which the present invention is applied to a device for preventing a screen shake of an image signal, that is, a camera shake in an image pickup device such as a video camera or an electronic camera. FIG.

Reference numeral 10 denotes an input terminal of an image signal, 12 denotes a low-pass filter (LPF), 14 denotes a motion vector calculation circuit for obtaining a motion vector by correlation calculation or matching calculation, and 16 denotes an area of the input image to be image-stabilized (hereinafter, referred to as "hereafter")
Image stabilization area determination circuit for determining the
Reference numeral 8 denotes a bandpass filter (BPF), reference numeral 20 denotes a motion vector calculation circuit for obtaining a motion vector by correlation calculation or matching calculation, and reference numeral 22 denotes an output terminal of a signal indicating the amount of motion of an image.

The image signal input to the input terminal 10 is divided into two systems, a circuit for determining the image stabilizing area and a circuit for detecting the amount of screen shake caused by the shake of the screen due to hand shake or the like. In order to determine the image stabilization area, it is necessary to divide the image into many blocks and to detect partial fine movements in the image. Of a frequency characteristic that does not cause loss of The motion vector calculation circuit 14 calculates a motion vector for each block from the output of the LPF 12 by a correlation calculation or a matching calculation. Arithmetic circuit 14
The size and shape of the block at are fixed. Then, the anti-shake area determination circuit 16 determines the anti-shake area from the output of the arithmetic circuit 14.

Referring to FIG. 2, the procedure for determining the image stabilizing area will be described. From the output image of the LPF 12 (FIG. 2A), the motion vector calculation circuit 14 outputs a motion vector of each block obtained by equally dividing the input image horizontally and vertically. The motion vectors on this screen are distributed, for example, as shown in FIG. Since the motion vector causes mismatching due to insufficient detection range, and its direction and size vary, the anti-shake area determination circuit 16 performs a statistical process on the motion vector information to prevent the motion vector information as shown in FIG. Determine the shake area.

On the other hand, an input image signal is input to the BPF 18 in order to detect a screen shake due to a hand shake or the like. B
The operation of the PF 18 will be described in detail with reference to FIGS. The motion vector calculation circuit 20 calculates and outputs a screen shake amount due to hand shake or the like from the output of the BPF 18 by the correlation calculation or the matching method for the image signal in the image stabilization area determined by the image stabilization area determination circuit 16. The size and shape of the block in the motion vector calculation circuit 20 at this time depend on the pattern period T of the output signal of the BPF 18.

In the cross-correlation method and the matching method, essentially,
Since it is necessary to include a characteristic pattern in the block, a block having a sufficiently large size corresponding to the period T is selected. When such a large block is selected, the spatial resolution is significantly reduced. Since the motion vector generated in the area is not due to the motion of the subject but due to a camera shake, the motion vector has a uniform size in a uniform direction, and the spatial resolution does not decrease.

Next, the operation of the BPF 18 will be described in detail with reference to FIGS. The input image can be decomposed into each frequency component constituting the input image by Fourier transform. One-dimensional Fourier transform is performed on a waveform indicating a density change in a certain cross section of the input image, and only a component of a frequency 1 / T is extracted to obtain 1
When the dimensional inverse Fourier transform is performed, a period T as shown in FIG.
Can be obtained. The vertical axis y indicates the image density value, the horizontal axis x indicates the position in the cross section, A indicates the amplitude of the sine wave, and B
Indicates a sine wave bias.

When the matching method is applied to this pattern, the following is obtained. Here, y1 represents the image before the movement, and y2 represents the image moved by α in the x direction.

Here, assuming that the residual is e (α), e (α) = ΣB | y1−y2 | = ΣB | Asinωt−Asinω (t−α) | = (4AT / π) · | sin πα / T | (∵ω = 2π / T). However, the size of the block was made equal to the period T of the image pattern.

FIG. 4 shows the displacement amount α of the image in pixel units on the horizontal axis and the residual error e (α) on the vertical axis, and the following can be understood from FIG.

1). The cycle of e (α) is equal to the cycle T of the image pattern.

2). The detection range is ± T / 2, beyond which mismatching occurs.

From these, the period T of the image pattern
Thus, it can be understood that the motion detection range can be freely selected.

Therefore, in this embodiment, as shown in FIG.
A pre-processing filter for determining a vibration-proof area, that is, L
In addition to the PF 12, a pre-processing filter for calculating a final motion vector, that is, a BPF 18, is provided to achieve both high resolution and a wide detection range.

In this embodiment, the detection range is expanded by the BPF 18, but the same effect can be obtained by using a low-pass filter that cuts a pattern with a small cycle instead of the BPF 18. .

In the embodiment shown in FIG. 1, two pre-processing filters 12, 18 are arranged in parallel, but they may be arranged directly.

FIG. 5 is a block diagram of the circuit configuration in the modified embodiment. 60 is an image signal input terminal;
Is a LPF, 64 is a motion vector calculation circuit by a cross-correlation method or a matching method, 66 is a vibration-proof area determination circuit,
Reference numeral 68 denotes a delay circuit for adjusting the calculation time by the motion vector calculation circuit 64 and the anti-shake area determination circuit 66, reference numeral 70 denotes an analog switch whose opening and closing is controlled by the output of the anti-shake area determination circuit 66, reference numeral 72 denotes a BPF, and reference numeral 74 denotes A motion vector calculation circuit 76 based on a cross-correlation method or a matching method is an output terminal for a signal indicating a final motion vector amount.

The LPF 62, the motion vector calculation circuit 64, and the image stabilization area determination circuit 66 correspond to the LPF of FIG.
The operation is the same as that of the portion including the F12, the motion vector calculation circuit 14, and the anti-shake area determination circuit 16. That is, the motion vector calculation circuit 64 calculates the distribution of the motion vector for a block of a certain size, and the image stabilization area determination circuit 66 detects the image stabilization area having the same size and direction of the motion vector. The analog switch 70 is closed only inside the vibration-proof area. As a result, the image signal in the image stabilization area is input to the BPF 72.

The BPF 72 performs a filtering process for expanding the detection range. After that, the motion vector calculation circuit 74 performs a motion analysis on a block having a size and a shape corresponding to the pass band of the BPF 72 based on a cross-correlation method or a matching method. Find a vector. It goes without saying that a low-pass filter may be used instead of the BPF 72. In this way, a signal indicating the amount of motion of the target image is obtained at the output terminal 76.

Next, a case where the above-described motion vector detecting device is actually applied to a vibration isolator of a video camera will be described with reference to FIGS. 6 and 7.

FIG. 6 shows an example in which a variable vertex angle prism for optically correcting the shake by changing the optical axis of the taking lens is used as the shake correcting means.

Referring to FIG. 1, reference numeral 101 denotes a variable apex angle prism for changing the direction of the optical axis of the photographing lens optical system, that is, the apex angle. For example, a silicon-based liquid is filled between two parallel glass plates. Reference numeral 102 denotes a photographing lens; 103, an image pickup device such as a CCD that photoelectrically converts a subject image formed by the photographing lens 102 and outputs an image pickup signal;
Reference numeral 4 denotes a preamplifier; 105, a camera signal processing circuit that performs various processes such as blanking processing, addition of a synchronization signal, and gamma correction on an image signal output from the image sensor to output a standardized video signal; 106 is the aforementioned FIG.
A motion vector detecting circuit 107 including the motion vector detecting circuit described in each embodiment of FIG. 5 fetches the motion vector information of the image supplied from the motion vector detecting circuit 106 and is a variable for canceling the image motion due to camera shake. A system control circuit that calculates information on the driving direction of the apex angle prism and a drive amount necessary for shake correction, and a drive circuit 108 that drives the variable apex angle prism 101 based on the information calculated by the system control circuit 107 .

Accordingly, the motion vector based on the image blur (camera blur) is detected by the motion vector detecting circuit 106 for detecting the motion vector in each of the above-described embodiments, and the motion vector is variable based on the motion vector. The driving direction and the driving amount of the apex angle prism are calculated, and the calculated apex drives the variable apex angle prism to perform blur correction. The operation of the motion vector detection circuit itself is the same as that described in the above embodiment, and the description is omitted.

FIG. 7 shows that the optical system is not used for the shake correcting means,
An image is temporarily fetched into a memory and the movement of the image is corrected by changing the range of reading from the memory.

The same components as those in the embodiment of FIG. 6 are denoted by the same reference numerals, and the description thereof is omitted.

The image signal output from the preamplifier 104 is converted into a digital signal by the A / D converter 109 and is taken into the memory in the digital signal processing circuit 110.
At this time, the rate and timing of A / D conversion for taking in an image to the memory, and further, the writing timing and address of the memory are controlled by the memory control circuit 11.
3. The control of the address and timing of reading from the memory is also controlled by this memory control circuit.

The digital image signal read from the memory 110 is subjected to various types of camera signal processing by a camera signal processing circuit 111, converted into an analog signal by a D / A conversion circuit 112, and output as an image signal. The digital video signal may be output as it is.

On the other hand, in the motion vector detecting circuit 115,
As in the embodiment of FIG. 1, a motion vector due to camera shake is detected and supplied to a system control circuit 114. The system control circuit 114 generates an image based on the motion vector detected by the motion vector detection circuit 115. Then, the direction and the amount of movement are calculated, and based on this, the memory control circuit 113 is controlled to control the read range of the memory. That is, an image which is output in advance to the memory but is wider than the nucleus is taken in by the nucleus, and when the memory is read out, the reading range is varied to correct the movement. By shifting the readout range in the direction of the movement, the movement of the image can be corrected.

In the above configuration, the camera signal processing circuit may be disposed after the D / A converter 112 to perform analog signal processing. However, performing digital signal processing makes processing easier, and It is also advantageous in terms of noise.

As described above, the motion vector detecting circuit according to the present invention can perform blur correction in a video camera. In addition, the motion detection is not limited to shake correction, and many other applications such as camera panning detection are possible.

Thus, a motion detection range is extremely wide, and detection and correction can be performed for various sizes of motion, and a high-performance video camera with a shake correction function can be realized.

[0045]

As described above, according to the motion vector detection device of the present invention, when determining the motion vector detection area, the calculation is performed with priority given to the spatial resolution over the detection range. The detection area for detecting the motion vector can be set with high accuracy, and in the case where a motion vector of a uniform size is generated in a uniform direction, such as a screen shake, the detection range is set to be smaller than the spatial resolution. By performing the calculation with priority, it is possible to achieve both a wide detection range of the motion vector and a high spatial resolution.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of a motion vector detecting device according to the present invention.

FIG. 2 is an explanatory diagram of a procedure for determining an image stabilizing area.

FIG. 3 is a sectional view of an image pattern having a period T.

FIG. 4 is a diagram showing a distribution of residuals when a moving amount is estimated by applying a block matching method to an image pattern having a period T.

FIG. 5 is a block diagram showing a configuration according to another embodiment of the present invention.

FIG. 6 is a block diagram showing an embodiment of a video camera to which the motion vector detection circuit according to the present invention is applied.

FIG. 7 is a block diagram showing another embodiment of a video camera to which the motion vector detection circuit of the present invention is applied.

[Explanation of symbols]

 Reference Signs List 10 image signal input terminal 12 low-pass filter 14 motion vector calculation circuit 16 image stabilization area determination circuit 18 band-pass filter 20 motion vector calculation circuit 22 image signal output terminal

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) H04N 5/232 G06T 7/20

Claims (1)

(57) [Claims] An area determining unit that calculates a motion vector based on a correlation between a plurality of images using an image signal that has passed through a first preprocessing filter unit, and determines an image area in which a desired motion vector is to be obtained. And motion vector calculating means for calculating a motion vector for the image area based on an image signal corresponding to the image area determined by the area determining means passed through the second preprocessing filter means. Characteristic motion vector detection device.