JP2612092B2 - Screen shake correction device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for correcting a screen shake caused by a swing or the like of an imaging apparatus.

[Conventional technology]

4 to 6 are views for explaining the operation of the conventional screen shake correction apparatus. FIG. 4 shows an original image of a subject, and FIG. 5 shows a set position of a detection area for detecting the screen shake. FIG. 6 is a diagram showing the operation of screen shake correction.
FIG. 6A shows an image before correction, and FIG. 6B shows an image after correction.

 Next, the operation of the conventional screen shake correction device will be described.

A video camera that captures a moving image of a subject image, and a video movie to which a magnetic recording / reproducing device is added to this video camera, that is, a video camera that is hand-held without using a means for fixing the imaging device such as a tripod or the like In many cases, images are often taken from a moving object such as a car or a train because of its portability.

In photographing in such an environment, the fluctuation of the apparatus is always accompanied by the fluctuation of the apparatus, and the fluctuation of the apparatus is immediately manifested as the fluctuation of the screen.

As an apparatus for correcting such a so-called screen shake, for example, a technical report of the Institute of Television Engineers of Japan vol.11, No.
3 (1987), pages 43 to 48, are known.

This device, when the image is distorted as shown in FIG.
As shown in FIG. 5, a range (detection area) where motion is desired to be stopped is set, and a size and a direction (motion vector) corresponding to the motion of the image in the detection area are obtained. The timing of the image signal to be read out is controlled in the direction opposite to the movement to produce an image as shown in FIG. 6 with less image fluctuation.

[Problems to be solved by the invention]

The above-described conventional motion vector detecting means performs image comparison between frames to obtain a motion vector.

However, image comparison generally requires a large amount of memory, so that it is not suitable for consumer use because it is not economically viable.

SUMMARY OF THE INVENTION The present invention has been made in order to solve such a problem, and has as its object to obtain an apparatus which detects a motion vector based on image information without using a large-capacity frame memory and corrects a screen shake. And

[Means for solving the problem]

The image stabilization apparatus according to the present invention includes: an imaging unit that performs photoelectric conversion of a subject image; a video signal processing unit that reads a signal corresponding to a vertical synchronization signal from the imaging unit and converts the signal into a video signal; Sample and hold means for sampling and holding a luminance signal corresponding to a horizontal synchronizing signal, taking in the luminance signal from the sample and hold means, and integrating the luminance signal level corresponding to the synchronizing signal for each detection area division Calculating means for calculating the luminance barycentric position between the first field and the second field, and calculating the difference between the luminance centroid positions between the first field and the second field for each detection area section to obtain a motion vector between the fields. The readout timing of the signal from the imaging means is controlled in accordance with the arithmetic means and the motion vector. Characterized by comprising a means.

[Operation] The motion vector detecting means according to the present invention divides the detection area into a plurality of parts in the X-axis direction and the Y-axis direction, and synchronizes the luminance centroid of the Y signal in the first field and the second field in each section. By calculating the level of the luminance signal corresponding to the signal, the difference between the positions of the luminance centroids of the respective sections between the fields is obtained. And the readout timing from the imaging means is controlled in accordance with the motion vector, so that highly accurate screen shake correction can be performed by relatively simple image processing.

(Example of the invention)

FIG. 1 is a block circuit diagram showing one embodiment of the present invention.

In the figure, (1) is a lens, (2) is an image sensor (CCD) that performs photoelectric conversion of a subject image, and (3) is a signal generation circuit, which outputs a signal from the CCD (2) according to ▲ (21) described later. A clock pulse to be taken out and horizontal and vertical synchronizing signals HD (9) and ▼ (13) are generated. (4) is a signal processing circuit which converts the output signal of the CCD (2) into a composite video signal (5). (6) is a monitor TV, and the input composite video signal (5) is projected on a visible image. (8) is a sample-and-hold circuit, which receives Y from the signal processing circuit (4).
The signal (7) and the horizontal synchronizing signal (HD) (9) are inputted from the synchronizing signal generating means (3). (11) is A / D converter, S / H
The S / H signal (10) is input from the circuit (8), and the output is input to the microcomputer (12). The microcomputer (12) receives an HD (9), a vertical synchronization signal (VD) (14), and a screen shake correction command (16). Here, VD (14) is obtained by reversing the theoretical level of the output (13) of the synchronizing signal generating means (3) by the inverter (15). The output VDX (17) of the microcomputer (12) is input to the flip-flop (18) and is triggered by the HD (9), and the inverted ▲ (19) is input to the changeover switch (20). You. ▲ ▼ (13) is input to the other input terminal of the changeover switch (20) from the synchronizing signal generation means (3), and the changeover switch (20) is used to perform a screen shift correction command (16).
Is switched and if this command (16) is input,
The changeover switch (20) selects ▼ (19), and if there is no command (16), selects ▼ (13). The output ▲ ▼ (21) of the changeover switch (20) is input to the synchronizing signal generating means (3) and used as a vertical synchronizing signal (V-SYNC) for driving the CCD (2).

FIG. 2 is a flowchart for explaining the operation of the first calculating means for detecting a motion vector in this embodiment, and FIG. 3 is for explaining the operation of the second calculating means in this embodiment. FIG.

 Next, the operation of this embodiment will be described.

First, an operation of the imaging device for visually displaying a subject image will be described.

The image of the subject, not shown, is captured by the CCD using the lens (1).
An image is formed on the imaging surface of (2). The image charges accumulated in the CCD (2) are read out to the video signal processing circuit (4) at the timing of the vertical synchronizing signal (V-SYNC), converted into a composite video signal (5), and monitored on the monitor TV (6). And is projected on the image.

 Next, the screen shake correction operation will be described.

Brightness signal (Y signal) from video signal processing circuit (4)
(7) is input to the S / H circuit (8), sampled and held (S / H) for each H, and input to the A / D converter (11). At this time, assuming that the screen shake correction command (16) has already been issued, the microcomputer (12)
The digital data output from the A / D converter (11) is fetched, and from this data, the luminance centroid position of each of m × n sections obtained by dividing the detection area into m in the X-axis direction and n in the Y-axis direction is obtained. , HD (9), the first calculation is performed by integrating the luminance levels, and then the difference between the luminance centroid positions of the same section in the first and second fields is calculated. A second operation for obtaining a motion vector between fields from the difference between the luminance centroid positions is performed. In the case where the screen is shaken by the first and second calculations, the screen movement amount is calculated by calculating a true inter-field motion vector from the difference between the luminance centroids of m × n sections between the fields. it can.

The charge of the CCD (2) is sent to the video signal processing circuit (4) at the timing of V-SYNC, and as described above, the symbol (21) is used as the V-SYNC. Further, under the condition that the screen shake correction command (16) is issued, ▲ ▼ (21) is equivalent to the timing of VDX (17). Here, if the timing of VDX (17) generation is controlled in accordance with the motion vector obtained in the second operation, that is, the amount and direction of the screen shift between fields, the timing of charge extraction from the CCD (2) can be varied. It becomes possible.

Next, the operation of the first calculation means for obtaining the luminance centroid of the m × n sections will be described with reference to the flowchart of FIG.

In this embodiment, for the sake of simplicity, the detection area is divided into two in the Y-axis direction and a motion vector in the Y-axis direction is detected. The detection area further includes the count number (HDCNT) of HD (9). Is divided into the upper half of 50-139 and the lower half of 140-229, and the number of HD counts in each section (HCNT)
Are 90 in each case.

In FIG. 2, the process first waits until the rising edge of the VD (14) (101). The next time VD (14) enters, (HDCNT) is cleared (102) and (HCNT)
Is set to 90, and (M1) and (M2) are cleared (103).
Here, wait for HD (9) (104), and when HD (9) comes (H
DCNT) is incremented (105).

Next, in (106), the value of (HCNT) is tested, and 50 ≦ (HDC
NT) <140, that is, if it is within the upper half of the screen, the flow goes to processing (108); otherwise, it flows to processing (107). At (107), the value of (HDCNT) is also tested, and if 140 ≦ (HDCNT) <230, that is, the lower half, the flow goes to (108); otherwise, the flow goes to (103). At (108), the output of the A / D converter (11) is fetched into the A register, and it is added to (M1) at (109), whereby the incoming signal level is integrated every time HD comes. Go.

Next, in (110), after multiplying the A register value by (HCNT) and adding it to (M2), the signal level × (HCNT) is integrated for each HD.

Next, in (111), (HCNT) is decremented, and its value is tested to perform the following operation. That is, (HCN
If (T) is 0, go to (112), otherwise go to (103). At this point in time through the processing of (108) to (110), the integrated value of the upper half or lower half of the detection area of the value of the S / H signal (10) sampled for each HD is stored in (M1). In M2), the integrated value of the upper half or lower half of the detection area of the value obtained by multiplying the value of the S / H signal (10) sampled for each HD by the count value of H is stored.

Next, (112) tests the (HDCNT) value. Here, it is determined whether or not the integration of the upper half of the detection area has been completed. If it is the upper half, the flow goes to (113), and the luminance centroid value (GIX) of the upper half in a certain field X is (M2) / (M1) The value of H is obtained by the operation described above, and then the flow goes to (103) to perform the lower half processing.

Similarly, in the lower half of the detection area, the flow goes to (114) to find the luminance centroid value (G2X), and then flows to (101) to wait for the head of the next field.

Next, a method of calculating a motion vector by the second calculating means of this embodiment will be described with reference to FIG. If the subject in a certain field, that is, the field 1, moves upward by ΔH lines on the screen in the next field 2, the S / H signals (10) of the fields 1 and 2 are respectively shown in FIG. ) And (b).

First, when the luminance center of gravity is obtained by the first calculating means, if the detection area is divided into one without dividing, the luminance centers of gravity of the screens of the fields 1 and 2 are represented by G1, G2 shown in FIG.
Is obtained as That is, the motion vector ▲
▼ points downward between fields 1-2 on the screen as shown in FIG. 3 (c). However, it is clear that this motion vector ▲ ▼ is not correct information,
This method of calculating the luminance centroid without dividing the detection area involves the possibility of malfunction.

Next, let us consider a case where the detection area is divided vertically into two parts in the Y-axis direction. As shown in FIG. 3 (d), the first arithmetic means determines the luminance centroids G11 and G21 in the field 1, In 2, G12 and G22 are required. When the motion vector between the fields 1-2 at this time is obtained, the motion vector is different in the direction and the size in the upper half of the detection area, and in the lower half, and in this case, the lower half of the detection area is obtained. The motion vector obtained by ▲
▼ indicates the correct direction and size.

However, in this embodiment, when the directions of the two motion vectors are different from each other, the second calculating means makes a judgment of "wrong", outputs a motion vector of 0 as a true motion vector, and detects a malfunction of the correction. I try to prevent it.

If a motion vector is obtained by dividing the screen into a larger number in this way, a motion vector with higher accuracy can be obtained.

In the above embodiment, the operation of detecting the motion vector in the Y-axis direction has been described. However, the detection of the motion vector in the X-axis direction is performed by dividing the detection area in the X-axis direction. If the screen position is corrected so that the X-axis direction motion vector and the Y-axis direction motion vector obtained in the above embodiment approach zero, the difference in the diagonal direction can be obtained. Screen shake can be corrected. In this case, the sampling pulse input to the S / H circuit (8) is set to be m times the pulse of HD (9), and the detection area in the microcomputer (12) is X-axis. If the count number of the sampling pulse is set so as to be divided into m in the direction and n in the Y-axis direction, motion vectors in the X-axis direction and the Y-axis direction can be detected in the same procedure as in the above embodiment.

Further, in the above-described embodiment, the first and second calculation means are performed by software using the microcomputer (12). However, the first and second calculation means may be configured by hardware capable of performing the calculation.

In the above embodiment, the shake correcting means is a CCD (2)
The control is performed by controlling the timing at which electric charges are extracted from the device. However, the correction may be performed by mechanically moving a unit to which the lens (1) and the CCD (2) are attached.

Further, in the above embodiment, the second arithmetic means outputs the 0 vector as a true motion vector if the directions of the two motion vectors obtained from the first and second fields are opposite. It is not limited to the configuration,
For example, the average of the sum of m × n motion vectors obtained by dividing the detection area by m × n may be output as a true motion vector.

〔The invention's effect〕

As described above, according to the present invention, the detection area is divided into a plurality of sections in the X-axis direction and the Y-axis direction, and the luminance centroid of each of the divided sections is integrated with the luminance signal level corresponding to the synchronization signal. The motion vector in the X-axis direction and the Y-axis direction is obtained from the difference between the luminance centroids of the respective sections between the first and second fields, and the position of the screen is corrected so that the motion vector becomes zero. In order to control the timing of the signal read from the imaging means to the video signal processing means, the motion vector can be detected without using a large amount of image memory, etc. There is an effect that only a slight increase is required and a highly accurate screen shake correction device can be obtained.

[Brief description of the drawings]

FIG. 1 is a block circuit diagram showing a configuration of an imaging device including a motion vector correction device according to an embodiment of the present invention.
FIG. 3 is a flow chart of the first calculating means of this embodiment, FIG. 3 is a waveform diagram for explaining the operation of the second calculating means of this embodiment, and FIGS. FIG. 10 is a diagram for explaining an operation of a conventional motion vector detection device. (2) ... image sensor (CCD), (3) ... synchronization signal generator circuit, (4) ... video signal processing circuit, (8) ... sample and hold circuit (S / H), (11) A / D converter, (12) microcomputer, (15) inverter, (18) flip-flop, (20) changeover switch. In the drawings, the same reference numerals indicate the same or corresponding parts.

Claims (1)

(57) [Claims] A detection area is set in a part of a screen, and the detection area is divided into a plurality of sections in the X-axis direction and the Y-axis direction to form a large number of detection area sections. The position of the luminance center of gravity is obtained, and the movement of the position of the luminance center of gravity in the detection area is detected based on the movement to correct the screen shake. A video signal processing means for reading a signal corresponding to a vertical synchronization signal from the imaging means and converting the signal into a video signal; a sample and hold means for sampling and holding a luminance signal from the video signal processing means in accordance with a horizontal synchronization signal; The first signal is obtained by taking in the luminance signal from the sample and hold means and integrating the luminance signal level corresponding to the synchronization signal for each of the detection area sections. A first calculating means for calculating a luminance centroid position between the first field and the second field, and a second computing means for obtaining a difference between the luminance centroid positions of the first field and the second field for each of the detection area sections to obtain a motion vector between the fields. A screen shake correction apparatus, comprising: a calculation unit; and a unit that controls timing of reading a signal from the imaging unit in accordance with the motion vector.