JP2662103B2 - Motion compensator - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion compensator for detecting the direction and magnitude of a screen shake caused by a swing of an image pickup apparatus such as a video movie and correcting the screen shake. It is.

[0002]

2. Description of the Related Art FIGS. 7 to 9 are diagrams for explaining the operation of a conventional motion compensating device. FIG. 7 is a diagram for explaining an original image of a subject, and FIG. 9A and 9B are views for explaining the state of the operation of the screen shake correction. FIG. 9A shows an image before correction, and FIG. 9B shows an image after correction.

A video camera capable of capturing a moving image of a subject image to obtain a visible image on a display monitor, or a video camera in which a magnetic recording / reproducing device is added to a video camera, has been developed by electronic technology. Thus, miniaturization, weight reduction, and mass production are possible. Recent advances in electronic technology have weighed less than 1 kg and
It has become possible to provide video movies of less than 6 sizes, and the price has become affordable enough to meet demand for consumer use, and the image quality has reached a level close to that of commercial equipment. I have.

[0004] By the way, as an environment for using such a video movie, there are many cases of hand-held shooting that does not use means for fixing an image pickup device such as a tripod, and because of its good portability, it may be used outdoors or in some cases. There are also many shootings on moving objects such as automobiles and trains. When shooting in such an environment, shake of the device due to camera shake is always a problem,
Since this is manifested by shaking of the screen, the viewer of the reproduced image often feels uncomfortable.

As an apparatus provided with a means for correcting such a so-called screen shake, for example, a motion correction apparatus described in the Technical Report of the Institute of Television Engineers of Japan, Vol. 111, No. 3 (1987), pp. 43-48. It has been known. As shown in FIG. 7, this motion compensating device uses a monitor TV.
Assuming a scene where the image on the screen of FIG. 6 is shaking, as shown in FIG. 8, a range 6A (hereinafter, referred to as a detection area) in which the movement is to be stopped is set at an arbitrary position and The size is set, and a motion amount and a direction Vr (hereinafter, referred to as a motion vector) in the detection area 6A are obtained.
The motion vector Vr controls the frame memory in the direction opposite to the motion to create an image with little screen fluctuation as shown in FIG. 9B from the state before correction in FIG. 9A.

[0006]

The conventional motion compensator is constructed as described above, and compares images between frames in order to obtain a motion vector. This image comparison requires a large amount of memory and a large amount of memory. Large-scale IC that suppresses
However, there is a problem that it is expensive and is not suitable for consumer use.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and has an inexpensive configuration without using a frame memory or a large-scale IC for controlling the frame memory.
It is an object of the present invention to provide a motion compensator capable of detecting a motion vector and accurately performing a predetermined motion compensation.

[0008]

SUMMARY OF THE INVENTION A motion compensating apparatus according to the present invention is capable of detecting a shake set on a screen from video signal information.
Detects the center of gravity of brightness in the exit area for each field
Center of gravity detecting means and the center of gravity
The difference in the center of gravity of the brightness of each field
Motion vector detecting means for detecting the
Based on the motion vector detected by the vector detection means
Motion correction means to correct the position of the screen
The brightness peak position within the specified motion detection area
A brightness peak position detecting means for detecting for each field;
Brightness peak from the brightness peak position for each field
Moving vector detecting means for detecting a moving vector of a position
And the brightness within the motion detection area set on the screen.
Feature extraction method for determining the cloth state based on the movement vector
By the step and the feature extracting means, the movement vector is
If it is determined that the value is larger than the predetermined value,
The brightness information is inverted, and the brightness
The position of the center is obtained, and the motion vector is calculated from the position of the center of gravity of the brightness.
Of the screen by the motion vector detected by the
After performing position correction, the movement vector is smaller than a predetermined value.
Is determined by the motion vector detecting means.
Screen position correction using the detected motion vector
And control means for performing such control .

[0009]

According to the present invention, the position of the center of gravity of each field in the motion detection area set on the screen is detected from the image signal, and the motion vector is determined from the change of the position of the center of gravity of the brightness of each field. Is detected. Then, a control operation for displacing the position of the screen via the motion correcting means is performed so that the detected motion vector becomes zero, and based on the image signal in the motion detection area set on the screen. To extract the features of the image brightness distribution,
By controlling the motion vector detecting means or the motion correcting means itself, the motion correction for eliminating the screen shake is performed as predetermined.

[0010]

An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a video movie including a motion compensation device according to an embodiment of the present invention. In FIG. 1, reference numeral 1 denotes a lens for forming a subject image;
Reference numeral 2 denotes an image sensor (hereinafter, CC) that performs photoelectric conversion of a subject image.
Reference numeral 3 denotes a clock pulse for extracting an output from the CCD 2, a horizontal synchronizing signal (hereinafter referred to as HD) 9, and a synchronizing signal generating circuit for generating a vertical synchronizing signal (hereinafter referred to as VD) 13. The signal processing circuit 6 for converting the signal of the CCD 2 into a composite video signal 5 is a monitor TV on which the composite video signal 5 is projected.

Reference numeral 8 denotes a sample / hold circuit (hereinafter referred to as S / H).
A luminance signal (hereinafter, referred to as a Y signal) 7 extracted from the signal processing circuit 4;
HD 9 is input from the synchronization signal generation circuit 3.
The S / H signal 10 output from the S / H circuit 8 is A / D
The signal is input to the converter 11, and the output is input to the microcomputer 12. This microcomputer 1
2, HD9 and VD14 are input.
Here, the VD 14 is the same as the VD 13
Is a signal whose logic level has been inverted. The output signal VDX17 of the microcomputer 12 is input to the flip-flop 18, where it is triggered by the HD9 and becomes a signal VDX19 whose logic level is inverted and input to the changeover switch 20. This is the other input terminal of the changeover switch 20 and the VD13 is input, is further input as signal switching correction command 16 shaking screen, on the other hand, is VDX19 when this screen shaking correction command 16 is input also this VD13 when this screen shaking correction command 16 is not input
Output as VDRV21 . And this V DRV2
1 is a vertical synchronizing signal (V-S
YNC).

In the above configuration, the S / H circuit 8, the A / D
The converter 11 and the microcomputer 12 constitute a motion vector detecting means 30, and the synchronizing signal generating circuit 3, the microcomputer 12, the flip-flop 1
8, the changeover switch 20 and the screen shake correction command 16 constitute a motion correction means 40.

FIG. 2 is a diagram for explaining a situation in which the screen shake is corrected by a motion vector in the above embodiment, FIG. 3 is a signal waveform diagram for explaining the operation of the motion vector detecting means 30, and FIG. FIG. 5 is a diagram for explaining the operation of motion compensation at the time of backlight, and FIGS. 5 and 6 are flowcharts each showing a calculation procedure for detecting a motion vector in the microcomputer 12.

Next, the operation of the motion vector detecting means 30 in the above embodiment will be described. In the configuration of FIG. 1, a subject image (not shown) is
An image is formed on D2. The charges stored in the CCD 2 are sent to a signal processing circuit 4 at the timing of a vertical synchronizing signal (V-SYNC), and a composite video signal 5 output from the signal processing circuit 4 is input to a monitor TV 6 to display an image. It is.

Next, the operation of the screen shake correction will be described. The Y signal 7 output from the signal processing circuit 4 is input to the S / H circuit 8, sampled and held every horizontal scanning period, and input to the A / D converter 11. At this time, if the screen shake correction command 16 is given to the microcomputer 12, the microcomputer 12
The digital data output from the D converter 11 is fetched, and from this digital data, the V-SYNC, that is, the barycentric position of the brightness for each field, is obtained as the count value of the horizontal scanning line by the first arithmetic means.

Next, the microcomputer 12 obtains a difference in the center of gravity of the brightness between the fields from the center of the brightness of each field by the second calculating means. The motion vector is obtained as the count value of the horizontal scanning line by the second calculating means. The charge of CCD2 is V-SY
The signal is sent to the signal processing circuit 4 at the timing of NC, and the VDRV 21 is used as the V-SYNC.
21 is equivalent to the timing of VDX17. Here, if the generation timing of the VDX 17 is controlled in accordance with the motion vector obtained by the above-described second calculation means, that is, the amount and direction of the screen shake between the fields, C
The timing of extracting the charge from the CD 2 can be changed, and the screen shake can be corrected.

Hereinafter, the control method of the VDX 17 will be described.
This will be described with reference to FIG. In the following description, V
What is DX17 and VDX19 whose logic level is inverted?
Since they are equivalent in timing, the explanation is made using VDX19.
Advance Ming. On the screen of the monitor TV6, suppose that 6a
The subject with high brightness is projected
Then, the brightness of the luminance signal in the field at that time,
That is, the distribution of the value of the S / H signal 10 is 10a in FIG.
It is represented as By the way, VDX19 of this time
Assume that the timing is the same as VD13. This and
In this case, the S / H signal 10
The center of gravity of the brightness in the field is determined as G0.
The screen shake causes a high brightness in the next field.
If the holding part moves to 6b in FIG.
That is, assuming that the high-luminance portion has moved to the top of the screen,
The 10a portion of the H signal 10 moves to 10b,
The heart is determined as G1, and the motion is detected by the second arithmetic means.
The vector ΔH is obtained. That is, the motion vector ΔH is
formula It is represented by ΔH = G1-G0 ...  Here, the unit of H is the number of horizontal scanning lines.

Next, in the microcomputer 12, the VDX 19 of the current field which now matches the VD 13
As shown in FIG. 2, when the signal is generated by delaying by ΔH, the signal is applied to the synchronization signal generating circuit 3 as the VDRV 21 via the changeover switch 20. Since the electric charge of the CCD 2 is taken out at the timing of VDRV21, that is, VDX19, a high-brightness portion can be seen again at the position 6a on the screen, and the screen shake is corrected. So far, the correction operation in the case where the high-luminance portion has moved upward on the screen has been described.
That is, it is only necessary to perform correction so that 19 is advanced with respect to VD13.

In summary, when ΔH> 0, VD
X19 is delayed from VD13 by ΔH, and ΔH <0
In this case, VDX19 is advanced by ΔH with respect to VD13. In this manner, the screen shake is corrected.

Next, the operation of the first calculating means in the microcomputer 12 for obtaining the position of the center of gravity of the brightness of the field will be described with reference to FIG. In addition,
Here, the A / D converter 11 has 8 bits. Also, the first
In this embodiment, as shown in FIG. 3, the area of integration (fluctuation detection area) by the calculating means is the horizontal scanning period from the 52Hth to the 230Hth.

Referring to FIG. 5, the microcomputer 12
Has received the VD 14 (step 101), immediately after that, initializes the H count, stores the minimum value 0 of the 8-bit A / D conversion in the memory 0, 2 is cleared (step 102). In this case, the count value (H count value) of the horizontal scanning line is 230-52 = 178. Next, V
It waits until the 52Hth of HD9 comes from D14 (step 103). When the 52H comes, the A / D conversion data of the S / H signal 10 is read through the step 104 which is the processing of the HD 9 (step 105). The following steps 106 to 108 are processes for obtaining the maximum level of the S / H signal 10 in the integration area as the H count value. The memory 0 finally stores the maximum point of the S / H signal 10 in one field. Is stored as the H count value.

Thereafter, the read A / D conversion data is added to the memory 1 (step 109), and a value obtained by multiplying the data by the count value of H is added to the memory 2 (step 110). Then, reduce the H count value by 1,
It is determined whether the value is 0 (step 111). If the value is not 0, the process returns to step 104, and the process from step 105 to step 111 is repeated each time HD9 is detected. If it is determined to be 0 in step 111, a process of calculating the position of the center of gravity is performed (step 112). After that, the motion vector vg is obtained by the second calculating means (step 113), and the motion is corrected based on the motion vector vg (step 114), and all the motion correction processes in one field are completed.

Next, the details of the processing step 113 for obtaining the motion vector vg by the second arithmetic means of the microcomputer 12 will be described with reference to FIGS. 4 and 6. First, in FIG. 6, a test is performed to determine whether or not the field is immediately after the polarity of the A / D converted S / H signal 10 is inverted (step 2).
01). As a result of the test, if the field is not the field immediately after the inversion of the fetched polarity of the S / H signal 10, that is, if two or more fields have passed since the inversion, the brightness in the current field obtained by the above-described first arithmetic means is calculated. The motion vector vg is obtained from the difference between the center of gravity of the image and the center of gravity of the brightness in the previous field (step 202). Further, the difference vp between the positions of the brightness peaks between the fields
(Step 203), and a test based on the difference is performed (step 204). Here, when the difference vp in the position of the peak of the brightness, that is, the amount of movement of the peak position exceeds a predetermined value α (see FIG. 4), a signal in which the periphery is bright and the center is dark, specifically, in the case of backlight. Recognizing that it is a subject image,
The motion vector vg is cleared (step 205).
The D / S converted S / H signal 10 is supplied to the microcomputer 1
Then, the polarity of the data to be taken into the device 2 is inverted (step 206). If the amount of movement of the peak position does not exceed the predetermined value α in the test at step 204, the process ends without performing any operation.

In other words, as shown in FIG. 4, when the screen is bright and the center is dark, such as when the subject is backlit, the true motion vector vr that has not been detected and the center of gravity of the brightness are determined. Motion vector vg, which is the difference between fields
The directions are opposite to each other, and if the correction is performed using the motion vector vg, a malfunction will occur, but this cannot be known as it is. Now, when the difference vp between the fields of the position of the brightness peak is introduced, in the above case, the difference vp between the fields of the position of the brightness peak is obtained.
It can be seen that fluctuates greatly from the upper end to the lower end or vice versa in the shake detection area. Therefore, by testing whether or not the difference vp between the fields of the position of the peak of the brightness exceeds the predetermined value α, it becomes possible to test whether or not the motion vector vg is correctly obtained. By the way,
The predetermined value of the difference vp is desirably about half of the H count value of the shake detection area, and is set to 90H in the present embodiment, but is not limited thereto.

If it is determined in the processing step 201 that the current field is the field immediately after the fetch polarity has been inverted, the processing is skipped from step 202 to step 206 and the processing is terminated without doing anything.

A motion correction process (step 114) as shown in FIG. 5 is performed using the motion vector vg obtained by the above process. The process shown in FIG. 6 for obtaining the motion vector vg as described above is performed. As is apparent, immediately before and immediately after inverting the polarity of the A / D-converted S / H signal 10, the malfunction is prevented by forcibly setting the motion vector vg itself to the zero vector. The reason is as follows. That is, S /
Immediately before the polarity of the capture of the H signal 10 is inverted, the direction of the true motion vector vr that has not been detected and the direction of the motion vector vg obtained from the difference vp between the fields of the position of the brightness peak are opposite. If the correction is performed as it is, a malfunction occurs. In addition, in the field immediately after the polarity of the capture of the S / H signal 10 is reversed, if no restriction is applied, the position between the fields immediately before and after the polarity of the capture of the S / H signal 10 is reversed. This is because a motion vector vg is obtained from the difference, and a malfunction at the time of correction also occurs.

The reason for inverting the polarity of A / D conversion is as follows. That is, in the example as shown in FIG. 4, in the case of the normal fetching method of the S / H signal 10, the above-described malfunction occurs. However, if the polarity is inverted when the data is input to the microcomputer 12, the microcomputer 12 recognizes that the dark portion is bright and the bright portion is dark, so that the S / H signal 10 as shown in FIG. Then, the motion vector vg obtained from the difference vp between the fields of the position of the center of gravity of the brightness can be obtained with the same accuracy.

In the above embodiment, the first arithmetic means and the second arithmetic means are performed by software using the microcomputer 12, but any means capable of performing the above arithmetic operation may be used. It is not limited.

In this embodiment, the correction of the screen shake is
Although the control is performed by controlling the timing of extracting the electric charge from the CCD 2, a configuration in which the lens 1 and the unit to which the CCD 2 is attached may be mechanically moved may be used. Further, in the above embodiment, the calculation area in the first calculation means is limited in order to eliminate the influence of surroundings outside the subject, and is not limited to the area from 52H to 230H as in this embodiment. Further, in this embodiment, the S / H luminance signal level is taken in every horizontal scanning period, but may be taken at appropriate intervals.

[0030]

As described above, according to the present invention, the center of gravity of the brightness of each field is detected from the level of the luminance signal, and the motion vector is detected from the difference in the center of gravity of the brightness between the fields. , The position where the image is projected on the screen is corrected so that the motion vector becomes zero, and the feature of the brightness of the image is extracted, for example,
As in the case of light, the detected motion vector and the true motion
When there is a possibility that the error of
Invert video signal information instead of motion vector
Motion vector is detected using the center of gravity
Because the position of the surface is corrected,
Even if there is, it is possible to accurately and accurately perform motion compensation
This has the effect. In addition, according to this configuration, large capacity
High-precision movement without using memory or large-scale IC
There is an effect that correction can be performed .

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a video movie including a motion compensation device according to an embodiment of the present invention.

FIG. 2 is a diagram for explaining a screen shake correction operation based on a detected motion vector according to the embodiment.

FIG. 3 is a signal waveform diagram for explaining the operation of the motion vector detecting means of the embodiment.

FIG. 4 is a diagram for explaining an operation of motion compensation at the time of backlight in the embodiment.

FIG. 5 is a flowchart showing an operation of a first arithmetic unit in the microcomputer.

FIG. 6 is a flowchart showing an operation of a second arithmetic unit in the microcomputer.

FIG. 7 is a diagram illustrating the operation of a conventional motion compensation device.

FIG. 8 is a diagram illustrating a setting state of a detection area by a conventional motion correction device.

FIG. 9 is an explanatory diagram of an operation of correcting screen shake based on a detected motion vector.

[Explanation of symbols]

 12 microcomputer 30 motion vector detecting means 40 motion correcting means

Claims (1)

(57) [Claims] 1. An information processing apparatus which is set on a screen from information of a video signal.
The center of gravity of brightness in the shake detection area is set for each field.
Center position detecting means for detecting the brightness of each field detected by the center position detecting means.
Motion vector that detects the difference between the centroid positions of
Vector detecting means , and a motion vector detected by the motion vector detecting means.
Motion correction means for correcting the position of the screen based on the screen, and the brightness peak within the motion detection area set on the screen
Brightness peak position detection that detects the position for each field
Means and the brightness peak position from the brightness peak position for each field.
Movement vector detecting means for detecting the movement vector of the lock position
And the brightness distribution in the motion detection area set on the screen
Feature extraction means for determining the state based on the movement vector
And the feature extraction means determines that the movement vector
If it is determined that the value is larger than the value,
And the brightness center of gravity position is detected by the center of gravity position detection means.
And the motion vector is detected from the position of the center of gravity of the brightness.
Screen position based on the motion vector detected by the
Positive, and it is determined that the movement vector is smaller than a predetermined value.
The motion detected by the motion vector detecting means.
Control to perform screen position correction by vector
A motion compensating device comprising a control means .