JP3751800B2 - Apparatus and method for correcting shake of imaging apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a shake correction apparatus and method for an image pickup apparatus that corrects a motion of an image in a video signal caused by a shake of a video camera, for example.
[0002]
[Prior art]
As is well known, for example, a video camera used for field monitoring may be shaken together with the video camera due to external factors such as wind depending on the installation location. For example, in a road surveillance, when a video camera is attached to a pole, the video camera attached to the pole is shaken in the same manner as the pole is shaken by wind or car vibration. In that case, the image of the video camera becomes very unsightly because the angle of view fluctuates with the shaking of the pole.
[0003]
In order to solve such a problem, the conventional video camera adds a shake correction circuit to reduce the movement of the image in the final video signal due to the shake so that the shake is not noticeable. In this type of shake correction circuit, a motion vector between temporally continuous screens obtained by the image sensor is detected, and the motion of the image in the video signal due to the shake of the video camera is corrected based on the detection result. The first method or the second method for correcting the motion of the image in the video signal from the detection result of the angular velocity sensor using the angular velocity sensor that physically detects the amount of shaking of the video camera is employed.
[0004]
The details of the shake correction technique by the above-described methods are described in Japanese Patent Application Laid-Open No. 11-275441 and Japanese Patent Publication No. 5-23545, and the description thereof is omitted here.
[0005]
[Problems to be solved by the invention]
By the way, in the first method, the detection accuracy is high, there is no variation due to individual differences and aging, and the memory extraction can be performed with the actual amount of movement (pixel pitch) itself. have.
(1) For a moving subject, the motion of the subject is detected as a vector quantity instead of the motion of the video camera.
(2) Vectors cannot be detected well for subjects with low contrast (including dark subjects).
(3) In the high sensitivity mode, correction becomes difficult because the exposure sample time of the video is reduced.
[0006]
Further, in the second method, since the video camera itself is detected, it is not affected by the state of the subject (video signal) like the motion vector. However, it has the following problems.
(4) The detection signal is small when the movement is not accompanied by rotation depending on the video camera mounting state.
(5) The sensor output is weak against noise because it is an analog minute signal. For this reason, when the image is enlarged by zooming the lens, the amount of motion of the video with respect to the shake of the video camera increases, the sensor output signal per unit pixel (the amount of shake converted to the video) decreases, and the SN is poor. Detection accuracy deteriorates.
(6) Variation in sensor output (sensitivity, offset) is large.
Therefore, the object of the present invention is to avoid fluctuations in the imaging device without being affected by variations in sensor output due to individual differences and differences between the actual shaking of the imaging device due to installation conditions of the imaging device and the conversion rate of the rotational motion. An object of the present invention is to provide a shake correction apparatus and method for an imaging apparatus that can improve detection accuracy and can reliably correct the movement of an image due to the shake of the imaging apparatus.
[0007]
[Means for Solving the Problems]
The shake correction apparatus for an image pickup apparatus according to the present invention includes a motion vector detection unit that detects a motion vector from a video signal output from an image pickup device, and a shake of the image pickup apparatus based on a detection result by the motion vector detection unit. In a shake correction apparatus for an imaging apparatus that includes a motion correction unit that corrects a motion of an image in a video signal, a physical sensor that physically detects the shake amount of the imaging apparatus, and an output of the physical sensor at an arbitrary amplification factor Amplifying means for amplifying, signal converting means for converting an output signal of the amplifying means into a signal corresponding to a motion vector detected by the motion vector detecting means, and an output signal of the signal converting means in accordance with a control instruction given from the outside Amplification control means for controlling the amplification factor of the amplification means so that the output signal of the motion vector detection means substantially coincides with the output signal of the signal conversion means and the motion An execution / stop control means for judging the reliability of the motion vector by comparing the output signal of the shuttle detection means, and controlling the execution / stop of the process for the motion correction means according to the degree of the reliability. It is intended to provide.
[0008]
In this configuration, when correcting the motion of the image in the video signal due to the shaking of the imaging device using the motion vector, the actual amount of shaking of the imaging device is detected using the physical sensor, and the motion vector When the amount of shaking detected by the physical sensor with respect to the magnitude is small, it is determined that the reliability of the motion vector is low, and the motion correction process is stopped. At this time, in the comparison between the two, problems due to variations in individual output of the physical sensor and differences between the conversion rate of the actual shaking of the imaging device and the rotational motion due to the installation conditions of the imaging device become a problem. Therefore, when the imaging apparatus is installed, the imaging apparatus is shaken, and a control operation using a switch or the like is performed so that the motion vector equivalent signal and the motion vector detection signal detected from the video signal are substantially matched. By controlling the amplification factor, the above-mentioned problems are solved, and the shake detection accuracy by the physical sensor is improved, and the accuracy of comparison and determination between the physical sensor output and the motion vector is improved.
[0009]
A shake correction apparatus for an image pickup apparatus according to the present invention includes a physical sensor that physically detects a shake amount of the image pickup apparatus, and a video signal output from the image sensor based on the shake amount of the image pickup apparatus detected by the physical sensor. In a shake correction apparatus for an image pickup apparatus comprising a motion correction means for correcting the movement of an image in the image, a motion vector detection means for detecting a motion vector from a video signal output from the image sensor, and an arbitrary amplification of the output of the physical sensor Amplifying means for amplifying at a rate, a signal converting means for converting the output signal of the amplifying means into a signal corresponding to a motion vector detected by the motion vector detecting means, and giving to the motion correcting means, and according to a control instruction from the outside And an amplification control means for controlling the amplification factor of the amplifying means so that the output signal of the signal converting means substantially coincides with the output signal of the motion vector detecting means. Than it is.
[0010]
In this configuration, when the movement of the image in the video signal due to the shaking of the imaging device using the physical sensor is corrected, the variation of the physical sensor output due to individual differences and the actual imaging device depending on the installation conditions of the imaging device. Focusing on the problem of the difference between the fluctuation of the vibration and the conversion rate of the rotational motion, the imaging device is shaken when the imaging device is installed, the output of the physical sensor is amplified by the control operation such as a switch, and the movement By converting the signal into a vector-equivalent signal, comparing it with the motion vector, and controlling the amplification factor for the physical sensor output so that the two are almost the same, the above problem is solved, and the amount of shake detection by the physical sensor is detected. Has improved.
[0011]
Further, in the above configuration, the amplification control means includes storage means for storing a control result when the output signal of the signal conversion means and the detection signal by the motion vector detection means substantially coincide with each other during the control operation. Is not provided, the amplification factor corresponding to the control result stored in the storage unit is set in the amplification unit.
[0012]
In this configuration, only when the image pickup device is installed or when there is a control operation, the amplification factor is controlled for the physical sensor output, and the signal corresponding to the motion vector and the motion vector detected from the video signal at that time are substantially the same. If the control result is stored in the storage means, then the amplification factor corresponding to the control result stored in the storage means is set in the amplification means, and the physical sensor shake amount at the installation location without the control operation The detection accuracy can be maintained, and the motion of the image in the video signal due to the shaking of the imaging device can be corrected without being affected by the state of the subject. In addition, since the control result stored in the storage means is updated each time the installation location is changed and a control operation is performed, it is possible to deal with installation conditions and secular changes.
[0013]
In the above configuration, the storage means stores an amplification factor according to the comparison result between the output signal of the signal conversion means and the detection signal of the motion vector detection means, and the amplification control means is given a control instruction from the outside. If not, the output signal of the signal conversion means is compared with the detection signal from the motion vector detection means, and the amplification factor corresponding to the comparison result is read from the storage means and set in the amplification means.
[0014]
In this configuration, an amplification factor corresponding to the comparison result between the output signal of the signal conversion unit and the detection signal of the motion vector detection unit is stored in the storage unit, so that the actual imaging apparatus can be used without a control operation. Corresponding to the amount of shaking, the movement of the image in the video signal can be finely corrected.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0016]
(First embodiment)
FIG. 1 is a block diagram showing the configuration of an embodiment of a video camera shake correcting apparatus according to the present invention.
[0017]
In FIG. 1, an image picked up via the zoom lens 1 is converted into an electric signal by the image pickup device 2 and subjected to sampling processing in a sample / hold circuit (S / H circuit) 3. The output signal of the sample and hold circuit 3 is converted into a digital signal by an A / D (analog / digital) converter 4 and processed by the signal processing circuit 5 such as white balance, gamma correction, and detail. Stored in
[0018]
The output signal of the A / D converter 4 is also input to the motion vector detection circuit 7 to obtain a motion vector signal S1. This motion vector signal S1 indicates how many pixels the pixel has moved per unit time (for example, one field) in the X and Y directions. The motion vector signal S1 is input to the memory control circuit 8 and the microcomputer 10. The memory control circuit 8 controls the reading of the memory 6 by determining a cut-out area from the writing area of the memory 6 so as to cancel the input vector amount (the amount of motion of the video camera). The output signal of the memory 6 is enlarged by an electronic zoom circuit 9 and converted to an NTSC video signal by an encoder 11 in order to match the angle of view of a television monitor, and an analog final signal is output by a D / A (digital / analog) converter 12. Get the video output signal.
[0019]
The microcomputer 10 includes an A / D conversion unit 102, a correction unit 103, a gain control unit 104, a non-volatile memory 105, and a comparison determination unit 106.
[0020]
The A / D converter 102 receives a signal obtained by amplifying the output of the angular velocity sensor 13 that physically detects the amount of shaking of the video camera with an AGC (auto gain controller) 15 at an arbitrary amplification factor. Note that the angular velocity sensor 13 is a dual configuration sensor for horizontal and vertical directions, and the AGC amplifier 15 is also a dual configuration amplifier that amplifies detection signals in each direction.
[0021]
The amplified signal of the AGC amplifier 15 is converted into a digital signal by the A / D conversion unit 102, converted into a signal S2 corresponding to a motion vector detected by the motion vector detection circuit 7 by the correction unit 103, and then a gain control unit. 104 and the comparison determination unit 106, respectively. The correction unit 103 performs correction processing so that a signal corresponding to a motion vector is obtained at any zoom position by changing the correction amount according to the zoom position of the zoom lens 1.
[0022]
The gain control unit 104 compares the motion vector signal S1 from the motion vector detection circuit 7 with the output signal S2 of the correction unit 103, and outputs the comparison result signal S4 to the AGC amplifier 15 to control the amplification factor. Further, the gain control unit 104 stores the amplification setting value when the motion vector signal S1 and the output signal S2 of the correction unit 103 coincide with each other in the nonvolatile memory 105.
[0023]
The comparison / determination unit 106 compares the motion vector signal S1 with the output signal S2 of the correction unit 103 and sees the correlation thereof, so that the motion vector signal S1 is the amount of shaking of the video camera or the amount of movement of the subject. The memory control circuit 8 is controlled based on the determination result.
[0024]
Further, when receiving the control start notification S5 from the switch (SW) 14, the gain control unit 104 controls the amplification factor of the AGC amplifier 15 so that S1 and S2 coincide with each other, and the amplification setting value when the coincidence is obtained is nonvolatile. Stored in the memory 105. Thereafter, the AGC amplifier 104 operates with the amplification set value stored in the nonvolatile memory 105.
[0025]
The control start operation by the switch 14 is performed on a vibrating jig with a production line or when the video camera is shaking with the video camera actually installed. Thereby, it is possible to absorb the influence of the variation in the angular velocity sensor 13 and the conditions of the installation location, improve the accuracy of the comparison determination unit 106 (can set the threshold value small), and prevent erroneous determination. Is possible.
[0026]
Next, the principle of shake correction in the above configuration will be described with reference to FIG. In the memory 6, the memory cut-out area (the actual reading range) is moved by the memory control circuit 8 in the direction opposite to the direction in which the video camera has moved. Since the readout signal is smaller than the actual imaging range, the electronic zoom circuit 9 enlarges the readout signal with the electronic zoom and matches the angle of view of the television monitor.
[0027]
Next, a correlation determination method in the comparison determination unit 106 will be described with reference to FIG.
The comparison determination unit 106 calculates the difference between the motion vector signal S1 and the output signal S2 of the correction unit 103, and determines that the motion vector has detected the motion of the video camera if the value is within the threshold range. That is, since the output of the angular velocity sensor 13 reacts only to the movement of the video camera, if the correlation with the detection result of the angular velocity sensor 13 is strong, it can be determined that the shake of the video camera has been detected. Note that the threshold value is given to the judgment because the output of the angular velocity sensor 13 varies due to individual differences, and the difference between the actual camera shake and the angular velocity conversion rate depending on the mounting state of the video camera. This is because the relationship with the actual motion vector varies even if it is a signal (not completely matched).
[0028]
Next, the operation of the comparison determination unit 106 will be described with reference to the flowchart of FIG.
First, the comparison / determination unit 106 obtains a difference value α between the motion vector signal S1 and the output signal S2 of the correction unit 103 (step a1), and determines whether the difference value α is smaller than a threshold value TH ( Step a2). If the difference value α is larger than the threshold value TH (No), the comparison / determination unit 106 stops the image motion correction process of the memory control circuit 8 (step a3).
[0029]
On the other hand, if the difference value α is less than the threshold value, the comparison / determination unit 106 gives an image motion correction control instruction S3 to the memory control circuit 8 (step a4).
[0030]
As a result, the influence of the detection sensitivity of the angular velocity sensor 13 due to the variation (individual difference, secular change) of the angular velocity sensor 13 and the conditions of the installation location of the video camera is absorbed, and the accuracy of the comparison determination unit 106 is improved (the threshold value is reduced). Can be set), and erroneous determination can be prevented.
[0031]
As described above, in the first embodiment, when the motion of the image in the video signal due to the shake of the video camera is corrected using the motion vector, the angular velocity sensor 13 is used to correct the actual video camera. The amount of shaking is detected, the output of the angular velocity sensor 13 is amplified by the AGC amplifier 15, converted into a motion vector equivalent signal S2 by the correction unit 103, and then compared with the motion vector signal S1 by the comparison / determination unit 106. The reliability of the vector is determined, and when the reliability of the motion vector is obtained to some extent based on the determination result, the memory control circuit 8 is caused to execute a correction process, and when the reliability is low, the memory control circuit 8 Processing is stopped.
[0032]
For this reason, when the amount of shaking of the video camera detected by the angular velocity sensor 13 is small relative to the magnitude of the motion vector, it can be determined that the reliability of the motion vector is low, and the processing of the memory control circuit 8 is stopped. By doing so, it is possible to prevent the movement of the image in the video signal from being corrected when the reliability of the motion vector is low.
[0033]
Further, the gain controller 104 compares the motion vector signal S2 with the motion vector signal S1 by receiving the control start notification S5 by the switch 14 while the video camera is shaken when the video camera is installed. Since the amplification factor of the AGC amplifier 15 is controlled so that they substantially coincide with each other, the variation due to individual differences in the output of the angular velocity sensor 13 and the conversion rate between the actual shaking and the rotational motion of the video camera due to the installation conditions of the video camera And the accuracy of the shake amount detection by the angular velocity sensor 13 and the comparison determination accuracy of the comparison determination unit 106 can be improved.
[0034]
Further, the gain control unit 104 stores the amplification set value when the motion vector equivalent signal S2 and the motion vector signal S1 substantially coincide with each other during the control operation in the nonvolatile memory 105, and there is no control operation by the switch 14 Thus, the amplification setting value stored in the nonvolatile memory 105 is read out and set in the AGC amplifier 15.
[0035]
In this way, the accuracy of detecting the amount of shaking of the angular velocity sensor 13 at the installation location can be maintained without the control operation by the switch 14, and the video camera has been shaken without being affected by the state of the subject. The motion of the image in the video signal can be corrected. In addition, every time the installation location changes and a control operation is performed by the switch 14, the amplification setting value stored in the non-volatile memory 105 is updated, so that various installation conditions and secular changes can be dealt with.
[0036]
(Second Embodiment)
FIG. 5 is a block diagram showing the configuration of the second embodiment of the video camera shake correcting apparatus according to the present invention. 5, the same parts as those in FIG. 1 are denoted by the same reference numerals, and different parts will be described here.
[0037]
The difference from FIG. 1 is that the control process of the memory control circuit 8 is performed based on the detection result of the angular velocity sensor 13. Since the angular velocity sensor 13 detects the motion of the video camera itself, the comparison / determination unit 106 used in the first embodiment is not necessary. That is, the output signal S2 of the correction unit 103 is input to the memory control circuit 8, and memory cutout control is performed based on the signal S2.
[0038]
In the second embodiment, when the motion of the image in the video signal due to the shaking of the video camera using the angular velocity sensor 13 is corrected, variation due to individual differences in the output of the angular velocity sensor 13 or the installation of the video camera. Focusing on the fact that the difference between the actual shake of the video camera depending on the conditions and the conversion rate of the rotational motion becomes a problem, the video camera is shaken when the video camera is installed, and the angular velocity sensor 13 is controlled by the control operation by the switch 14. Is amplified by the AGC amplifier 15, converted into a motion vector equivalent signal S2 by the correction unit 103, compared with the motion vector signal S1 by the gain control unit 104, and amplified by the AGC amplifier 15 so that the two substantially coincide with each other. By controlling the rate, the detection accuracy of the shaking amount by the angular velocity sensor 13 is improved. At this time, the amplification set value when the motion vector signal S1 and the output signal S2 of the correction unit 103 substantially match is stored in the nonvolatile memory 105. Thereafter, when there is no control operation by the switch 14, The amplification set value stored in the nonvolatile memory 105 is set in the AGC amplifier 15 in advance.
[0039]
For this reason, even when the control operation by the switch 14 is not performed, it is affected by the variation due to individual differences in the output of the angular velocity sensor 13 and the difference between the actual shake of the video camera and the conversion rate of the rotational motion due to the installation conditions of the video camera. In addition, it is possible to reliably correct the movement of the image due to the shaking of the video camera while improving the accuracy of detecting the amount of shaking of the angular velocity sensor 13 without being affected by the state of the subject.
[0040]
(Other embodiments)
The present invention is not limited to the above embodiments. For example, the non-volatile memory 105 may store an amplification setting value corresponding to the comparison result between the motion vector signal S1 and the output signal S2 of the correction unit 103. In this case, the gain control unit 104 always compares the motion vector signal S1 and the output signal S2 of the correction unit 103 without the control start notification S5 by the switch 14, and sets the amplification set value corresponding to the comparison result. The data is read from the nonvolatile memory 105 and set in the AGC amplifier 15.
[0041]
In this way, by storing the amplification set value according to the comparison result between the motion vector signal S1 and the output signal S2 of the correction unit 103 in the nonvolatile memory 105, the actual operation can be performed without the control operation by the switch 14. The motion of the image in the video signal obtained by the image sensor 2 can be finely corrected in accordance with the amount of shaking of the video camera.
[0042]
Further, in each of the above embodiments, each part in the microcomputer 10 has a hardware configuration. However, even a software configuration can be similarly implemented.
[0043]
In addition, the configuration and type of the video camera, the type of angular velocity sensor, and the content stored in the nonvolatile memory can be variously modified and implemented without departing from the gist of the present invention.
[0044]
【The invention's effect】
As described above in detail, according to the present invention, even when the installation location of the imaging device is changed, the detection sensitivity of the angular velocity sensor can be corrected only by performing a simple control operation. Ensures the movement of the image in the video signal due to the shaking of the imaging device without being affected by variations due to individual differences or the difference between the actual shaking of the imaging device due to the installation conditions of the imaging device and the conversion rate of the rotational motion It is possible to provide a shake correction apparatus and method for an imaging apparatus that can correct the image to the minimum.
[Brief description of the drawings]
FIG. 1 is a block diagram showing the configuration of a first embodiment of a shake correction apparatus for a video camera according to the present invention.
FIG. 2 is a view for explaining the principle of shake correction in the first embodiment;
FIG. 3 is a view for explaining a correlation determination method in the comparison determination unit shown in FIG. 1;
4 is a flowchart shown for explaining the operation of the comparison / determination unit shown in FIG. 1; FIG.
FIG. 5 is a block diagram showing the configuration of a video camera shake correction apparatus according to a second embodiment of the present invention;
[Explanation of symbols]
1 ... zoom lens,
2 ... Image sensor,
3. Sample hold circuit (S / H circuit),
4 ... A / D converter,
5 ... Signal processing circuit,
6 ... Memory,
7: Motion vector detection circuit,
8: Memory control circuit,
9 ... Electronic zoom circuit,
10 ... Microcomputer,
11: Encoder,
12 ... D / A converter,
13 ... Angular velocity sensor,
14 ... Switch (SW)
15 ... AGC (auto gain controller) amplifier,
102 ... A / D converter,
103 ... correction part,
104: Gain control unit,
105 ... non-volatile memory,
106: Comparison determination unit.

Claims (6)

Motion vector detection means for detecting a motion vector from the video signal output from the image sensor, and motion correction for correcting the motion of the image in the video signal due to the shaking of the imaging device based on the detection result by the motion vector detection means A shake correction apparatus for an imaging apparatus comprising:
A physical sensor that physically detects the amount of shaking of the imaging device;
Amplifying means for amplifying the output of this physical sensor at an arbitrary amplification factor;
A signal conversion means for converting the output signal of the amplification means into a signal corresponding to a motion vector detected by the motion vector detection means;
Amplification control means for controlling the amplification factor of the amplification means so that the output signal of the signal conversion means substantially coincides with the output signal of the motion vector detection means in response to a control instruction given from the outside,
The output signal of the signal conversion means and the output signal of the motion vector detection means are compared to determine the reliability of the motion vector, and depending on the degree of the reliability, A shake correction apparatus for an image pickup apparatus, comprising execution / stop control means for controlling stop. A physical sensor that physically detects the amount of shaking of the imaging device, and a motion correction unit that corrects the motion of the image in the video signal output from the imaging device based on the amount of shaking of the imaging device detected by the physical sensor In a shake correction device for an imaging device comprising:
Motion vector detection means for detecting a motion vector from a video signal output from the image sensor;
Amplifying means for amplifying the output of the physical sensor at an arbitrary amplification factor;
A signal converting means for converting the output signal of the amplifying means into a signal corresponding to a motion vector detected by the motion vector detecting means, and giving the signal to the motion correcting means;
Amplification control means for controlling the amplification factor of the amplification means so that the output signal of the signal conversion means substantially matches the output signal of the motion vector detection means in response to an external control instruction. A shake correction apparatus for an image pickup apparatus characterized by the above. The amplification control means includes storage means for storing a control result when the output signal of the signal conversion means substantially coincides with the detection signal from the motion vector detection means during a control operation, and is given a control instruction from the outside. 3. The shake correction apparatus for an imaging apparatus according to claim 1, wherein an amplification factor corresponding to a control result stored in the storage unit is set in the amplification unit when there is not. The storage means stores an amplification factor according to a comparison result between the output signal of the signal conversion means and the detection signal of the motion vector detection means,
The amplification control means compares the output signal of the signal conversion means and the detection signal from the motion vector detection means when no external control instruction is given, and stores the amplification factor corresponding to the comparison result. 4. The shake correction apparatus for an image pickup apparatus according to claim 3, wherein the shake correction apparatus is read from the means and set in the amplification means. In a shake correction method for an imaging apparatus that detects a motion vector from a video signal output from an imaging element and corrects a motion of an image in the video signal due to the shaking of the imaging apparatus based on the detection result.
Physically detecting the amount of shaking of the imaging device by a physical sensor;
Amplify the detection output of this physical sensor with an arbitrary amplification factor,
This increased signal is converted into a signal corresponding to the motion vector,
In accordance with a control instruction given from the outside, the amplification factor when amplifying the physical sensor output is controlled so that a signal corresponding to the motion vector substantially matches a motion vector detected from the video signal,
Comparing the motion vector equivalent signal and the motion vector detected from the video signal to determine the reliability of the motion vector, and controlling the execution / stop of the motion correction process according to the degree of the reliability. A method for correcting shake of an imaging apparatus. In the shake correction method for an image pickup apparatus that corrects the movement of an image in a video signal output from an image pickup device based on the shake amount of the image pickup apparatus detected by a physical sensor,
Detecting a motion vector from a video signal output from the image sensor;
Amplify the output of the physical sensor at an arbitrary amplification factor,
The amplified signal is converted into a signal corresponding to a motion vector detected from the video signal, and the motion of the image in the video signal is corrected by the signal corresponding to the motion vector,
In accordance with an external control instruction, an amplification factor for amplifying the physical sensor output is controlled such that a signal corresponding to the motion vector substantially matches a motion vector detected from the video signal. A shake correction method for an imaging apparatus.

Images