JP3564247B2 - Image motion compensation device - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image motion correction device used for camera shake correction of an imaging device.
[0002]
[Prior art]
For example, the following devices (1) to (3) have been proposed as image motion correcting devices for imaging devices.
[0003]
{Circle around (1)} A method in which an image pickup unit having an image pickup optical system and a solid-state image pickup device is supported by a gimbal mechanism, and drive control is performed based on movement information of the image pickup apparatus itself obtained from an angular velocity sensor, thereby correcting image movement. (For example, see "Development of Image Shake Prevention Technology for Video Cameras," Technical Report of the Institute of Television Engineers of Japan, Vol. 11, No. 28, pp. 19-24 (1987)).
[0004]
{Circle around (2)} A method of correcting a motion of an image by providing a variable apex angle prism in front of an imaging optical system and driving and controlling the same based on information from an angular velocity sensor (for example, an “optical image stabilization system” TV) See John Technical Report Vol. 17, No. 5, pp. 15-20 (1993)).
[0005]
{Circle around (3)} A method of detecting the presence or absence of image movement from an image signal obtained by capturing an image of a subject, and clipping a part of the original image based on the detection result to correct the image movement (for example, “” Pure electronic image stabilization system, "Technical Report of the Institute of Television Engineers of Japan, Vol. 15, No. 7, pp. 43-48 (1991)").
[0006]
[Problems to be solved by the invention]
The above methods (1) and (2) optically correct the motion of an image based on information from the angular velocity sensor, but in such a method, the output drift due to the sensitivity and temperature characteristics of the angular velocity sensor Due to such problems as above, it is difficult to realize highly accurate motion correction.
[0007]
In the method of (3), since all of the motion compensation is performed by signal processing, high-precision motion compensation can be realized, but on the other hand, the larger the focal length of the photographing lens, the more the motion compensation is possible. The angle of view becomes smaller.
[0008]
The present invention solves the above-described conventional problems, and can always secure a fixed angle of view that enables motion correction without being affected by the focal length, and can realize highly accurate motion correction. It is an object to provide an image motion correction device.
[0009]
[Means for Solving the Problems]
In order to solve this problem, the present invention provides a solid-state imaging device, an imaging optical system for forming a subject image on the solid-state imaging device, and an optical shake correction system for optically correcting the movement of a captured image. The following configuration is employed in an image motion compensating apparatus including an image pickup system composed of: and an optical shake correction system drive control unit that drives and controls the optical shake correction system.
[0010]
In the invention according to claim 1, a sensor for detecting the movement of the imaging apparatus itself and a movement detecting means for detecting the movement of an image based on an image signal obtained from the imaging system are provided in parallel, and the sensor and the sensor Signal combination means for combining both signals of the motion detection result by the motion detection means to form a drive signal,The signal synthesizing unit converts the information on the motion of the imaging device itself detected by the sensor and the motion detection result by the motion detecting unit into a predetermined multiplier α, 1−α (where 0 ≦ α ≦ 1). The weighted signal is then combined to synthesize the signal,The optical shake correction system is drive-controlled by a drive signal output from the signal synthesizing means to correct the motion of the image. As a result, highly accurate motion correction can be realized.
[0011]
In the invention according to claim 2, in the configuration according to claim 1,The combining by the signal combining means is a process of adding both signals weighted by a predetermined multiplier α, 1−α.
[0012]
In this case, as the synthesizing process by the signal synthesizing means, both signals weighted by predetermined multipliers α and 1−α can be added. The multiplier α can also be determined based on information on one or both of the vibration frequency band of the imaging apparatus itself and the focal length of the imaging optical system. This makes it possible to realize high-precision motion correction while securing a fixed correction angle of view regardless of the focal length. Further, the multiplier α can be determined according to the ratio between the high frequency band component and the low frequency band component of the vibration of the imaging device itself, or the difference between the two components. Furthermore, the frequency band component of the vibration of the imaging device itself can be extracted from the output of the sensor or the output of the motion detecting means.
[0013]
Claim5In the invention according to the description, the claims3 or Claim 4The image motion compensating device according toThe frequency band component of the vibration of the imaging device itself can be extracted from the output of the sensor or the output of the motion detecting means.
[0014]
Claim6In the invention according to the description, the claims1StatedIn addition to the configuration, a storage unit for storing an image signal obtained from the imaging system, and a storage unit driving control unit for controlling the reading of the image signal by driving the storage unit are provided, and output from the combining unit. The optical shake correction system is driven and controlled by a drive signal, and the storage unit drive control unit is configured to output an image signal from the storage unit by another drive signal calculated based on a motion detection result by the motion detection unit. The readout of images to correct image movementThings. Thereby, high-precision motion correction can be realized while securing a fixed correction angle of view and without recognizing the deterioration of resolution due to the motion correction of the photographer, regardless of the focal length.
[0015]
Claim7In the invention according to the descriptionAn electronic zooming means for performing an electronic enlarging process on the image signal read from the storage means, in addition to the configuration according to claim 6, wherein the optical signal is outputted by the synthesizing means. The drive control unit controls the shake correction system, and the storage unit drive control unit calculates the value based on the motion detection result by the motion detection unit only when executing the electronic enlargement process in accordance with a command from the electronic zoom unit. The reading of the image signal from the storage means is controlled by the other drive signal to correct the motion of the image. Thereby, high-precision motion correction can be realized while securing a fixed correction angle of view and without recognizing the deterioration of resolution due to the motion correction of the photographer, regardless of the focal length.
[0016]
In the invention according to claim 8,A sensor for detecting the motion of the imaging device itself and a motion detection unit for detecting the motion of the image based on the image signal obtained from the imaging system are provided in parallel, and a motion detection result of the sensor and the motion detection unit is provided. Signal synthesizing means for synthesizing both signals to form a drive signal,
The signal synthesizing unit includes a unit that stops outputting a drive signal obtained based on a detection result of the motion detection unit during a period in which the focal length is changed by the imaging optical system, and the focus is controlled by the imaging optical system. During the period in which the distance is changing, only the drive signal formed from the information on the movement of the imaging device itself detected by the sensor, and in the period in which the focal length is fixed, the sensor and the movement detection are performed. Each of the optical shake correction systems is driven by a drive signal formed by synthesizing both signals of the motion detection result by the means.This is to control the movement of the image by controlling. Accordingly, it is possible to realize high-precision motion correction while ensuring a fixed correction angle of view regardless of the focal length, and without malfunction even while the focal length is changing.
[0017]
BEST MODE FOR CARRYING OUT THE INVENTION
(Embodiment 1)
FIG. 1 is a block diagram illustrating an image motion correction device according to Embodiment 1 of the present invention.
[0018]
In FIG. 1, an optical shake correction system 1 is a means for optically correcting the movement of an image caused by the shake of an imaging device. In this example, a variable apex angle prism (hereinafter abbreviated as “VAP”) is used as an example. Apply). Since the details of VAP1 are disclosed in, for example, Japanese Patent Application Laid-Open No. Hei 2-13901, a detailed description thereof will be omitted.
[0019]
The optical shake correction system drive control unit 2 is a unit for driving and controlling the VAP 1. The angle detecting means 3 is a means for detecting and outputting the actual rotation angle of the plane parallel plate of the VAP 1, and a feedback control loop for driving and controlling the VAP 1 together with the optical vibration correction system driving control means 2. Is composed.
[0020]
The imaging optical system 4 includes a lens system that performs an optical zoom operation and a focusing operation, and an imaging optical system drive control unit 5 controls the driving of the imaging optical system 4 to perform the optical zoom operation and the focusing operation. A system control unit 16 for performing a focusing operation, which stores information on the focal length of the imaging optical system 4 as described later.₁Output.
[0021]
The solid-state imaging device 6 converts an image incident through the VAP 1 and the imaging optical system 4 into an electric signal. The device 6 is driven and controlled by a solid-state imaging device drive control unit 17. I have.
[0022]
The analog signal processing unit 7 is a unit for performing analog signal processing such as gamma correction on an image signal obtained by the solid-state imaging device 6. The A / D converter 8 converts an analog signal into a digital signal. The digital signal processor 9 removes noise and enhances the contour of the image signal converted into a digital signal by the A / D converter 8. And so on.
[0023]
The image signal converted into a digital signal by the A / D converter 8 is also sent to the motion detector 10 at the same time, where a vector (hereinafter, referred to as a vector) indicating the horizontal / vertical displacement of the image between the fields. This is called a detection vector) is detected for each field.
[0024]
The sensor 11 is for detecting the movement of the imaging device itself, and in the first embodiment, an angular velocity sensor that detects an angular velocity is applied as an example.
[0025]
Although two angular velocity sensors 11 are required to detect movement in two directions, yaw and pitch, only one angular velocity sensor 11 is shown in the figure.
[0026]
The gain adjusting means 12 adjusts the signal level of the output of the angular velocity sensor 11, and is composed of, for example, an amplifier circuit. The HPF 13 is a high-pass filter for removing a drift component contained in the output of the angular velocity sensor 11, and the LPF 14 is a low-pass filter for removing a noise component contained in the output of the angular velocity sensor 11. The D conversion means 15 is a means for converting the output of the angular velocity sensor 11 into a digital signal.
[0027]
The system control means 16a corresponds to the synthesizing means in the claims, and performs the motion correction of the image based on the information obtained from the motion detection means 11, the imaging optical system drive control means 5 and the angular velocity sensor 11, respectively. Information (hereinafter, referred to as a motion correction signal) required for the calculation, and sends the calculated information to the optical shake correction system drive control means 2. The optical shake correction system drive control means 2 drives and controls the VAP 1 based on the motion correction signal.
[0028]
FIG. 2 is a block diagram showing an example of the internal configuration of the motion detection means 10.
[0029]
The motion detection means 10 includes a representative point storage circuit 101, a correlation operation circuit 102, and a motion vector detection circuit 103.
[0030]
The representative point storage circuit 101 divides the image signal of the current field input through the A / D converter 8 into a plurality of regions, and converts the image signal corresponding to a specific representative point included in each region into a representative point signal. In addition, the representative point signal one field before the current field which has already been stored is read out and supplied to the correlation operation circuit 102.
[0031]
The correlation operation circuit 102 performs a correlation operation between the previous representative point signal and the image signal of the current field, and compares the difference between the previous representative point signal and the image signal of the current field. 103.
[0032]
The motion vector detection circuit 103 detects a motion vector (detection vector) of an image between the previous field and the current field on a pixel-by-pixel basis from the operation result of the correlation operation circuit 102.
[0033]
Accordingly, in this configuration, a time delay of one field period occurs in order to obtain a detection vector in order to perform matching before and after one field of the image.
[0034]
FIG. 3 is a block diagram showing an example of the internal configuration of the system control means 16a.
[0035]
The first gain / phase compensator 161 adjusts the gain and phase of the detection vector obtained by the motion detector 10.
[0036]
The conversion unit 162 converts the detection vector adjusted by the first gain / phase compensation unit 161 into a unit time based on information on the focal length of the imaging optical system 4 obtained from the imaging optical system drive control unit 5. This is converted into a moving angle (angular velocity) per hit. The unit time in this case can be set to, for example, one cycle when sampling the output of the angular velocity sensor 11.
[0037]
The second gain / phase compensator 163 adjusts the gain and phase of the detection output of the angular velocity sensor 11 obtained through the A / D converter 15 in the same manner as the first gain / phase compensator 161.
[0038]
The first band extracting unit 164 is a kind of filter that extracts only a part of band components (here, a low-frequency component suitable for motion correction) from the angular velocity signal obtained through the converting unit 162. Further, the second band extracting unit 165 is a type of extracting only a part of band components (here, a high-frequency component suitable for motion correction) from the angular velocity signal obtained through the second gain / phase compensating unit 163. Filter. The signals passing through the first and second band extracting means 164 and 165 are added by the adder 166.
[0039]
The correction signal generation means 167 integrates the result of addition by the adder 165, and calculates a displacement angle (movement angle) from a reference position set in advance on the screen to generate a motion correction signal.
[0040]
Next, the operation of the image motion correction device according to the first embodiment will be described. A series of operations such as a motion detection operation based on an image signal obtained by the solid-state imaging device 7, a motion detection operation detected by the angular velocity sensor 11, and a drive control of the VAP1 are performed in both the horizontal and vertical directions. However, the contents are basically the same in both the horizontal and vertical directions, so that the description will be made without distinguishing between the horizontal and vertical directions in order to simplify the description.
[0041]
The image signal obtained by the solid-state imaging device 6 is passed through an analog signal processing unit 7, digitized by an A / D conversion unit 8, and sent to a digital signal processing unit 9 and a motion detection unit 10.
[0042]
The motion detecting means 10 detects the amount of displacement (detection vector) of the image before and after one field, and sends the detected vector to the system control means 16a.
[0043]
In the system control unit 16a, first, the gain and phase for the detection vector are adjusted by the first gain / phase compensation unit 161. In particular, the time (phase) delay of one field period which occurs until the motion detecting means 10 obtains the detection vector is compensated. The conversion unit 162 converts a detection vector obtained on a screen in pixel units into an angular velocity per unit time based on information on the focal length of the imaging optical system 4 obtained from the imaging optical system drive control unit 5.
[0044]
The first band extracting unit 164 extracts a low frequency component from the signal output from the converting unit 162 as a band component suitable for motion correction. The reason is that, in the motion detection based on the image signal, more accurate motion detection can be performed in a low frequency band than in the motion detection based on the detection output of the angular velocity sensor 11. That is, when detecting motion based on an image signal, the frequency at which motion can be detected is limited by the sampling period of the image. For example, in the case of the NTSC system, the sampling frequency is 60 Hz. Therefore, when motion is detected based on an image signal, the band of motion detection is limited to 30 Hz or less by the sampling theorem.
[0045]
On the other hand, the second band extracting unit 165 extracts a high-frequency component from the output of the second gain / phase compensating unit 163 as a band component suitable for motion correction. The reason is that when detecting motion based on the detection output of the angular velocity sensor 11, the HPF 13 is provided to remove low-frequency components such as drift contained in the output of the sensor 11, so that the detection accuracy for low-frequency motion is detected. This is because the value has decreased.
[0046]
Both outputs of the first and second band extracting means 164 and 165 are added by an adder 166, and a correction signal generating means 167 obtains a motion correction signal.
[0047]
The optical shake correction system drive control means 2 corrects the motion of the image by controlling the drive of the VAP 1 according to the motion correction signal.
[0048]
That is, in the first embodiment, since the VAP1 is controlled by both the information on the movement in the low frequency band obtained by the movement detecting means 10 and the information on the movement in the high frequency band obtained from the angular velocity sensor 11, the VAP1 is controlled. It is possible to highly accurately correct the motion of the image ranging from the high frequency range to the high frequency range.
[0049]
Further, the first and second band extracting means 164 and 165 are not limited to the case provided immediately before the adder 166 as shown in FIG. 3, for example, the first band extracting means 164 A stage before the gain / phase compensator 161 or between the A / D converter 8 and the motion detector 10, and the second band extractor 165 is between the A / D converter 15 and the system controller 16 a. May be installed separately.
[0050]
(Embodiment 2)
FIG. 4 is a block diagram illustrating a configuration of a system control unit of the image motion compensating apparatus according to the second embodiment of the present invention, and the same reference numerals are given to parts corresponding to FIG. 3 according to the first embodiment.
[0051]
The feature of the second embodiment resides in the configuration of the system control means 16b. That is, the system control unit 16b has a configuration in which first and second multipliers 171 and 172 and a vibration band detection unit 173 are added to the configuration shown in FIG.
[0052]
The first and second multipliers 171 and 172 apply multipliers α and 1−α of a predetermined weight to the respective outputs of the first and second band extracting means 164 and 165 (where 0 ≦ α ≦ 1). Is multiplied by Further, the vibration band detecting unit 173 has a function of performing frequency decomposition of a time series signal such as a fast Fourier transform, for example, and based on an output of the angular velocity sensor 11 obtained through the A / D converting unit 15. The main frequency band of the motion of the imaging device is detected, and the values of the multipliers α, 1−α to be multiplied by the first and second multipliers 171 and 172 are determined based on the result.
[0053]
The other configuration is the same as that shown in FIG. 3 of the first embodiment, and a detailed description is omitted here.
[0054]
Next, the operation of the image motion correction device according to the second embodiment will be described.
[0055]
The vibration band detecting means 173 detects the main vibration band from the movement of the imaging device detected by the angular velocity sensor 11, and based on the result, the multipliers α, 1−1 in the first and second multipliers 171 and 172. Determine α.
[0056]
The first and second multipliers 171 and 172 multiply the outputs of the first and second band extracting means 164 and 165 by the multipliers α and 1-α determined by the vibration band detecting means 173, respectively. .
[0057]
Here, the vibration band detecting means 173 sets α to 0 when the high frequency component (which is determined by the sampling frequency of the image) which makes the motion of the imaging device difficult only by the motion detection based on the image data becomes dominant. Set to. Therefore, the output of the first multiplier 171 is stopped.
[0058]
On the other hand, the vibration band detecting means 173 sets α to 1 when the movement of the imaging device is dominated by low frequency components that are difficult to detect by the angular velocity sensor 11. Therefore, the output of the second multiplier 172 is stopped.
[0059]
As described above, it is possible to stop using one of the two systems of motion detecting means depending on the shooting state, so that the motion can be detected more effectively than in the first embodiment.
[0060]
Further, when there is no noticeable deviation between the high frequency component and the low frequency component, the multiplier α can be changed substantially continuously in the range of 0 or more and 1 or less according to the ratio of the high frequency component and the low frequency component.
[0061]
For example, as shown in FIG. 5A, the value of α is changed linearly according to the ratio between the high-frequency component and the low-frequency component, or the high-frequency component becomes somewhat large as shown in FIG. 5B. Up to α = 1, and the high-frequency component has a constant value ρ₁Is exceeded, the value of α is changed linearly according to the ratio of the high-frequency component to the low-frequency component, and the high-frequency component is further changed to a constant value ρ₂Is exceeded, α is set to 0, or as shown in FIG. Α can be continuously changed in a curved shape in accordance with the ratio between the high frequency component and the low frequency component.
[0062]
The other configuration, operation, operation, and the like in the second embodiment are the same as those in the first embodiment, and thus description thereof will be omitted.
[0063]
As described above, according to the second embodiment, the weighting of the means for detecting the movement of the two systems can be changed in accordance with the photographing state, so that the image from low frequency to high frequency can be further enhanced as compared with the first embodiment. Can be corrected with high accuracy.
[0064]
In the second embodiment, the vibration band detecting unit 173 detects the main frequency band of the motion of the imaging device based on the output of the angular velocity sensor 11, but the present invention is not limited to this. Instead, for example, the vibration band detection unit 173 may be configured to detect a main frequency band of the motion of the imaging device based on the output of the motion detection unit 10. Further, the vibration band detecting means 173 may be configured by a filter means such as a high-pass filter and a low-pass filter.
[0065]
Further, in the second embodiment, the case has been described where the ratio between the high frequency component and the low frequency component is used to determine the weighting multipliers α, 1−α. However, the present invention is not limited to this. The multiplier α, 1−α can also be determined based on the difference between the low frequency component and the low frequency component.
[0066]
(Embodiment 3)
FIG. 6 shows a system control unit 16 of the image motion compensating apparatus according to the third embodiment of the present invention.₃5 is a block diagram showing the configuration of the second embodiment, and portions corresponding to FIG. 4 according to the second embodiment are denoted by the same reference numerals.
[0067]
In the above-described second embodiment, the system control unit 16b includes the vibration band detection unit 173. However, the system control unit 16c of the third embodiment includes a focal length detection unit 183 instead of the vibration band detection unit 173. Provided.
[0068]
That is, the focal length detection unit 183 detects the focal length of the imaging optical system 4 obtained from the imaging optical system drive control unit 5, and weights the multipliers 181 and 182 based on the information on the focal length. Α, 1−α (where 0 ≦ α ≦ 1).
[0069]
The other configuration is the same as that of the above-described second embodiment, and the detailed description is omitted here.
[0070]
Next, the operation of the image motion correction device according to the third embodiment will be described.
[0071]
In general, in the imaging apparatus, the angle of view is narrower as the focal length of the imaging optical system 4 is longer, and conversely, the angle of view is wider as the focal length is shorter. For this reason, even if the imaging device moves by the same angle, the amount of movement appearing on the screen changes depending on the focal length. That is, even when the imaging device moves by the same angle, the motion that appears on the screen increases as the focal length increases, and conversely, the motion that appears on the screen decreases as the focal length decreases.
[0072]
In addition, the camera shake which causes the movement of the imaging apparatus is described in the above-mentioned literature (“Development of Image Shake Prevention Technology for Video Cameras” Technical Report of the Institute of Television Engineers of Japan, Vol. It is known that, when it comes to the unit time, the lower the frequency component, the larger the amount of movement change, and the higher the frequency component, the smaller the amount of movement change.
[0073]
For this reason, when the focal length of the imaging optical system is short, the motion appearing in the captured image is dominated by low frequency components. Also, when the focal length is long, the motion of the image appears largely as the motion of the image in both the low-frequency component and the high-frequency component. However, there is actually a limit to the corrected angle of view. The motion becomes so remarkable that it becomes difficult to correct low frequency components.
[0074]
Therefore, when the focal length of the imaging optical system is short, the low-frequency component is dominant in the motion appearing in the captured image, and the motion correction can be sufficiently performed only by the motion detection by the motion detection unit 10. It is possible, so α is set to one. Thus, the output of the second multiplier 172 is stopped. On the other hand, when the focal length of the imaging optical system is long, it is difficult to correct the motion by the motion detection by the motion detection unit 10 and the high-frequency component becomes dominant. Is set to 0 as described above. Thus, the output of the first multiplier 171 is stopped.
[0075]
As described above, depending on the shooting state, the use of one of the two systems for detecting the movement can be stopped, so that the power consumption can be reduced.
[0076]
When the focal length is set to an intermediate level, α can be changed substantially continuously within a range of 0 or more and 1 or less according to the focal length. In this case, as the method of setting α, the method shown in FIG. 5 can be employed as in the case of the second embodiment.
[0077]
Note that other configurations, operations, operations, and the like in the present embodiment are the same as those in the first and second embodiments, and thus description thereof will be omitted.
[0078]
As described above, according to the third embodiment of the present invention, the motion of the image from low frequency to high frequency can be corrected with high accuracy, and one of the means for detecting the motion of the two systems according to the shooting state is used. It can be stopped, thereby reducing power consumption.
[0079]
In the above-described second and third embodiments, the configuration in which the multiplier α for weighting is automatically changed has been described. However, for example, switch means that can set the multiplier α to 0 or 1 depending on the photographer's intention is provided. If provided, it is possible to stop the use of one of the two means of detecting movement based on the judgment of the photographer, and it is clear that a configuration capable of reducing power consumption can be realized.
[0080]
Note that a configuration in which both the vibration band detection unit 173 described in the second embodiment and the α value setting by the focal length detection unit 183 described in the third embodiment are combined is also conceivable. In this case, the value of the multiplier α can be set more accurately, and the operability according to the shooting situation can be improved.
[0081]
(Embodiment 4)
FIG. 7 is a block diagram illustrating an overall configuration of an image motion compensating apparatus according to Embodiment 4 of the present invention, and FIG. 8 is a block diagram illustrating a configuration of a system control unit of the image motion compensating apparatus. Parts corresponding to FIG. 3 are denoted by the same reference numerals.
[0082]
The output of the motion detecting means 10, that is, the motion included in the image captured by the solid-state imaging device 6 corresponds to a residual motion that cannot be completely corrected only by the VAP1.
[0083]
Therefore, in the fourth embodiment, an image memory 19, an image memory drive control unit 20, and an electronic zoom unit 21 are provided in order to remove the residual motion, as compared with the first embodiment. An integrating means 221 for integrating the output of the motion detecting means 10 is added as the system control means 16d.
[0084]
The image memory 19 temporarily stores the image signal that has passed through the digital signal processing means 9, and is constituted by, for example, a RAM.
[0085]
The image memory drive control means 20 controls writing and reading of the image signal to and from the image memory 19, and when reading out a part of the image signal stored in the image memory 19, converts the image to a desired size. Is sent to the electronic zoom means 21 to be described later.
[0086]
The electronic zoom means 21 performs electronic enlargement processing (electronic zoom processing) on a part of the image signals read from the image memory 19 to enlarge the image.
[0087]
Also, the system control means 16₄Is an integration means for integrating the output of the motion detection means 10.
[0088]
The other configuration is the same as that of the first embodiment, and the detailed description is omitted here.
[0089]
FIG. 9 is a schematic diagram for explaining the principle of the control of reading the image from the image memory 19 and the principle of the motion correction by the enlargement processing of the image by the electronic zoom means 21.
[0090]
In the figure, F1, F2, and F3 are three consecutive field images stored in the image memory 19, and a part of the image (an inner area surrounded by a broken line) is cut out according to the movement. Then, the movement of the image can be corrected, and a part of the cut-out image is enlarged by the electronic zoom means 21.
[0091]
The portion other than the cutout portion of the image (the region outside the region surrounded by the broken line) is a portion corresponding to a margin for motion compensation, and the wider this margin is, the smaller the cutout portion of the image is. The greater the setting, the greater the range in which motion correction can be performed by controlling the reading of the image from the image memory 19, but on the other hand, the enlargement magnification by the electronic zoom means 21 increases, and the resolution of the output image deteriorates.
[0092]
Next, the operation of the image motion correction device according to the fourth embodiment will be described.
[0093]
As described above, the output of the motion detection means 10 corresponds to a residual motion that could not be completely corrected by VAP1 alone.
[0094]
The image memory 19 stores the image for one field while the motion detection means 10 detects the motion of the image. Then, the residual motion of the image detected by the motion detecting means 10 is integrated by the integrating means 221 to obtain the displacement amount from the reference position on the screen, and the information on the displacement amount is sent to the image memory drive control means 20. . The image memory drive control means 20 controls the reading position of the image from the image memory 19 based on the information of the displacement amount.
[0095]
As a result, the residual motion that cannot be completely corrected by only the VAP1 is effectively removed, and thereafter, the image is enlarged to a desired size by the electronic zoom unit 21.
[0096]
Note that other operations, operations, and the like are the same as those in the first embodiment, and a description thereof will not be repeated.
[0097]
As described above, in the fourth embodiment, the motion of the image can be corrected with higher accuracy than in the first embodiment.
[0098]
In the fourth embodiment, a case has been described in which the image signal read from the image memory 19 is subjected to enlargement processing by the electronic zoom means 21. However, the present invention is not limited to this. A solid-state imaging device having a larger number of pixels than the solid-state imaging device 6 is used to capture an image having a number of pixels equal to or more than the number of pixels required for final output, and to store this in the image memory 19. A configuration in which an image is read out for several minutes is also conceivable. In this case, since it is not necessary to electronically enlarge the image, a high-quality output image can be obtained. In this case, in order to perform the motion correction for one pixel and one line or less, it is apparent that the electronic zoom means 21 only needs to perform zoom processing with a zoom magnification of 1 (mere interpolation processing).
[0099]
Further, it is possible to combine the configurations of the second and third embodiments with the configuration of the fourth embodiment.
[0100]
(Embodiment 5)
The feature of the fifth embodiment is that an electronic zoom operation means 23 (see FIG. 7) is further provided in the configuration of FIGS. 7 and 8 according to the fourth embodiment.
[0101]
The electronic zoom operation means 23 is used by the photographer to control the presence / absence (ON / OFF) of electronic enlargement processing (electronic zoom processing) and the zoom magnification when performing the electronic zoom processing. However, the present invention can be realized by an indicating device (zoom lever), a button switch, a dial-shaped indicating device, or the like, but is not limited thereto.
[0102]
The other configuration is the same as that of the fourth embodiment, and the detailed description is omitted here.
[0103]
In the fifth embodiment, when the electronic zoom processing is performed (when the electronic zoom is ON) in accordance with a command from the electronic zoom operation means 23, the image memory drive control means 20 controls the amount of displacement of the image obtained by the integration means 221. The read position of the image from the image memory 19 is controlled based on the zoom magnification, and the image is enlarged by the electronic zoom unit 21.
[0104]
On the other hand, when the electronic zoom process is not performed (when the electronic zoom is OFF), the image memory drive control unit 20 stops the read position control of the image from the image memory 19 and stops the enlargement of the image by the electronic zoom unit 21.
[0105]
As described above, when the electronic zoom is OFF, the motion correction is performed only by the drive control of the VAP1, so that the photographer does not recognize the deterioration of the resolution due to the motion correction of the photographer, and does not deteriorate the resolution of the output image. Is performed. Also, when the electronic zoom is ON, in addition to the correction by the VAP1, the motion correction by cutting out the image from the image memory 19 is performed.
[0106]
The other operations, operations, and the like in the fifth embodiment are the same as those in the fourth embodiment, and thus description thereof is omitted.
[0107]
In the fifth embodiment, it goes without saying that the same effect can be obtained with the solid-state imaging device 6 having a standard number of pixels or a solid-state imaging device having a larger number of pixels than the standard. Furthermore, it is also possible to combine this fifth embodiment with the configurations of the second and third embodiments.
[0108]
(Embodiment 6)
FIG. 10 is a block diagram showing an overall configuration of an image motion compensating apparatus according to Embodiment 6 of the present invention, and FIG. 11 is a block diagram showing a configuration of a system control unit of the apparatus, and FIG. 1 to 3 are denoted by the same reference numerals.
[0109]
The feature of the sixth embodiment is that an optical zoom operation unit 24 is further provided in the configuration of FIG. 1 according to the first embodiment, and a system control unit 16e is further provided in the configuration shown in FIG. , Switch 251 and switch control means 252 are added.
[0110]
The optical zoom operation means 24 is for the photographer to manually operate the focal length of the imaging optical system 4, and the imaging optical system drive control means 5 adjusts the focal length of the imaging optical system by a command from this means 24. The optical zoom is performed by changing.
[0111]
The switch 251 turns ON / OFF the output of the first band extraction unit 164, and the switch control unit 252 sets the focal length of the imaging optical system 4 obtained from the imaging optical system drive control unit 5. The change is detected, and the switch 251 is turned on / off based on the detected change.
[0112]
Other configurations are the same as those of the first embodiment shown in FIGS. 1 to 3, and thus detailed description is omitted here.
[0113]
Next, the operation of the image motion correction device according to the sixth embodiment will be described.
[0114]
During the optical zoom that changes the focal length, the captured image moves radially from the center. For this reason, in the motion detection means 10, since the motion of the image between the fields is not uniform within the detection range, the accuracy of detecting the motion is reduced. On the other hand, the accuracy of the angular velocity sensor 11 detecting the movement of the imaging device itself does not decrease even during the optical zoom.
[0115]
As in the first embodiment, when performing motion correction based on both the results of the motion detection by the motion detection unit 10 and the motion detection by the angular velocity sensor 11 regardless of whether or not the optical zoom is being performed, During the optical zoom, the detection accuracy of the motion detection unit 10 may be reduced, which may cause a malfunction.
[0116]
Therefore, in the sixth embodiment, a change in the focal length of the imaging optical system 4 obtained from the imaging optical system drive control means 5 is detected, and if there is a change in the focal length (that is, during the optical zoom), the switch is turned on. The control unit 252 turns off the switch 251 so that the motion detection result by the motion detection unit 10 is not used for motion correction.
[0117]
As a result, it is possible to realize an imaging apparatus which does not malfunction even during the optical zoom and can perform motion correction with high accuracy.
[0118]
Note that other operations, operations, and the like in the sixth embodiment are the same as those in the first embodiment, and thus description thereof is omitted here.
[0119]
In addition, during the optical zoom, the switch 251 is not simply turned off, but the entire process from the motion detecting unit 10 to the first band extracting unit 164 is stopped by the switch 251 to reduce power consumption. It is also possible.
[0120]
Further, the first and second multipliers 171 and 172 as shown in the second and third embodiments are provided, and the same as the sixth embodiment even when the multiplier α of the first multiplier 171 is set to 0 during the optical zoom. The effect can be realized.
[0121]
In addition, as described in the second and third embodiments, one or both of the vibration band detecting unit 173 and the focal length detecting unit 183 and the two multipliers 171 and 172 are different from the configuration of the sixth embodiment. It is also possible to provide.
[0122]
(Embodiment 7)
FIG. 12 is a block diagram showing the overall configuration of an image motion compensating apparatus according to Embodiment 7 of the present invention, and FIG. 13 is a block diagram showing the configuration of system control means of the image motion compensating apparatus. 8 are given the same reference numerals.
[0123]
The feature of the seventh embodiment is that an optical zoom operation unit 24 is further provided in the configuration of FIG. 7 according to the fourth embodiment, and a system control unit 16f is further provided in the configuration shown in FIG. That is, two switches 251, 261 and a switch control means 252 are added.
[0124]
The optical zoom operation unit 24, the switch 251 on one side, and the switch control unit 252 included in the seventh embodiment correspond to the configuration described in the sixth embodiment.
[0125]
The other switch 261 turns ON / OFF a signal input from the motion detecting means 10 to the integrating means 221, and both switches 251 and 261 are turned ON / OFF based on the detection output of the switch control means 252. It is controlled.
[0126]
The other configuration is the same as that of the fourth embodiment shown in FIGS. 7 and 8, and a detailed description is omitted here.
[0127]
Next, the operation of the image motion correcting apparatus according to the seventh embodiment will be described.
[0128]
In the sixth embodiment, it has been described that during the optical zoom for changing the focal length, the accuracy of the motion detection by the motion detection unit 10 is reduced, which may cause a malfunction. This is also true in the configuration of the fourth embodiment. It just applies.
[0129]
Therefore, in the seventh embodiment, as in the sixth embodiment, the switch 261 is simultaneously turned off together with the one switch 251 during the optical zoom, so that the motion correction based on the motion detection of the image is stopped. At the same time, the motion correction by adjusting the cutout position of the image from is stopped.
[0130]
With this configuration, it is possible to realize an imaging apparatus that does not malfunction even during the optical zoom and can perform motion correction with high accuracy.
[0131]
Note that other configurations, operations, operations, and the like in the present embodiment are the same as those in the fourth and sixth embodiments, and a description thereof will not be repeated.
[0132]
During the optical zoom, the switch 261 is not simply turned off, but the processing by the motion detecting means 10 and the processing by the integrating means 221 and the image memory drive control means 20 are simultaneously stopped to reduce the power consumption. It is also possible to aim.
[0133]
In addition, as described in the second and third embodiments, one or both of the vibration band detecting means 173 and the focal length detecting means 183 and the two multipliers 171 and 172 are different from the configuration of the seventh embodiment. It is also possible to provide.
[0134]
In the description of each of the first to seventh embodiments, the display method of the imaging device is not particularly described. However, the present invention relates to an imaging device employing any display method such as NTSC, PAL, or SECAM. Is also applicable. Although the number of the solid-state imaging devices 6 of the imaging device is not particularly described, the present invention is effective for any imaging device of a single-chip imaging device, a two-chip imaging device, and a three-chip imaging device. . In addition, the present invention is similarly effective in an imaging device using an imaging tube instead of a solid-state imaging device.
[0135]
Other modifications
The following configuration changes can be added to the above-described first to seventh embodiments.
[0136]
(1) In each of the first to seventh embodiments, the optical shake correction system 1 has been described as a variable apex angle prism (VAP). Means for realizing optical axis correction by being driven in a dynamic manner, for example, means for moving the optical axis by shifting a part or all of a plurality of lenses in a direction orthogonal to the optical axis (for example, Even the means disclosed in Japanese Patent Application No. 63-201622) can be used as the optical shake correction system 1.
[0137]
In addition, as the optical shake correction system 1, a configuration in which the imaging optical system 4, the solid-state imaging device 6, and the like are rotatably supported and driven with respect to the housing of the imaging device to correct the movement (for example, a “video camera”) Development of Technology for Preventing Image Shake "," Technical Report of the Institute of Television Engineers of Japan, Vol. 11, No. 28, pp. 19-24 (1987)).
[0138]
(2) In each of the first to seventh embodiments, the motion detection by the motion detection unit 10 has been described as being performed for each field. However, the present invention is not limited to this. For example, the motion detection is performed once for every two fields. In a field in which no motion detection is performed, a configuration in which the motion of an image is predicted based on the motion detection results obtained up to the previous field and the motion of the image is corrected may be considered. Furthermore, a configuration in which motion detection is performed for each frame is also conceivable.
[0139]
(3) It goes without saying that each of the system control means 16a to 16f can be realized by an electronic circuit or software processing on a microcomputer.
[0140]
(4) Information on the focal length of the imaging optical system 4 obtained by the imaging optical system drive control unit 5 when the conversion unit 162 constituting the system control units 16a to 16f converts the detection vector into a movement angle (angular velocity). However, the present invention is not limited to this.For example, a distance between the imaging device and the subject may be measured by a distance measuring unit using infrared rays or the like, and this distance information may be used instead of the focal length information. It is possible.
[0141]
【The invention's effect】
According to the present invention, there are provided means for detecting the movement of the image pickup apparatus itself, and means for detecting the movement of the image from the image signal, and the shake of the image is determined based on both information obtained from these two detection means. Since the correction is performed, a fixed correction angle of view can be ensured irrespective of the focal length, and an image motion correction device that can realize high-precision motion correction can be realized. In addition to this, there is an effect that the power consumption of the image motion correction device can be reduced.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating an entire image motion compensating apparatus according to a first embodiment of the present invention.
FIG. 2 is a block diagram showing a motion detecting means constituting the apparatus.
FIG. 3 is a block diagram showing a system control means constituting the apparatus.
FIG. 4 is a block diagram illustrating a system control unit included in the image motion compensation device according to the second embodiment of the present invention.
FIG. 5 is a graph illustrating a method for determining a value of a multiplier α for weighting according to a second embodiment of the present invention.
FIG. 6 is a block diagram illustrating a system control unit included in an image motion correction device according to a third embodiment of the present invention.
FIG. 7 is a block diagram illustrating an entire image motion correcting apparatus according to a fourth embodiment of the present invention.
FIG. 8 is a block diagram showing system control means constituting the apparatus.
FIG. 9 is a diagram for explaining the principle of motion correction by image enlargement processing according to the fourth embodiment of the present invention.
FIG. 10 is a block diagram showing an entire image motion compensating device according to a sixth embodiment of the present invention.
FIG. 11 is a block diagram showing a system control means constituting the apparatus.
FIG. 12 is a block diagram showing an entire image motion correcting apparatus according to Embodiment 7 of the present invention.
FIG. 13 is a block diagram showing system control means constituting the apparatus.
[Explanation of symbols]
1. Optical shake correction system
2. Optical shake correction system drive control means
3. Angle detection means
4: Image pickup optical system
5: imaging optical system drive control means
6 ... Solid-state image sensor
7. Analog signal processing means
8 A / D conversion means
9 Digital signal processing means
10. Motion detection means
11… Angular velocity sensor
12 ... Amplifier
13 ... HPF
14 ... LPF
15 A / D conversion means
16a to 16f: System control means (synthesis means)

Claims (8)

An imaging system including a solid-state imaging device, an imaging optical system for forming a subject image on the solid-state imaging device, and an optical shake correction system for optically correcting the movement of a captured image; and the optical shake correction system Optical shake correction system drive control means for driving and controlling the
A sensor for detecting the motion of the imaging device itself and a motion detection unit for detecting the motion of the image based on the image signal obtained from the imaging system are provided in parallel, and a motion detection result of the sensor and the motion detection unit is provided. Signal synthesizing means for synthesizing both signals to form a drive signal,
The signal synthesizing unit converts the information on the motion of the imaging device itself detected by the sensor and the motion detection result by the motion detecting unit into a predetermined multiplier α, 1−α (where 0 ≦ α ≦ 1). The weights are combined to synthesize the signal,
Image motion compensation device, characterized in that said by the drive signal outputted from the signal combining means, for correcting the movement of the image by driving and controlling the optical blur correction system. The image motion compensator according to claim 1,
The image motion compensating apparatus is characterized in that the combining by the signal combining means is a process of adding both signals weighted by predetermined multipliers α and 1−α . The image motion compensator according to claim 1 ,
The multiplier α is determined based on information on one or both of the vibration frequency band of the imaging device itself and the focal length of the imaging optical system . The image motion compensator according to claim 1 ,
The multiplier α is determined according to a ratio between a high frequency band component and a low frequency band component of the vibration of the imaging apparatus itself , or a difference between the two components . The image motion compensator according to claim 3 or 4 ,
An image motion compensator wherein a frequency band component of vibration of the image pickup device itself is extracted from a sensor output or an output of a motion detecting means . In addition to the configuration according to claim 1, storage means for storing an image signal obtained from the imaging system, and storage means drive control means for driving the storage means to control reading of the image signal,
The optical shake correction system is driven and controlled by a drive signal output from the synthesizing unit, and the storage unit drive control unit uses another drive signal calculated based on a motion detection result by the motion detection unit. An image motion compensating device, wherein the motion of an image is compensated by controlling reading of an image signal from the storage means . 7. An electronic zooming device for electronically enlarging an image signal read from said storage means in addition to the configuration according to claim 6 , wherein said optical shake is caused by a drive signal output from said synthesizing means. The drive control of the correction system, and the drive control means of the storage means are calculated based on the motion detection result by the motion detection means only when executing the electronic enlargement processing according to the command from the electronic zoom means. An image motion compensating device for controlling reading of an image signal from the storage means by another drive signal to correct the motion of an image. An imaging system including a solid-state imaging device, an imaging optical system having an adjustable focal length and forming a subject image on the solid-state imaging device, and an optical shake correction system for optically correcting the movement of a captured image And an optical shake correction system drive control means for driving and controlling the optical shake correction system,
A sensor for detecting the motion of the imaging device itself and a motion detection unit for detecting the motion of the image based on the image signal obtained from the imaging system are provided in parallel, and a motion detection result of the sensor and the motion detection unit is provided. Signal synthesizing means for synthesizing both signals to form a drive signal,
The signal synthesizing unit includes a unit that stops outputting a drive signal obtained based on a detection result of the motion detection unit during a period when the focal length is changed by the imaging optical system,
During the period when the focal length is changed by the imaging optical system, only by the drive signal formed from the information on the movement of the imaging device itself detected by the sensor, and during the period when the focal length is fixed, A drive signal formed by synthesizing signals of both the sensor and the motion detection result by the motion detection means drives and controls the optical shake correction system to correct image motion. Image motion compensation device.

Images