JP2008124850A - Image motion correction apparatus, lens unit, and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image motion correction apparatus capable of reducing a deterioration in image quality of a photographed image even when an error is generated in drive of a movable lens, a variable apex angle prism and an imaging device, etc. when motion correction to camera shake, etc. is performed during photography. <P>SOLUTION: The image correction apparatus is composed of: a lens unit 50 having an imaging optical system 1; and an imaging apparatus 60 having a solid-state imaging element 6 and a digital processing means 9 for performing sharpness improvement processing of an image signal, and the image correction apparatus transfers a lens group L32 in the direction perpendicular to an optical axis by information from an angle speed sensor 10 which detects the motion of the imaging apparatus and corrects a motion of a subject image by transferring a position of the subject image formed by the imaging optical system 1, wherein a degree of improvement of sharpness of the image is changed by a control signal for controlling the lens group L32 and a correction state of the motion based on detection information of a transfer position detection means 3 for detecting a transfer position of the lens group L32. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image motion correction apparatus used for camera shake correction or the like of an image pickup apparatus, and more particularly to improve its performance.

  In recent years, digital cameras and video cameras for consumer use (hereinafter referred to as “video movies”) have become smaller, lighter, and optical zoom has become more powerful. Cameras and video movies are one of the home appliances used in daily life. However, the downsizing, weight reduction, high optical zoom magnification, and the spread of these products to consumers who are not familiar with photography have caused problems such as screen instability due to camera shake during photography. It was. Therefore, in order to solve this problem, many digital cameras and video movies equipped with an image motion correction device have been developed and commercialized.

  As an image motion correction device for an image pickup device, a variable apex angle prism is provided in an image pickup optical system, a motion of the image pickup device itself is detected by an acceleration sensor, and drive control of the variable apex angle prism is performed based on a result of the motion detection. There is a method of correcting the motion (see, for example, Patent Document 1).

  In the above method, two glass plates are connected with a bellows made of a special film, and a variable apex prism filled with a liquid with a high refractive index is provided in front of the solid-state image sensor, and acceleration is performed. Based on the movement information of the imaging device obtained from the sensor, the two glass plates of this variable apex prism are tilted in the horizontal and vertical directions, respectively, to bend the optical axis of the incident light and stabilize the movement of the captured image. It is something to be made.

  As another example, for example, there is a proposal concerning an imaging optical system having a variable power optical group or a focus adjustment group as in Patent Document 2 and a correction optical mechanism that decenters or tilts the optical axis of the imaging optical system. Has been made. In this proposal, in an imaging optical system composed of four lens groups as shown in FIG. 2 of Patent Document 2, a part of the lenses is moved up, down, left and right via a slide shaft as shown in FIG. A configuration is disclosed in which the optical axis of the imaging optical system is decentered or tilted by being incorporated in a movable mechanism and moved by an electromagnetic actuator including a coil and a magnet. In this configuration, image disturbance due to camera shake of the imaging apparatus is corrected by moving a slidable portion with a slide shaft (45y and 45p in FIG. 4 of Patent Document 2) with an electromagnetic actuator in accordance with camera shake during shooting. Can do. In Patent Document 3, a relatively small and light lens group constituting a part of the variable magnification optical system is moved in a direction perpendicular to the optical axis so as to correct image shake when the variable magnification optical system vibrates. A variable magnification optical system having a configured image stabilization function is disclosed.

  Patent Document 4 also discloses a method of correcting the movement of an image by moving an image sensor (CCD) in a direction orthogonal to the optical axis.

In the image motion correction device as described above, an angular velocity sensor (gyro sensor) that detects a shake of the imaging device itself is mainly used for detecting the motion. Specifically, the angular velocity of the motion of the imaging device is detected by a sensor. The deflection angle of the image pickup apparatus is obtained by performing an integral operation on the detected value, and the above-described variable apex angle prism and the like are driven and controlled in accordance with the deflection angle.
Japanese Patent Laid-Open No. 62-153816 JP-A-5-66450 Japanese Patent Laid-Open No. 7-128619 JP-A-9-261524

  As described above, it is well known that various methods have been proposed and put to practical use for the image motion correction apparatus, but there are still some problems to be solved. For example, in an optical system in which a part of the imaging system lens is moved in a direction perpendicular to the optical axis to decenter the optical axis and perform vibration isolation, the movable lens is held through a mechanical holding mechanism. In order to move this, the position of the movable portion is always perfectly accurate due to the rattling caused by the presence of a clearance (clearance) between the slide shaft and the portion receiving this shaft, for example, as disclosed in Patent Document 2. There is no guarantee that the movable lens can be moved to. In addition, when driving and controlling a movable lens, there is always a frequency characteristic in the mechanism system and electrical control system that realize it, so it is difficult to achieve complete driving in all frequency ranges to be driven. May cause an error in the driving (movement) of the movable lens, and the desired image motion correction may not be performed. In that case, a phenomenon such as an afterimage occurs in the photographed image, and the resolution of the image is impaired. This is also a problem that may occur when the variable apex angle prism is used or when the movement of the image is corrected by moving the CCD image sensor.

  The present invention has been made to solve the above problems, and even when an error occurs in driving of a movable lens, a variable apex angle prism, an image sensor, etc., when performing motion correction for camera shake or the like during shooting, It is an object of the present invention to provide an image motion correction device, a lens device, and an imaging device that can reduce deterioration in image quality of a captured image.

  In order to solve the above-described problem, an image motion correction device according to a first invention includes a lens group including at least one lens and includes an imaging optical system that forms an image of a subject on an imaging surface. An imaging apparatus comprising: an apparatus; an imaging element that converts a subject image formed on an imaging surface by an imaging optical system into an image signal; and a resolution recovery unit that performs resolution recovery processing on the image signal obtained by the imaging element An image motion correction device comprising: a motion detection means for detecting the motion of the imaging device, and a movement of the imaging device by moving the position of the subject image formed by the imaging optical system A motion correction unit that corrects the motion of the subject image, a control signal generation unit that generates a control signal for controlling the motion correction unit based on an output of the motion detection unit, and a subject image generated by the motion correction unit. Correction state detecting means for detecting the correction state of the image, and based on the control signal generated from the control signal generating means and the correction state of the movement of the subject image detected by the correction state detecting means, by the resolution recovery means It is characterized by changing the characteristics of resolution recovery processing. Thereby, when motion correction for camera shake or the like is performed during shooting, even if an error occurs in driving of the movable lens, the variable apex angle prism, the image sensor, or the like, it is possible to reduce deterioration in the image quality of the shot image.

  In the image motion correction device according to the second invention, the resolution recovery means recovers the resolution of the image by improving the sharpness of the image.

  In the image motion correction device according to the third aspect of the invention, the improvement of the image sharpness makes the improvement degree of the image sharpness proportional to the absolute value of the difference value between the control signal and the motion image correction state.

  In the image motion correction device according to the fourth aspect of the present invention, the improvement of the image sharpness is made to correspond non-linearly to the absolute value of the difference value between the control signal and the motion image correction state of the subject image. .

  In the image motion correction device according to the fifth aspect of the present invention, the improvement in the sharpness of the image is determined by determining the degree of improvement in the sharpness of the image, and the absolute value of the difference value between the control signal and the motion image correction state is equal to or less than a predetermined value If so, fix it.

  In the image motion correction device according to the sixth aspect of the present invention, the sharpness of the image is improved by adjusting the subject image motion correction state or the absolute value of the difference value between the control signal and the subject image motion correction state to a predetermined magnitude. In the above case, the improvement degree of the sharpness is made constant.

  In the image motion correction device according to the seventh aspect of the invention, the sharpness of the image is improved by correcting the movement of the subject image or the absolute value of the difference value between the control signal and the correction state of the movement of the subject image to a predetermined magnitude. In the above case, the sharpness improving process is stopped. Thereby, it is possible to prevent an image from becoming unnatural due to an increase in sharpness more than necessary.

  In the image motion correction device according to the eighth aspect of the invention, the improvement of the sharpness of the image emphasizes at least the horizontal or vertical high-frequency component of the image signal in the image space.

  In the image motion correcting apparatus according to the ninth aspect of the invention, the improvement of the image sharpness is enhanced in a frequency space mapped by orthogonal transformation at least a horizontal or vertical high-frequency component of the image signal. As a result, finer control of sharpness can be performed in units of orthogonal transform blocks.

  In the image motion correction device according to the tenth aspect of the invention, the enhancement of the high frequency component independently changes the degree of enhancement of the horizontal and vertical high frequency components. This makes it possible to improve the sharpness with higher accuracy in accordance with the state of camera shake correction at the time of shooting.

  In the image motion correcting device according to the eleventh aspect, the resolution recovery means recovers the resolution using a point spread function. Thereby, a captured image without camera shake can be restored, and the resolution deteriorated by camera shake can be recovered.

  In the image motion correction device according to the twelfth aspect of the present invention, the resolution recovery is performed by changing the characteristics of the resolution recovery process every time the resolution recovery process is performed once. As a result, the resolution does not change frequently and the user does not feel uncomfortable.

  In the image motion correction device according to the thirteenth aspect of the present invention, the motion correction means is driven relative to the imaging optical system to decenter the optical axis of the imaging optical system, thereby correcting the motion of the subject image. .

  In the image motion correction device according to the fourteenth aspect of the invention, the correction of the movement of the subject image is rotationally driven around two axes orthogonal to the optical axis of the imaging optical system.

  In the image motion correction device according to the fifteenth aspect of the invention, the movement of the subject image is corrected by individually driving at least one lens in a direction orthogonal to the optical axis of the imaging optical system. Make the shaft eccentric.

  In the image motion correction device according to the sixteenth aspect of the present invention, the motion correction means corrects the motion of the subject image by displacing the image sensor in a direction orthogonal to the optical axis. Thereby, it is not necessary to incorporate a correction lens in the lens itself, and the lens barrel can be reduced in size.

  In the image motion correction device according to the seventeenth aspect of the present invention, the correction state detection means detects the motion correction state of the subject image based on the displacement amount of the moving part constituting the motion correction means.

  In the image motion correction device according to an eighteenth aspect of the present invention, the moving part is a lens group including at least one lens.

  In the image motion correcting device according to the nineteenth invention, the moving part is an image sensor.

  In the image motion correction apparatus according to the twentieth invention, the control signal generation means limits the signal width of the control signal to a predetermined value based on the detection result of the motion correction state detection means and the amount of motion correction of the motion correction means. Apply clip processing. Thereby, it is possible to prevent an image from becoming unnatural due to an increase in sharpness more than necessary.

  The image motion correction device according to the twenty-first aspect performs clip processing when the control signal indicates a correction amount exceeding the range of motion correction of the motion correction means.

  An image motion correction apparatus according to a twenty-second invention detects a motion correction state of a subject image based on a control signal before performing clip processing. Thereby, even when a camera shake exceeding the correction range occurs, the necessary sharpness can be improved.

  In an image motion correction apparatus according to a twenty-third aspect of the present invention, the motion detection means is an angular velocity sensor that detects an angular velocity of movement of the imaging device itself.

  In an image motion correction apparatus according to a twenty-fourth aspect of the present invention, the motion detection means is motion vector detection means for detecting a motion vector of the image from the captured image.

  A lens apparatus according to a twenty-fifth aspect of the present invention is composed of a lens group including at least one lens, has an imaging optical system that forms an image of the subject on the imaging surface, and the subject imaged on the imaging surface by the imaging optical system A lens device that constitutes an image motion correction device in combination with an imaging device that includes an imaging device that converts an image into an image signal and a resolution recovery means that performs resolution recovery processing on the image signal obtained by the imaging device. , Motion detection means for detecting the motion of the imaging device, and motion correction means for correcting the motion of the subject image caused by the motion of the imaging device by moving the position of the subject image formed by the imaging optical system A control signal generating means for generating a control signal for controlling the motion correcting means based on the output of the motion detecting means, and a correction state for detecting a correction state of the movement of the subject image by the motion correcting means And changing the characteristics of the resolution recovery processing by the resolution recovery means based on the control signal generated from the control signal generation means and the correction state of the movement of the subject image detected by the correction state detection means. It is characterized by. Thereby, even if the imaging apparatus does not have a camera shake correction function, the camera shake correction function can be realized by replacing the lens device. In addition, even when an error occurs in the driving of the movable lens or the variable apex angle prism during motion correction for camera shake or the like during shooting, it is possible to reduce deterioration in the image quality of the shot image.

  An image pickup apparatus according to a twenty-sixth aspect of the present invention comprises an image motion correction apparatus together with a lens apparatus having an image pickup optical system that is formed of a lens group including at least one lens and forms an image of a subject on an image pickup surface. An imaging apparatus comprising: an imaging element that converts a subject image formed on an imaging surface into an image signal; and a resolution recovery unit that performs resolution recovery processing on the image signal obtained by the imaging element. Motion detection means for detecting motion, motion correction means for correcting the movement of the subject image caused by the motion of the imaging device by moving the position of the subject image formed by the imaging optical system, and motion detection Control signal generating means for generating a control signal for controlling the motion correcting means based on the output of the means, and a correction state detecting means for detecting the correction state of the movement of the subject image by the motion correcting means; And a characteristic of resolution recovery processing by the resolution recovery means is changed based on the control signal generated from the control signal generation means and the correction state of the motion of the subject image detected by the correction state detection means. . Thereby, even if the lens apparatus does not have a camera shake correction function, the camera shake correction function can be realized by the imaging apparatus itself. In addition, even when an error occurs in driving of the image sensor when performing motion correction for camera shake or the like during shooting, it is possible to reduce deterioration in the image quality of the shot image.

  According to the present invention, when motion correction for camera shake or the like is performed during shooting, even if an error occurs in driving of a movable lens, a variable apex angle prism, an image sensor, or the like, image motion that can reduce deterioration in image quality of a shot image A correction device, a lens device, and an imaging device can be provided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram of an image motion correction apparatus according to Embodiment 1 of the present invention. The image motion correction device includes a lens device 50 and an imaging device 60. The lens device 50 includes an imaging optical system 1, a lens group L32 drive control unit 2, a moving position detection unit 3, an imaging optical system drive control unit 4, an A / D conversion unit 5, an angular velocity sensor 10, an HPF 11, an LPF 12, an amplifier 13, An A / D converter 14, a microcomputer (hereinafter abbreviated as “microcomputer”) 15, a D / A converter 16, and an A / D converter 18 are provided. The imaging device 60 includes a solid-state imaging device 6, an analog signal processing unit 7, an A / D conversion unit 8, a digital signal processing unit 9, and a solid-state imaging device drive unit 17. In the figure, an imaging optical system 1 is an imaging lens including four lens groups L1, L2, L3, and L4. The lens group L2 performs zooming by moving in the optical axis direction, and the lens group L4 is light. Focusing is performed by moving in the axial direction. The lens unit L3 includes two lens units L31 and L32 arranged on the image plane side with respect to the lens unit L2, and the lens unit L32 which is a part of the lens unit L3 moves in a direction orthogonal to the optical axis. Thus, the optical axis is decentered to correct the motion of the image. At this time, the displacement amount (movement distance) of the lens unit L32 from the optical axis center and the angle at which the optical axis is decentered are in a proportional relationship. Therefore, in order to decenter the optical axis by a desired angle, the lens unit L32 may be moved from the optical axis center by an amount necessary for that purpose.

  The lens group L32 drive control means 2 is means for driving and controlling the lens group L32 which is a shake correction lens, and moves the lens group L32 up, down, left and right within a plane orthogonal to the optical axis of the imaging optical system 1. . The moving position detection means 3 is a means for detecting and outputting the position of the lens group L32, and forms a feedback control loop for driving and controlling the lens group L32 together with the lens group L32 drive control means 2.

  The imaging optical system drive control means 4 is means for driving and controlling the lens groups L2 and L4 in the imaging optical system 1, performing zooming and focusing operations, and outputting focal length information of the imaging optical system 1. The A / D conversion means 5 is means for converting the focal length information of the imaging optical system 1 output from the imaging optical system drive control means 4 into a digital signal.

  The solid-state imaging device 6 is an imaging device that converts an image incident through the imaging optical system 1 into an electrical signal, and the analog signal processing means 7 performs analog signal processing such as gamma processing on the image signal obtained by the solid-state imaging device 6. The A / D conversion means 8 is means for converting an analog signal into a digital signal. The image signal converted into a digital signal by the A / D conversion means 8 is subjected to digital signal processing such as separation of luminance signal and color difference signal, noise removal, sharpness improvement processing and the like by the digital signal processing means 9.

  The angular velocity sensor 10 is an angular velocity sensor for detecting the movement of the imaging apparatus itself, and outputs angular velocity signals in both positive and negative directions depending on the direction of movement of the imaging apparatus with reference to the output when the imaging apparatus is stationary. Two angular velocity sensors 10 are required to detect movement in two directions of yawing and pitching, but FIG. 1 shows only one direction. The HPF 11 is a high-pass filter for removing, for example, a DC drift component in the unnecessary band component included in the output of the angular velocity sensor 10, and the LPF 12 is, for example, the resonance frequency component of the sensor in the unnecessary band component included in the output of the angular velocity sensor 10. The low-pass filter for removing noise components and the amplifier 13 are amplifiers for adjusting the signal level of the output of the angular velocity sensor 10, and the A / D conversion means 14 converts the output of the amplifier 13 into a digital signal. Means.

  The microcomputer 15 performs filtering, integration processing, phase compensation, movement amount calculation, output signal clipping processing, and the like on the output of the angular velocity sensor 10 taken in via the A / D conversion means 14, and a lens group necessary for motion correction An L32 drive control amount (hereinafter referred to as “control signal”) is obtained and sent to the lens group L32 drive control means 2 via the D / A conversion means 16. When the D / A conversion means 16 receives a signal from the microcomputer 15, it converts it into an analog signal almost simultaneously and sends it to the lens group L32 drive control means 2.

  The lens group L32 drive control means 2 corrects the movement of the image by driving the lens group L32 based on the control signal. Finally, the solid-state image sensor driving means 17 is a means for controlling the transfer of charges converted from the subject in the solid-state image sensor 6.

  Further, the output of the movement position detection means 3 is converted into a digital signal by the A / D conversion means 18 and taken into the microcomputer 15. The microcomputer 15 controls the sharpness improving process in the digital signal processing means 9 based on the output of the moving position detecting means 3 and the control signal.

  FIG. 2 shows an example of a shake correction optical mechanism for driving and controlling the lens group L32 in the imaging optical system 1 in a direction orthogonal to the optical axis. As shown in FIG. 2, the correction optical mechanism includes a lens group L32 (shake correction lens) 2001, main axes (slide shafts) 2002 and 2003 for moving the movable part in the pitch direction and the yaw direction, a detent 2004, and a magnet. 2005, 2006, yokes 2007, 2008, coils 2009, 2010, semiconductor position detecting elements (PSD) 2011, 2012, infrared light emitting diodes (LED) 2013, 2014. The magnet 2005, the yoke 2007, and the coil 2009 constitute an electromagnetic actuator that drives the movable portion in the pitch direction. Similarly, the magnet 2006, the yoke 2008, and the coil 2010 constitute an electromagnetic actuator in the yaw direction. Further, the semiconductor position detecting element 2011 and the infrared light emitting diode 2013 serve to detect the position of the movable part in the pitch direction, and this corresponds to the moving position detecting means 3 shown in FIG. Similarly, the semiconductor position detecting element 2012 and the infrared light emitting diode 2014 constitute the moving position detecting means 3 in the yaw direction.

  FIG. 3 is a block diagram showing an example of the configuration of the digital signal processing means 9. The image signal digitized through the A / D conversion unit 8 is separated into a luminance signal and a color difference signal by the luminance / color difference signal separation unit 3001, and a noise component is removed by the noise removal unit 3002. Further, the sharpness improving means 3003 emphasizes horizontal and vertical high-frequency components in the image signal, thereby improving the sharpness of the image.

  FIG. 4 is a block diagram showing an example of the configuration of the sharpness improving means 3003. In FIG. 4, 1H delay devices 4001 and 4002 are delay means for delaying a signal for one horizontal scanning period of an input image signal by one horizontal scanning period. The signals delayed by the two delay means are: Each of the weights 1/2, 1, and -1/2 shown in the figure is weighted and added by adders 4003 and 4004. The signals delayed by the one-pixel delay units 4005, 4006, 4007, and 4008 are weighted from K [0] to K [n] and summed by the adder 4009. The adder 4010 adds the output of the adder 4004. The signal added by the adder 4010 is a high-frequency component of the image signal, and this is added to the original image signal by the adder 4012, whereby the high-frequency component of the image signal is improved and the sharpness of the image is increased. improves. The adder 4011 is a means for performing weighting by multiplying the high-frequency component of the image signal by a number greater than or equal to 0 (expressed as α in this embodiment), and the value of α is the degree of sharpness improvement of the image. It is controlled by the microcomputer 15 to adjust.

  FIG. 5 is an example of a flowchart of a processing program stored in the microcomputer 15. When a camera shake correction switch (not shown) is turned on, a series of processes shown in FIG. 5 is started. Although not shown in FIG. 5, one processing loop is interrupted at a constant cycle by a timer built in the microcomputer 15, for example, and the loop processing is executed for each interrupt (for example, every 1 msec). Shall.

  First, when the camera shake correction switch is turned on, in step 101, setting values (HPF) used for filtering (high-pass filter (HPF) and low-pass filter (LPF)), integration processing, phase compensation, movement amount calculation, and clip processing are used. And cut-off frequency, integration constant, phase compensation band, movement amount calculation coefficient of the lens unit L32, and clip value) are set to initial values.

  When interrupted by the timer, first, in step 102, the output of the moving position detecting means 3 converted into a digital signal by the A / D converting means 18, that is, the position of the lens group L32 is taken into the microcomputer 15. At this time, the cycle in which the A / D conversion unit 18 converts the output of the movement position detection unit 3 into a digital signal is synchronized with one processing cycle of the microcomputer 15.

  Next, in step 103, a coefficient that determines the degree of sharpness improvement of the image in the sharpness improvement means 3003 based on the position of the lens group L32 captured in step 102 and the value stored in step 111 described later. α is calculated.

  Next, the output of the angular velocity sensor 10 converted into a digital signal by the A / D conversion means 14 in step 104, that is, the angular velocity of the movement of the imaging device is taken into the microcomputer 15.

Subsequently, step 105 is a step of performing band limitation on the output of the angular velocity sensor 10 taken into the microcomputer 15 by a high-pass filter (HPF). The HPF in this step is for removing unnecessary signal components of low frequency such as temperature drift included in the output of the angular velocity sensor 10, and the transfer function is, for example, (1-Z ^-1 ) / (1-a The characteristic of the first-order filter expressed by Z ⁻¹ ) is obtained, and the filter coefficient a (0 <a <1) is determined by determining the filter coefficient a (0 <a <1) according to the cut-off frequency set in step 101. Passband (cut-off frequency) is determined.

Step 106 is a step of performing band limitation on the output of the angular velocity sensor 10 after the low-frequency removal in Step 105 by a low-pass filter (LPF). The LPF in this step is for removing unnecessary signal components such as high-frequency noise included in the output of the angular velocity sensor 10. For example, the transfer function is (1 + Z ⁻¹ ) / (1−b · Z ⁻¹ ). The filter coefficient b (0 <b <1) is determined in accordance with the cutoff frequency set in step 101, so that the filter passband (cut) OFF frequency) is determined.

Step 107 is a step of performing an integration process on the output of the angular velocity sensor 10 after filtering in Step 106 to obtain an angle from the angular velocity. In the processing at this step, for example, the transfer function is assumed to have a filter characteristic of 1 / (1-K · Z ⁻¹ ). K is an integration constant, and 0 <K <1.

Step 108 is a step for improving the phase characteristics of the signal that has undergone Step 107. This is because the phase change of the signal by the various filtering processes by the HPF 11, LPF 12, Step 105, and Step 106 with respect to the angular velocity sensor 10 itself and the angular velocity sensor output. For example, the transfer function has a filter characteristic of (c · d · Z ⁻¹ ) / (eg · Z ⁻¹ ), and the coefficients c, d, e, g Determines the phase compensation band.

  In step 109, the lens group L32 for canceling the movement of the imaging device is obtained by multiplying the angle information of the movement of the imaging device obtained from the output (angular velocity) of the angular velocity sensor 10 in step 107 by a predetermined coefficient. This is a step of converting into position information. This is because the eccentric angle of the optical axis by the lens unit L32 is proportional to the moving distance of the lens unit L32 from the optical axis, and therefore the lens unit L32 moves from the optical axis center to decenter the optical axis as much as necessary for motion correction. In order to obtain the distance, the eccentric angle of the optical axis, that is, the angle information of the movement of the imaging apparatus may be multiplied by a predetermined coefficient (movement amount calculation coefficient). Then, the obtained position information is transmitted to the lens group L32 drive control means 2 via the D / A conversion means 16, and the lens group L32 moves to a position indicated by this position information, so that the movement of the imaging apparatus is changed. Canceled.

  Step 110 is a clip in which the control signal sent from the microcomputer 15 to the lens group L32 drive control means 2 does not indicate a correction amount exceeding the motion correction range by the lens group L32, that is, the maximum movement range of the lens group L32. This is a step of limiting the signal width of the control signal with a value (clipping process).

  Step 111 is a step of recording the value of the control signal after clip processing in the microcomputer, and this recorded value is a coefficient α that determines the degree of sharpness improvement of the image in the sharpness improvement means 3003 in the above-mentioned step 103. It is used when calculating Further, the data after the clip processing is converted into an analog signal by the D / A conversion means 16 and sent to the lens group L32 drive control means 2.

  The operation of the image motion correction apparatus of the present embodiment configured as described above will be described below based on a processing program stored in the microcomputer 15. A series of operations such as angular velocity detection by the angular velocity sensor and drive control of the lens unit L32 are performed in both the horizontal and vertical directions, but the contents are substantially the same in both the horizontal and vertical directions, so the description is simplified. Therefore, processing for one direction will be described without distinguishing between the horizontal and vertical directions.

  When camera shake correction is activated according to an instruction from the operator of the imaging apparatus, a series of processes shown in FIG. 5 is started.

  First, the current position information of the lens group L32 is detected. This is based on the control signal calculated in steps 104 to 110 one cycle before the series of processing shown in FIG. This corresponds to the position information after moving. FIG. 6 shows an example of the output of the control signal calculated in step 110 and the detection timing of the position information of the lens unit L32.

  Next, in step 103, the value of the control signal stored in step 111 before one cycle is compared with the position information of the lens unit L32 detected in step 102. At this time, if the lens unit L32 has moved to the position indicated by the control signal of the previous cycle without any error, the current position information of the lens unit L32 and the value of the control signal of the previous cycle are completely Match. Therefore, the difference between them is zero. However, in reality, the lens group L32 is held through a mechanical holding mechanism and is moved by this mechanism system. Therefore, the gap between the slide shaft shown in FIG. There is no guarantee that the lens unit L32 can always be moved to a completely accurate position due to rattling due to the presence of clearance. In addition, when the lens group L32 is driven and controlled, there is always a frequency characteristic in the mechanism system and the electric control system that realize this, so that it is difficult to realize complete driving in all frequency ranges to be driven. Depending on the case, an error may occur in the driving (movement) of the lens unit L32, and the lens group L32 may not be moved to the position according to the control signal.

  In such a case, when the value of the control signal is compared with the position information of the lens unit L32 detected in step 102, the two do not coincide with each other, and some difference value is detected. In other words, the effect of the image motion correction cannot be obtained as intended, and as a result, the image deteriorates due to the motion of the imaging device. Specifically, since the exposure is performed while the imaging device is moving, a phenomenon such as an afterimage occurs in the photographed image, and the resolution of the image due to the afterimage decreases approximately in proportion to the amount of movement of the imaging device. become.

  Therefore, in step 103, the value of the control signal is compared with the position information of the lens unit L32 detected in step 102, and the degree of sharpness improvement of the image in the sharpness improvement means 3003 is controlled based on the difference value. Image degradation caused by failure to perform motion correction as intended is improved. This method will be described with reference to FIG.

  FIG. 7 shows an example of the relationship between the absolute value of the difference between the value of the control signal and the position information of the lens unit L32, and the coefficient α that determines the degree of sharpness improvement of the image in the sharpness improvement means 3003. . As shown in FIG. 7, when α is proportional to the absolute value of the difference between the control signal value and the position information of the lens unit L32, for example, the absolute value of the difference between the control signal value and the positional information of the lens unit L32 is shown. When the value is zero, that is, when the motion correction of the image by the lens unit L32 operates as intended, α = 0 and the sharpness improving unit 3003 does not perform the sharpness improving process on the image. Conversely, as the absolute value of the difference between the control signal value and the position information of the lens unit L32 increases, α also increases. As a result, the gain of the high frequency component added to the image signal in the sharpness improving means 3003 increases. The image becomes larger and the sharpness of the image is improved more strongly. FIG. 7 shows an example in which α, the value of the control signal, and the absolute value of the difference value of the position information of the lens unit L32 are proportional to each other. For example, as shown in FIGS. 8 and 9, α, the value of the control signal, and the lens If the absolute value of the difference value of the position information of the group L32 is made to correspond non-linearly, or if the absolute value of the difference value of the control signal value and the position information of the lens group L32 is equal to or less than a certain value, α is a fixed value. A method is also conceivable.

  As described above, in the present embodiment, the microcomputer 15 determines the coefficient α that determines the degree of sharpness improvement of the image in the sharpness improvement means 3003 based on the value of the control signal and the position information of the lens group L32. In addition, by improving the sharpness of the captured image, it is possible to reduce the degradation of the image quality of the captured image even when an error occurs in the driving of the movable lens for correcting the motion of the image such as the lens group L32. It is.

  In the above description, when the camera shake amount of the imaging apparatus exceeds the correction range, the value of the control signal is limited by the clipping process in step 110 in order to limit the movement of the lens group L32. Even in this case, since the coefficient α is determined in accordance with the difference between the control signal value limited by the clipping process and the actual movement amount of the lens unit L32, the intended sharpness improvement effect may not be expected. . Therefore, even when the correction range is exceeded, the sharpness improving means 3003 may be given a difference between the control signal value (control target value) before the clipping process and the actual movement amount as a correction error. In this way, even when a camera shake exceeding the correction range occurs, the necessary sharpness improvement coefficient α can be calculated. Further, when the correction amount itself or the correction error becomes larger than a predetermined amount, the value of the coefficient α may be fixed, or the sharpness improvement itself may be stopped depending on the case. Thereby, it is possible to prevent an image from becoming unnatural due to an increase in sharpness more than necessary.

  In the above description, the coefficient α that determines the degree of sharpness improvement is a coefficient that is commonly applied to the high-frequency components in the horizontal direction and the vertical direction, and acts equally in both directions. However, the coefficient α may be set independently for each of the horizontal direction and the vertical direction. This makes it possible to improve the sharpness with higher accuracy in accordance with the state of camera shake correction at the time of shooting.

FIG. 10 is a block diagram of sharpness improving means 3003 in which the coefficient α can be set independently for each of the horizontal and vertical directions. The output of the adder 4004 is multiplied by a coefficient α ₁ by a multiplier 4013 and input to the adder 4010. Similarly, the output of the adder 4009 is multiplied by the coefficient α ₂ by the multiplier 4014 and input to the adder 4010. Here, the outputs of the adder 4004 and the adder 4009 are high-frequency components in the vertical and horizontal directions of the image signal, respectively, and the respective components are multiplied independently by the coefficient α ₁ and the coefficient α ₂ . That is, for the camera shake correction of the imaging apparatus, the sharpness improvement coefficients α ₁ and α ₂ are determined from the difference value between the control signal value and the position information of the lens group L32 in each of the horizontal direction and the vertical direction, and the multiplier 4013 and the multiplier 4014. Next, these are added in the adder 4010, and finally the addition result of the adder 4010 and the original image signal are added in the adder 4012. As a result, it is possible to improve the sharpness with higher accuracy in accordance with the respective correction states in the horizontal direction and the vertical direction. For example, when the correction in the horizontal direction is complete but a correction error remains in the vertical direction, α ₁ = 0 and only the coefficient of α ₂ may be determined by a method similar to α. On the other hand, if the correction in the vertical direction is complete but a correction error remains in the horizontal direction, if α ₂ = 0 and only the coefficient of α ₁ is determined by the same method as α, Good. If the same amount of correction error remains in the horizontal direction and the vertical direction, α ₁ = α ₂ = α, which is the same as the example of FIG. In most cases, α1 and α2 take different values which are not zero, and the optimum sharpness is improved in the horizontal and vertical directions.

(Embodiment 2)
An image motion correction apparatus according to Embodiment 2 of the present invention will be described with reference to FIGS. Although the method for improving the image sharpness has been described in the first embodiment as means for recovering the resolution of an image that cannot be corrected by the motion correction in the optical system and the resolution has deteriorated, The camera shake is represented by a point spread function, and an image without camera shake is restored based on the point spread function. As a result, the resolution of the captured image that has deteriorated due to camera shake can be recovered. Other configurations and processes are the same as those in the first embodiment.

  FIG. 11 is a block diagram of the image motion correction apparatus according to the present embodiment. The image motion correction device according to the present embodiment includes a lens device 50 and an imaging device 61. Only the digital signal processing means 29 of the imaging device 61 is different from that in FIG. 1 and the other blocks are the same as those in FIG.

  FIG. 12 is a block diagram showing a configuration example of the digital signal processing means 29. The image signal digitized by the A / D conversion unit 8 is separated into a luminance signal and a color difference signal by a luminance / color difference signal separation unit 3001, and a noise component is removed by a noise removal unit 3002. Next, the image restoration unit 3004 restores an image without camera shake by using a point spread function that expresses camera shake of the imaging apparatus.

  FIG. 13 is a block diagram illustrating a configuration example of the image restoration unit 3004. The image restoration means 3004 is a Fourier transform means 5001 that Fourier transforms the image signal input from the noise removal means 3002, a point spread function generation means 5002 that generates a point spread function according to an instruction from the microcomputer 15, and a point spread function that is Fourier transformed. A Fourier transform unit 5003 for transforming, a division unit 5004 for dividing the output of the Fourier transform unit 5001 by an output of the Fourier transform unit 5003, and a Fourier inverse transform unit 5005 for performing Fourier inverse transform on the division result of the division unit 5004 are provided.

  FIG. 14 is a flowchart showing an operation procedure of the image restoration unit 3004 described above. First, in step 601, a point spread function corresponding to the camera shake amount of the image pickup apparatus that cannot be corrected by the optical camera shake correction is generated. In the next step 602, the point spread function is Fourier transformed by the Fourier transform means 5003. In step 603, the reciprocal of the Fourier transform result is obtained to calculate the frequency characteristic of the restoration inverse filter. In step 604, the image obtained after the camera shake correction is Fourier transformed by the Fourier transform unit 5001, and in step 605, the restoration inverse filter calculated in step 603 is multiplied by the result of the Fourier transform of the camera shake corrected image. That is, the output of the Fourier transform unit 5001 is divided by the output of the Fourier transform unit 5003. Finally, in step 606, the inverse Fourier transform is performed on the output of the dividing unit 5004 by the Fourier inverse transform unit 5005 to obtain a restored image without camera shake.

  In this image restoration unit 3004, an image g (x, y) in which a camera shake expressed by a point spread function h (x, y) is generated with respect to a captured image f (x, y) of the imaging device assumed to have no camera shake. ) Is expressed by the following convolution integral.

g (x, y) = ∬f (ε, η) × h (x−ε, y−η) dεdη + n (x, y)
here,
f (x, y): Image without camera shake g (x, y): Image with camera shake h (x, y): Point spread function n (x, y): Noise x, y: Spatial coordinates. When the Fourier transform is applied to both sides of the above equation and mapped to the frequency space, the convolution integral is replaced with the product.
G (u, v) = F (u, v) × H (u, v) + N (u, v)
It becomes.

F (u, v): Two-dimensional Fourier transform of an image without camera shake G (u, v): Two-dimensional Fourier transform of an image with camera shake H (u, v): Two-dimensional Fourier transform of a point spread function ( Spatial frequency transfer function of camera shake)
N (u, v): Two-dimensional Fourier transform of noise u, v: Spatial frequency. Therefore, the Fourier transform F (u, v) of the image without camera shake is
F (u, v) = G (u, v) / H (u, v) −N (u, v) / H (u, v)
It becomes. By performing inverse Fourier transform on this F (u, v), a restored image can be obtained in which the resolution deteriorated due to camera shake is recovered. However, if the value of H (u, v) becomes very small at a certain frequency, the value of N (u, v) / H (u, v) is much larger than the original signal near this frequency. It will be buried in noise. Therefore, the influence of this noise can be eliminated by performing inverse Fourier transform using only a relatively low frequency component w ₀ that is considered to have a large signal-to-noise ratio (SN ratio). In this case, the transfer function M (u, v) of the inverse filter is as follows.

  This image restoration technique is a basic technique for restoration of a deteriorated image using a point spread function, and a good restored image can be obtained as long as there is no error in the point spread function and the image has linearity.

  15 and 16 are diagrams for explaining the principle of image restoration. For the sake of simplicity, it is assumed that there is no correction in the optical system, and the camera shake of the image pickup device is uniformly generated in the x-axis direction by a = VT as shown in FIG. Here, V is the moving speed in the X direction due to camera shake, and T is the exposure time. The point spread function h (x) in this case is as shown in FIG. FIG. 16B shows the result of Fourier transform of this, which is a hand-shake spatial frequency transfer function H (u). Then, taking the inverse of this, an inverse filter transfer function M (u) as shown in FIG. As shown in FIG. 16C, the inverse filter transfer function M (u) has an infinite frequency. Therefore, in practice, measures such as the above-described frequency limitation are taken so as not to become infinite, but detailed description thereof is omitted here.

FIG. 17 is a diagram illustrating a relationship between the magnitude of camera shake of the imaging apparatus and the frequency characteristic of the transfer function H (u). The photographed image is sampled two-dimensionally based on the pixel structure of the solid-state imaging device 6, and FIG. 17 (a) shows the image sampling structure. Here, the movement amount of the image on the imaging surface of the solid-state image sensor 6 by the camera shake of the imaging device can be represented by what pixels of the pixel spacing x _0. If the amount of movement in the x direction is β ₁ times the pixel interval x ₀ as shown in FIG. 17A, the frequency of the transfer function H (u) is shown in FIG. 17B depending on the value of β ₁ . Characteristics change. Since the inverse filter for restoring the image is represented by the inverse of the transfer function H (u), the characteristics of the inverse filter for correcting the camera shake amount of the imaging apparatus, that depends on the value of the beta ₁ Become. Therefore, no captured images shake when optimally choose the value of the beta ₁ can be restored. Similarly, if the amount of movement in the y direction is β ₂ times the pixel interval y ₀ , an inverse filter in the y direction can be obtained.

  FIG. 18 is an example of a flowchart of a processing program stored in the microcomputer 15. The processing of the microcomputer in the present embodiment is different in only step 203 for determining the value of the coefficient β of the inverse filter, and the other processing is the same as in the first embodiment. The same reference numerals are used and description thereof is omitted. The processing shown in FIG. 18 is also executed for each interruption by a timer built in the microcomputer 15. When a camera shake correction switch (not shown) is turned on, a series of processes shown in FIG. 18 is started.

  When a camera shake correction switch (not shown) is turned on, an initial value necessary for the subsequent processing is set in step 101. When the timer is interrupted, the output of the moving position detecting means 3 converted into a digital signal by the A / D converting means 18 in step 102, that is, the position of the lens group L32 is taken into the microcomputer 15.

  Next, in step 203, on the basis of the position of the lens group L32 captured in step 102 and the value stored in step 111, an inverse process for restoring degradation in image resolution due to camera shake remaining after correction is performed. The filter coefficient β is determined. From step 104 to step 111, the same processing as that in the first embodiment is executed, and thus detailed description thereof is omitted.

  As described above, in the present embodiment, the coefficient β of the point spread function corresponding to the camera shake correction state of the imaging apparatus is determined based on the difference between the position of the lens unit L32 and its control target value. By applying an inverse filter of the spread function to the hand shake image, the resolution reduced by the hand shake can be recovered.

(Embodiment 3)
An image motion correction apparatus according to Embodiment 3 of the present invention will be described with reference to FIGS. The image motion correction apparatus according to the present embodiment performs image motion correction not by moving the lens group L32 but by moving the solid-state imaging device 6 in a direction perpendicular to the optical axis. The configuration and processing are the same as those in the first embodiment.

  FIG. 19 is a block diagram of the image motion correction apparatus according to the present embodiment. The image motion correction apparatus according to the present embodiment includes a lens device 51 and an imaging device 62. In the present embodiment, the lens device 51 includes the imaging optical system 1 and the imaging optical system drive control means 4. The imaging device 62 includes an A / D conversion unit 5, a solid-state imaging device 6, an analog signal processing unit 7, an A / D conversion unit 8, a digital signal processing unit 9, an angular velocity sensor 10, an HPF 11, an LPF 12, an amplifier 13, and an A. / D conversion means 14, microcomputer 15, solid-state image sensor drive means 17, solid-state image sensor drive control means 32, movement position detection means 33, D / A conversion means 36, and A / D conversion means 38. In FIG. 19, only the blocks related to motion correction are different from those in FIG. 1, and the other blocks are the same as those in FIG. The solid-state image sensor drive control means 32 is means for driving and controlling the solid-state image sensor 6, and is means for moving the solid-state image sensor 6 up and down and left and right within a plane orthogonal to the optical axis of the imaging optical system 1. is there. The moving position detection means 33 is a means for detecting and outputting the position of the solid-state image sensor 6 and forms a feedback control loop for driving and controlling the solid-state image sensor 6 together with the solid-state image sensor drive control means 32.

  The microcomputer 15 obtains the drive control amount (control signal) of the solid-state image sensor 6 necessary for motion correction from the output of the angular velocity sensor 10 and sends this to the solid-state image sensor drive control means 32 via the D / A conversion means 36. When the D / A converter 36 receives a signal from the microcomputer 15, it converts it into an analog signal almost simultaneously and sends it to the solid-state image sensor drive controller 32. The solid-state image sensor drive controller 32 corrects the motion of the image by driving the solid-state image sensor 6 based on the control signal. That is, the movement amounts of the solid-state imaging device 6 in the yaw direction and the pitch direction necessary for correcting the shake are calculated from the shake angle of the image obtained from the angular velocity sensor 10. The required amount of movement is determined by the size of the solid-state imaging device 6 used in the imaging apparatus, the focal length of the imaging lens, and the like. Then, based on the calculation result, the solid-state imaging device 6 is moved so that an image is always taken at the same position on the imaging surface of the solid-state imaging device 6. Thereby, the shake of the imaging apparatus is corrected.

  Here, the position detection and driving method of the solid-state imaging device 6 is substantially the same as the position detection and driving method of the lens unit L32 of the first embodiment. That is, the solid-state imaging device 6 is attached to a moving mechanism that can move in the yaw direction and the pitch direction, and the moving mechanism can be independently operated in the pitch direction and the yaw direction by an actuator constituted by an electromagnetic coil or a piezoelectric element. It can be driven. The position detection also includes a semiconductor position detection element (PSD) and a light emitting diode (LED).

  The output of the movement position detection means 33 is converted into a digital signal by the A / D conversion means 38 and taken into the microcomputer 15. The microcomputer 15 controls the sharpness improving process in the digital signal processing means 9 based on the output of the moving position detecting means 33 and the control signal. Since the moving mechanism of the solid-state imaging device 6 is also configured by a mechanical holding mechanism similar to the moving mechanism of the lens group L32, the moving mechanism of the solid-state imaging device 6 is always completely controlled depending on the optical axis direction and the rotation direction on the imaging plane. Since there is no guarantee that the solid-state imaging device 6 can be moved to an accurate position, sharpness improvement processing is required as in the first embodiment.

  FIG. 20 is an example of a flowchart of a processing program stored in the microcomputer 15. The same processing contents as those in the first embodiment are denoted by the same reference numerals as those in FIG. The processing shown in FIG. 20 is also executed for each interruption by the timer built in the microcomputer 15. When a camera shake correction switch (not shown) is turned on, a series of processes shown in FIG. 20 is started.

  When a camera shake correction switch (not shown) is turned on, in step 301, filtering (high-pass filter (HPF) and low-pass filter (LPF)), integration processing, phase compensation, and movement amount calculation of the solid-state image sensor 6 are calculated. , Set values used in clip processing (HPF and LPF cutoff frequencies, integration constants, phase compensation bands, movement amount calculation coefficients of the solid-state imaging device 6, and clip values) are set to initial values.

  When the timer is interrupted, the output of the movement position detection means 33 converted into a digital signal by the A / D conversion means 38 in step 302, that is, the position of the solid-state imaging device 6 is taken into the microcomputer 15.

  Next, in step 303, the degree of sharpness improvement of the image in the sharpness improvement means 3003 is determined based on the position of the solid-state imaging device 6 captured in step 302 and the value stored in step 311 described later. The coefficient α is calculated.

  Next, in step 104 to step 108, the motion information of the imaging device is fetched from the angular velocity sensor 10, and necessary processing is performed on it. Step 309 is a solid for calculating the amount of movement in the horizontal and vertical directions from the angle information of the movement of the imaging device obtained from the output (angular velocity) of the angular velocity sensor 10 in Step 107 and canceling the movement of the imaging device. This is a step of converting into position information of the image sensor 6. Since the moving distance of the image on the imaging surface of the solid-state imaging device 6 is proportional to the focal length × tan θ, where θ is the angle of movement of the imaging device, the moving distance of the solid-state imaging device 6 necessary for motion correction is that of the imaging device. It can be easily calculated from the angle information of the movement. The obtained position information is transmitted to the solid-state image sensor drive control means 32 via the D / A conversion means 36, and the solid-state image sensor 6 moves to the position indicated by this position information, so that the movement of the image pickup apparatus Will be cancelled.

  Step 310 is a clip value so that the control signal sent from the microcomputer 15 to the solid-state image sensor 6 does not indicate a correction amount exceeding the motion correction range by the solid-state image sensor 6, that is, the maximum movement range of the solid-state image sensor 6. In this step, the signal width of the control signal is limited (clip process).

  Step 311 is a step of recording the value of the control signal after clip processing in the microcomputer, and this recorded value is a coefficient α that determines the degree of sharpness improvement of the image in the sharpness improvement means 3003 in the above-mentioned step 303. It is used when calculating The data after the clip processing is converted into an analog signal by the D / A conversion means 36 and sent to the solid-state image sensor drive control means 32. There is a possibility that the solid-state image sensor 6 may not be moved according to the control signal due to the optical axis direction of the moving mechanism of the solid-state image sensor 6, the rotation direction on the imaging plane, the frequency characteristics of the drive system, and the like. In such a case, when the value of the control signal is compared with the position information of the solid-state imaging device 6 detected in step 302, they do not coincide with each other, and some difference value is detected. In other words, the effect of correcting the movement of the image cannot be achieved, and as a result, the resolution of the image is lowered due to the movement of the imaging device.

  Therefore, in step 303, the value of the control signal is compared with the position information of the solid-state imaging device 6 detected in step 302, and the degree of sharpness improvement of the image in the sharpness improvement means 3003 is controlled based on the difference value. This improves the deterioration of the image due to the fact that the intended motion correction cannot be performed. Since this method is the same as that of the first embodiment, description thereof is omitted.

  Thus, by incorporating the correction mechanism into the main body, it is not necessary to incorporate a correction lens into the lens itself, and the lens barrel can be made smaller. In an interchangeable lens camera such as a single-lens reflex camera, an existing lens can be used as it is.

(Embodiment 4)
An image motion correction apparatus according to Embodiment 4 of the present invention will be described with reference to FIGS. The image motion correction apparatus according to the present embodiment performs camera shake detection of the imaging apparatus by detecting a motion vector of a captured image instead of an angular velocity sensor. Other configurations and processes are the same as those of the first embodiment. The same.

  FIG. 21 is a block diagram of the image motion correction apparatus according to the present embodiment. The image motion correction apparatus according to this embodiment includes a lens device 52 and an imaging device 63. Then, the motion detection block is removed from the lens device 52, and an image motion vector detection means 40 is newly added to the imaging device 63 instead. In FIG. 21, since the blocks other than the motion detection block are the same as those in FIG. 1, they are denoted by the same reference numerals and description thereof is omitted.

  The output of the A / D conversion unit 8 is input to the image motion vector detection unit 40. The image motion vector detection means 40 detects the motion of the captured image by detecting the motion vector of the image by pattern matching between fields or frames. If this motion vector is used, the direction and amount of movement of the image can be directly obtained. This information is taken in by the microcomputer 15 and a control signal given to the lens group L32 drive control means 2 is calculated. Then, the lens group L32 is driven based on this control signal. Further, based on the difference between the output of the moving position detecting means 3 and the control signal, the digital signal processing means 9 controls the sharpness improving process.

  Next, a method for detecting a motion vector will be described. FIG. 22 is a block diagram showing a specific configuration example of the image motion vector detection means 40 shown in FIG. In FIG. 22, the representative point storage unit 6001 divides the image signal of the current field input from the A / D conversion unit 8 into a plurality of regions, and the image signal corresponding to a specific representative point in each region is represented as a representative point signal. It is something to remember as. The representative point storage means 6001 gives the correlation point calculation means 6002 a representative point signal of the previous field scanned one field before the current field. The correlation calculation means 6002 performs a correlation calculation between the previous representative point signal and the current field image signal and compares the difference between the previous representative point signal and the current field image signal, and the output is sent to the motion vector detection means 6003. Given. The motion vector detecting unit 6003 detects the motion vector (the moving amount and moving direction of the image) between the previous field and the current field from the calculation result in the correlation calculating unit 6002. Although the case where the motion vector between fields was detected was demonstrated above, it is not limited to this, You may carry out between frames when the speed of camera shake is small.

  FIG. 23 is an example of a flowchart of a processing program stored in the microcomputer 15.

  The same processing contents as those in the first embodiment are denoted by the same reference numerals as those in FIG. It is assumed that the processing shown in FIG. 23 is also executed for each interruption by the microcomputer 15 built-in timer. When a camera shake correction switch (not shown) is turned on, a series of processes shown in FIG. 23 is started.

  When a camera shake correction switch (not shown) is turned on, an initial value necessary for the subsequent processing is set in step 101. In the case of the present embodiment, since the angular velocity sensor is not used, an initial value such as filtering necessary for the processing of the output of the angular velocity sensor is unnecessary, but the size of the detection region for detecting the motion vector, the lens group L32 Set initial values such as the moving amount calculation coefficient and clip value.

  When an interruption by the timer is applied, the output of the moving position detecting means 3 converted into a digital signal by the A / D converting means 18 in step 102, that is, the position of the lens group L32 is taken into the microcomputer 15.

  Next, in step 103, a coefficient that determines the degree of sharpness improvement of the image in the sharpness improvement means 3003 based on the position of the lens group L32 captured in step 102 and the value stored in step 111 described later. α is calculated.

  Next, in step 404, the motion vector of the photographed image is taken into the microcomputer 15 from the image motion vector detection means 40. Step 409 is a step of converting the image motion information obtained from the image motion vector detection means 40 in step 404 into position information of the lens unit L32 for canceling the motion. Then, the obtained position information is transmitted to the lens group L32 drive control means 2 via the D / A conversion means 16, and the lens group L32 moves to the position indicated by this position information, so that the movement of the captured image is changed. Canceled. Since the following operations are the same as those in the first embodiment, description thereof will be omitted.

  As described above, if the motion vector of the captured image is used as the motion detection method of the imaging apparatus, the angular velocity sensor and the HPF 11 related thereto that are necessary in the configuration using the angular velocity sensor 10 shown in FIG. , LPF 12, amplifier 13, and A / D conversion means 14 are not required. Thereby, the shake detection part of an imaging device can be comprised cheaply and small.

  As described above, the use of the image motion correction apparatus of the present invention can improve the resolution of an image deteriorated due to camera shake even when camera shake that cannot be corrected during camera shooting occurs, and can provide a high-quality shot image.

  In the first embodiment of the present invention, an example in which a filter that emphasizes high-frequency components in the horizontal direction and the vertical direction on the image space has been described as the sharpness improving method. However, the sharpness improving method is described here. The conventional general method can be used without limitation. For example, an orthogonal transformation such as Fourier transformation is performed on a photographed image, and the image is temporarily mapped to a frequency space, and high frequency components such as horizontal, vertical, and diagonal are emphasized in this frequency space, and returned to the image space by inverse Fourier transformation. Similar effects can be obtained. Further, the image may be divided into, for example, 8 × 8 pixel blocks and subjected to orthogonal transformation such as discrete cosine transformation (DCT) or Hadamard transformation to emphasize the components of the block representing horizontal, vertical and diagonal high frequency components. . By doing so, finer sharpness control can be performed in units of the blocks.

  In the first embodiment of the present invention, as an example of an image motion correction unit, a configuration in which the optical axis is decentered by moving some lenses in the imaging optical system 1 in a direction perpendicular to the optical axis is taken as an example. The explanation is not limited to this. For example, two glass plates are connected by a bellows, filled with a high refractive index liquid, and the optical axis can be decentered by changing the angle of the glass plate. It goes without saying that the present invention is effective even when an optical system using a vertical angle prism (VAP) is used. If VAP is used, an existing lens can be used as it is, so there is no need to purchase a new lens.

  In the first embodiment of the present invention, the case where the lens unit L32 is moved has been described as the motion correction unit. However, the present invention is not limited to this, and the imaging optical system 1 and the solid-state imaging device 6 are included. A configuration is also conceivable in which the movement is corrected by supporting and driving the imaging device casing rotatably.

  Further, in the first to third embodiments of the present invention, the transfer function as a specific example is shown with respect to HPF, phase compensation, and integration processing among the processing in the microcomputer 15, but is not limited thereto. It goes without saying that the present invention is effective even when processing expressed by different transfer functions that have the same effect is used.

  Further, in all of the embodiments of the present invention, an example by program processing by a microcomputer is shown, but the present invention is not limited to this, and it is possible to realize program processing by a microcomputer by hardware such as an electronic circuit. Needless to say.

  Further, in all of the embodiments of the present invention, the configuration for changing the characteristics of the resolution recovery processing for each processing loop has been described. However, the present invention is not limited to this. For example, once every loop processing, A method of changing the characteristics of the resolution recovery processing may be used. As a result, the resolution does not change frequently and the user does not feel uncomfortable.

  In all of the embodiments of the present invention, the number of solid-state imaging elements of the imaging device is not particularly mentioned, but the present invention is applicable to any imaging device such as a single-plate imaging device, a two-plate imaging device, and a three-plate imaging device. It is clear that the invention is effective. Needless to say, the present invention is effective for both a CCD and a CMOS as a solid-state imaging device.

  In the embodiment of the present invention, the lens and the image sensor have been described as the motion correction unit. However, these may be used in combination. Further, although the angular velocity sensor and the motion vector detection have been described as the motion detection means, it goes without saying that these may be used in combination. Further, the correcting means and the motion detecting means may be used in any combination. This makes it possible to achieve both price and performance by combining the optimum method for each shooting condition and product.

  The present invention can be used for, for example, an imaging apparatus such as a digital camera or a video movie having a camera shake correction function.

Block diagram of an image motion correction apparatus in Embodiment 1 of the present invention 1 is a perspective view of a shake correction optical mechanism according to Embodiment 1 of the present invention. The block diagram which shows an example of a structure of the digital signal processing means in Embodiment 1 of this invention. The block diagram which shows an example of a structure of the sharpness improvement means in Embodiment 1 of this invention The flowchart for demonstrating the processing content by the microcomputer in Embodiment 1 of this invention The figure which shows an example of the output timing of the control signal in Embodiment 1 of this invention, and the detection timing of the positional information on the lens group L32. The figure which shows an example of the relationship between the value of the control signal in Embodiment 1 of this invention, the absolute value of the difference value of the positional information on the lens group L32, and the coefficient (alpha) which determines the sharpness improvement degree of an image FIG. 5 is a diagram in which the control signal value, the absolute value of the difference value of the position information of the lens unit L32, and the coefficient α are associated nonlinearly in the first embodiment of the present invention. FIG. 5 is a diagram in which the coefficient α is a fixed value when the absolute value of the difference value between the control signal value and the position information of the lens unit L32 in the first embodiment of the present invention is equal to or less than a certain value. A block diagram of sharpness improving means in which the coefficient α in the first embodiment of the present invention can be set independently for each of the horizontal and vertical directions. The block diagram which shows the image motion correction apparatus in Embodiment 2 of this invention. The block diagram which shows the structural example of the digital signal processing means in Embodiment 2 of this invention. FIG. 5 is a block diagram showing a configuration example of an image restoration unit in Embodiment 2 of the present invention. The flowchart which shows the operation | movement procedure of the image restoration means in Embodiment 2 of this invention. FIG. 6 is a diagram for explaining the principle of image restoration in Embodiment 2 of the present invention, where (a) is a diagram showing an image g (x, y) shaken by camera shake, and (b) is a diagram showing the amount of camera shake. c) A diagram showing an image f (x, y) without camera shake. It is a figure explaining the relationship between a point spread function and its Fourier transform, (a) is a diagram showing a point spread function h (x), (b) is a diagram showing a Fourier transform of h (x), and (c) is a diagram. The figure which shows the frequency transfer function M (u) of an inverse filter It is a figure which shows the relationship between the movement amount of the image on a solid-state image sensor, and transfer function H (u), (a) is a figure which shows the pixel structure on a solid-state image sensor, (b) is transfer function H (u). Of the relationship between the frequency characteristics and the amount of movement The flowchart of the processing program stored in the microcomputer in Embodiment 2 of this invention Block diagram of an image motion correction apparatus in Embodiment 3 of the present invention The flowchart of the processing program stored in the microcomputer in Embodiment 3 of this invention Block diagram of an image motion correction apparatus in Embodiment 4 of the present invention The block diagram which shows the specific structural example of the image motion vector detection means shown in FIG. 21 in Embodiment 4 of this invention. The flowchart of the processing program stored in the microcomputer in Embodiment 4 of this invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging optical system 2 Lens group L32 drive control means 3,33 Movement position detection means 4 Imaging optical system drive control means 5, 8, 14, 18, 38 A / D conversion means 6 Solid-state image sensor 7 Analog signal processing means 9, 29 Digital signal processing means 10 Angular velocity sensor 11 HPF
12 LPF
DESCRIPTION OF SYMBOLS 13 Amplifier 15 Microcomputer 16,36 D / A conversion means 17 Solid-state image sensor drive means 32 Solid-state image sensor drive control means 40 Image motion vector detection means 50, 51, 52 Lens apparatus 60, 61, 62, 63 Imaging apparatus

Claims (26)

A lens apparatus that includes an imaging optical system that includes a lens group including at least one lens and forms an image of a subject on an imaging surface;
An imaging device comprising: an imaging device that converts an object image formed on the imaging surface by the imaging optical system into an image signal; and a resolution recovery means that performs resolution recovery processing on the image signal obtained by the imaging device. An image motion correction device comprising:
Motion detection means for detecting the motion of the imaging device;
Motion correction means for correcting the movement of the subject image caused by the movement of the imaging device by moving the position of the subject image formed by the imaging optical system;
Control signal generating means for generating a control signal for controlling the motion correcting means based on the output of the motion detecting means;
Correction state detection means for detecting a correction state of movement of the subject image by the motion correction means,
A characteristic of resolution recovery processing by the resolution recovery means is changed based on the control signal generated from the control signal generation means and the correction state of the movement of the subject image detected by the correction state detection means. An image motion correction device. 2. The image motion correction apparatus according to claim 1, wherein the resolution recovery means recovers the resolution of the image by improving the sharpness of the image. 3. The improvement of the sharpness of the image is performed by making the improvement degree of the sharpness of the image proportional to an absolute value of a difference value between the control signal and the motion image correction state. The image motion correction apparatus described. The improvement of the sharpness of the image is performed by causing the improvement degree of the sharpness of the image to correspond non-linearly to an absolute value of a difference value between the control signal and the motion image correction state. Item 3. The image motion correction apparatus according to Item 2. The sharpness of the image is improved by fixing the degree of improvement of the sharpness of the image when the absolute value of the difference value between the control signal and the movement correction state of the subject image is a predetermined value or less. The image motion correcting apparatus according to claim 2, wherein the image motion correcting apparatus is performed. The sharpness of the image is improved when the absolute value of the difference value between the correction state of the movement of the subject image or the control signal and the correction state of the movement of the subject image is equal to or larger than a predetermined magnitude. 3. The image motion correction apparatus according to claim 2, wherein the improvement is made constant. The sharpness of the image is improved when the absolute value of the difference value between the correction state of the movement of the subject image or the control signal and the correction state of the movement of the subject image is equal to or larger than a predetermined magnitude. 3. The image motion correction apparatus according to claim 2, wherein the improvement is performed by stopping the improvement processing. 3. The image motion correction apparatus according to claim 2, wherein the sharpness of the image is improved by enhancing at least a horizontal or vertical high-frequency component of the image signal in an image space. 3. The image motion correction apparatus according to claim 2, wherein the sharpness of the image is improved by enhancing at least a horizontal or vertical high-frequency component of the image signal in a frequency space mapped by orthogonal transformation. 10. The image motion correction apparatus according to claim 8, wherein the high-frequency component enhancement is performed by independently changing a degree of enhancement of horizontal and vertical high-frequency components. 2. The image motion correction apparatus according to claim 1, wherein the resolution recovery means recovers the resolution using a point spread function. The image motion according to claim 1, wherein the resolution recovery is performed by changing the characteristics of the resolution recovery process every time the resolution recovery process is performed a plurality of times. Correction device. The motion correction means corrects the motion of a subject image by decentering the optical axis of the imaging optical system by being driven relative to the imaging optical system. The image motion correction apparatus according to any one of the above. The image motion correction apparatus according to claim 13, wherein the movement of the subject image is corrected by rotationally driving about two axes orthogonal to the optical axis of the imaging optical system. The movement of the subject image is corrected by decentering the optical axis of the imaging optical system by individually driving at least one lens in a direction orthogonal to the optical axis of the imaging optical system. The image motion correction apparatus according to claim 13, wherein The image motion correction according to claim 1, wherein the motion correction unit corrects the motion of the subject image by displacing the image sensor in a direction orthogonal to the optical axis. apparatus. The image according to any one of claims 1 to 16, wherein the correction state detection unit detects a motion correction state of a subject image based on a displacement amount of a moving part constituting the motion correction unit. Motion compensation device. The image motion correction apparatus according to claim 17, wherein the moving part is a lens group including at least one lens. The image motion correction apparatus according to claim 17, wherein the moving part is an image sensor. The control signal generating means performs clip processing for limiting the signal width of the control signal to a predetermined value based on a detection result of the correction state detecting means and a motion correction amount of the motion correcting means. The image motion correction apparatus according to claim 1. 21. The image motion correction apparatus according to claim 20, wherein the clip processing is performed when the control signal indicates a correction amount exceeding a motion correction range of the motion correction means. The image motion correction apparatus according to claim 20 or 21, wherein a motion correction state of a subject image is detected based on the control signal before the clip processing is performed. The image motion correction apparatus according to claim 1, wherein the motion detection unit is an angular velocity sensor that detects an angular velocity of a motion of the imaging apparatus itself. The image motion correction apparatus according to claim 1, wherein the motion detection unit is a motion vector detection unit that detects a motion vector of an image from a captured image. It is composed of a lens group including at least one lens, and has an imaging optical system that forms an image of a subject on an imaging surface.
Combined with an imaging device comprising an imaging device for converting a subject image formed on the imaging surface by the imaging optical system into an image signal, and resolution recovery means for performing resolution recovery processing on the image signal obtained by the imaging device A lens device constituting an image motion correction device,
Motion detection means for detecting the motion of the imaging device;
Motion correction means for correcting the movement of the subject image caused by the movement of the imaging device by moving the position of the subject image formed by the imaging optical system;
Control signal generating means for generating a control signal for controlling the motion correcting means based on the output of the motion detecting means;
Correction state detection means for detecting a correction state of movement of the subject image by the motion correction means,
A characteristic of resolution recovery processing by the resolution recovery means is changed based on the control signal generated from the control signal generation means and the correction state of the motion of the subject image detected by the correction state detection means. Lens device to do. An image motion correction device is configured with a lens device that includes an imaging optical system that includes a lens group including at least one lens and forms an image of a subject on an imaging surface.
An imaging apparatus comprising: an imaging element that converts a subject image formed on an imaging surface by the imaging optical system into an image signal; and a resolution recovery unit that performs resolution recovery processing on the image signal obtained by the imaging element. And
Motion detection means for detecting the motion of the imaging device;
Movement correcting means for correcting the movement of the subject image caused by the movement of the imaging device by moving the position of the subject image formed by the imaging optical system;
Control signal generating means for generating a control signal for controlling the motion correcting means based on the output of the motion detecting means;
Correction state detection means for detecting a correction state of movement of the subject image by the motion correction means,
A characteristic of resolution recovery processing by the resolution recovery means is changed based on the control signal generated from the control signal generation means and the correction state of the movement of the subject image detected by the correction state detection means. An imaging device.

Images