JP6271969B2 - Imaging apparatus and image correction method - Google Patents

Description

The present invention relates to an imaging apparatus and an image correction method , and more particularly to a technique that achieves both a reduction in resources required for image correction processing by geometric deformation and a correction effect.

  In recent imaging apparatuses such as a digital video camera and a digital camera, it is possible to acquire a higher quality image by improving the imaging device and the signal processing technology. Further, in the correction of distortion and camera posture change, downsizing and cost reduction of the optical system are required along with downsizing of the camera body. Regardless of whether it is a moving image or a still image, acquisition of more natural and stable video and composite processing of a plurality of images are highly important technologies. Lens distortion information and camera posture change information are acquired, a geometric deformation amount for the pixel of interest is given to each pixel, and distortion aberration correction and image blur correction can be realized by signal processing.

  In addition to a method using a detection device such as an angular velocity sensor (gyro sensor) or an acceleration sensor (shift sensor), the movement of the camera is detected by comparing the input image with a past reference image. There is a so-called motion vector detection method. When the camera rotates or translates, the change in the angle of view affects the image as image shake. For this reason, image blur can be suppressed by driving and controlling a correction optical member (correction lens) disposed in the optical system. In addition, image blur can be suppressed by performing geometric deformation processing with a high degree of freedom such as projective transformation on image data by signal processing.

  In the case of optical image blur correction by drive control of the correction lens, the correction lens is designed to obtain a large correction angle with a small driving amount. On the other hand, the influence of the image distortion due to the eccentric distortion due to the correction lens deviating from the center of the optical axis is unavoidable. In order to appropriately correct geometric deformation for non-eccentric distortion, camera posture change, and image distortion due to decentration distortion, it is necessary to apply correction for each factor and to correct in the reverse order of the factors causing the phenomenon. In this case, when the geometric deformation factor increases and the characteristics become complicated, the circuit resource and the number of parameters increase for each factor. Along with this, parameter setting time and execution time increase, so that the load and power consumption of correction processing increase. With respect to this problem, as a technique for adaptively performing image distortion correction, the apparatus disclosed in Patent Document 1 detects the amount of camera movement due to camera shake and automatically determines whether image blur correction is applicable. In the apparatus disclosed in Patent Document 2, the coordinate transformation method for shake correction is changed by changing the estimation method of the geometric deformation parameter for shake correction based on camera shake information.

JP 2006-254365 A JP 2007-166269 A

In Patent Document 1, since whether or not to apply image blur correction is binary controlled, control discontinuity becomes a problem. In the case of Patent Document 2, there is a problem that cannot be dealt with only by changing the estimation method of the geometric deformation parameter based on camera shake information. In these conventional techniques, control for complex geometric deformation is not considered for complex image distortion due to non-eccentric distortion or decentration distortion.
An object of the present invention is to reduce the load of correction processing while substantially securing the effect of geometric deformation correction on the posture change of the image pickup apparatus and image distortion caused by the photographing optical system, and reduce power consumption by reducing circuit resources. It is to suppress.

  In order to solve the above-described problems, the apparatus according to the present invention corrects an image by synthesizing coordinate transformations based on a plurality of geometric deformations on an image acquired by an imaging optical system and an imaging device, and performing coordinate transformation for each pixel. An imaging device that performs geometric deformation information from distortion amount data that changes according to optical parameters of the imaging optical system, a detection unit that detects a change in posture of the imaging device, and the detection unit Calculation means for obtaining detection information by calculating geometric deformation information, coordinate conversion control means for acquiring the geometric deformation information from the generation means and calculation means, and determining a composition ratio of the respective coordinate conversions, and the coordinates Geometric deformation amount control means for outputting a geometric deformation parameter in accordance with the composition ratio indicated by the conversion control means; Coordinate calculation means for outputting coordinates after geometric transformation from a standard, and geometric deformation processing means for performing geometric deformation processing by coordinate transformation instructed by the coordinate calculation means on an image acquired by the photographing optical system and the image sensor. .

  According to the present invention, it is possible to reduce the load of correction processing and to reduce power consumption by reducing circuit resources while substantially ensuring the effect of geometric deformation correction on the posture change of the imaging apparatus and image distortion caused by the photographing optical system. Can be suppressed.

It is a block diagram which shows the structural example of the imaging device which concerns on embodiment of this invention. It is a flowchart which shows operation | movement of the imaging device of 1st Embodiment. It is a flowchart which shows operation | movement of the imaging device of 2nd Embodiment. It is a figure explaining an aberration correction characteristic. It is a figure showing distortion by a non-eccentric distortion aberration and an eccentric distortion aberration. It is a figure showing distortion by camera posture change.

  Embodiments of the present invention will be described with reference to the accompanying drawings. In the following, an image obtained by performing coordinate transformation by a plurality of geometrical deformations sequentially on an image acquired by a photographing optical system (imaging optical system) and an image sensor, and combining them into one coordinate transformation to perform coordinate transformation for each pixel The apparatus will be described. Image correction by a plurality of geometric deformation processes includes correction of distortion aberration (eccentric distortion aberration, non-eccentric distortion aberration) and camera attitude correction. Eccentric distortion is distortion that occurs asymmetrically with respect to the optical axis of the optical system. Non-eccentric distortion is distortion that occurs point-symmetrically with respect to the optical axis of the optical system. Aberrations other than distortion, such as chromatic aberration of magnification, are excluded.

FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus according to an embodiment of the present invention.
The photographing optical system 101 includes a lens group that forms a subject image. The correction lens 102 is a correction member that can move in a direction different from the optical axis in the photographing optical system 101. For example, an optical member (shift lens) that can move in a direction orthogonal to the optical axis is used for image blur correction. A tilt lens or the like may be used as the correction member.
The image sensor 103 is a CCD (charge coupled device) sensor, a CMOS (complementary metal oxide semiconductor) sensor, or the like that photoelectrically converts a subject image formed by the photographing optical system 101. The development processing unit 104 generates a video signal from the electrical signal output from the image sensor 103. The development processing unit 104 includes an A (analog) / D (digital) conversion unit, an auto gain control unit (AGC), and an auto white balance (AWB) unit (not shown), and outputs a digital signal. The imaging optical system 101, the imaging element 103, and the development processing unit 104 constitute an imaging system that performs image acquisition. The memory 105 temporarily stores and holds image data of one frame or a plurality of frames of the video signal generated by the development processing unit 104.

  The aberration information storage unit 106 holds, as a plurality of data in the image height direction, aberration amounts that change depending on optical parameters such as the focal length and object distance of the photographing optical system 101. An aberration correction parameter generation unit (hereinafter referred to as a variable generation unit) 107 acquires a current optical parameter output from the aberration information storage unit 106, and based on the parameter, calculates an aberration correction amount of the photographing optical system 101 as a geometric deformation parameter. Output as (geometric deformation information).

  The posture change detection unit 108 is configured by a detection device such as a gyro sensor for acquiring detection information of movement and posture change of the imaging device such as camera shake and camera work. A geometric deformation parameter estimation unit (hereinafter referred to as a variable estimation unit) 109 acquires posture change detection information from the posture change detection unit 108 and estimates a camera posture correction parameter. The variable estimation unit 109 outputs the estimated calculation result as a geometric deformation parameter (geometric deformation information), calculates the driving amount of the correction lens 102, and outputs it to the drive control unit 116. The drive control unit 116 drives and controls the correction lens 102 according to the drive amount.

  The coordinate conversion control unit 110 acquires parameters output from the variable generation unit 107 and the variable estimation unit 109 and focal length information of the photographing optical system 101, and determines a combination ratio of each parameter. The geometric deformation amount control unit 111 acquires parameters respectively output from the variable generation unit 107 and the variable estimation unit 109, and handles each geometric deformation based on the synthesis ratio of each parameter instructed from the coordinate transformation control unit 110. The geometric deformation parameter to be calculated is calculated. The coordinate calculation unit 112 sequentially performs coordinate calculation of each geometric deformation process based on the geometric deformation parameter instructed from the geometric deformation amount control unit 111, and outputs coordinates after geometric conversion from coordinates before geometric conversion. The geometric deformation processing unit 113 reads the frame image data held in the memory 105, performs geometric deformation processing by coordinate transformation instructed from the coordinate calculation unit 112 on the data, and stores the processed image data in the memory 105. Write out. The image recording unit 114 reads out the image data subjected to aberration correction from the memory 105 and records it on a recording medium. The image display unit 115 displays the image subjected to aberration correction on the screen of the display.

Next, geometric deformation processing control in the present embodiment will be described in detail with reference to the flowchart shown in FIG.
After the processing starts, the parallel processing of S201, S203, and S207 is executed. In S201, the variable generation unit 107 acquires data held in the aberration information storage unit 106 based on the current optical parameter. FIG. 4A illustrates the change in the amount of aberration in the image height direction. The horizontal axis represents the image height h, and the vertical axis represents the non-decentration distortion amount D (h). In order to save the memory capacity, the data of the amount of non-eccentric distortion with respect to the optical parameter is stored discretely. For the optical parameters at the plot positions that are not held, the amount of non-eccentric distortion is calculated by interpolation from the amount of aberration at nearby positions.

と表すことができる。 In step S202 following step S201, the variable generation unit 107 estimates a geometric deformation parameter based on the aberration amount of the photographing optical system 101 obtained in step S201. For example, the non-eccentric distortion correction parameter is calculated by estimation. The amount of non-decentration distortion aberration Δd of the photographing optical system 101 indicates a change in the reference pixel position according to the image height. The distortion rate is D, the cell pitch of the image sensor 103 is Cp, the ideal image height is R, and the real image height is If R ′,

It can be expressed as.

FIG. 5 is an explanatory diagram of non-eccentric distortion and decentration distortion. If the photographic optical system 101 has no distortion, the lattice image 501 obtained by imaging is in a state where vertical and horizontal lines are orthogonal to each other as shown in FIG. When the photographing optical system 101 has non-eccentric distortion, for example, the subject image is distorted and formed as shown exaggeratedly in the image 502 of FIG. FIG. 5B shows a shape example in which the vertical and horizontal lines are deformed into a barrel shape.
In step S <b> 203, the posture change detection unit 108 detects a posture change of the imaging device and acquires motion information. When a gyro sensor is used as an example of motion information, motion information in the yaw direction, pitch direction, and roll direction of the imaging apparatus can be obtained. Note that the present invention is not limited to the gyro sensor, and another means (such as a motion vector detection means) for obtaining the motion information of the imaging apparatus may be used.

  FIG. 6 is an explanatory diagram of posture change. If the imaging apparatus is in a state where there is no change in posture, the grid image 601 obtained by forming an image is in a state where vertical and horizontal lines are orthogonal as shown in FIG. When the posture of the imaging apparatus is changed, for example, the subject image is distorted and formed as shown exaggeratedly in the image 602 in FIG. 6B. FIG. 6B shows an example of a shape in which vertical and horizontal lines cross each other.

After S203 of FIG. 2, the parallel processing of S204 and S205 is executed.
In S204, the variable estimation unit 109 estimates a geometric deformation parameter between a plurality of frame images based on the posture change amount of the imaging apparatus obtained in S203. In the present embodiment, a case where a 3 × 3 matrix called a homography matrix is used as an example representing geometric deformation will be described.

とする。点ａが次フレームにおいて点ａ′、

に移動したとする。（式２）および（式３）にて、添え字Ｔは転置行列であること示す。（式２）の点ａと（式３）の点ａ′との対応関係は、ホモグラフィ行列Ｈを用いることにより、

と表すことができる。 First, a certain point a on the image is

And Point a is the point a ′ in the next frame,

Suppose you move to. In (Expression 2) and (Expression 3), it is shown that the subscript T is a transposed matrix. The correspondence between the point a in (Expression 2) and the point a ′ in (Expression 3) is obtained by using the homography matrix H:

It can be expressed as.

（式５）のホモグラフィ行列Ｈは、下記の成分を含む。
・パラメータｈ₁₃、ｈ₂₃：並進の動き成分。
・パラメータｈ₁₁、ｈ₁₂、ｈ₂₁、ｈ₂₂：回転、変倍、せん断の動き成分。
・パラメータｈ₃₁、ｈ₃₂：あおりの動き成分。
あおりの動き成分までを防振制御の対象とする場合には、８個のパラメータ全てを推定しなければならないため、８自由度の最小二乗法を使用する必要がある。また、回転の動き成分までを防振制御の対象とする場合には、あおりの動き成分を表すパラメータｈ₃₁、ｈ₃₂を推定する必要がない。このため、それ以外の６個のパラメータの推定、つまり６自由度での最小二乗法による推定を行えばよい。また、並進の動き成分のみを防振の対象とする場合には、パラメータｈ₁₃、ｈ₂₃についてのみ推定を行えばよい。 The homography matrix H is a matrix indicating the amount of deformation caused by translation, rotation, scaling, shearing, and tilting between images, and can be expressed by the following equation.

The homography matrix H of (Formula 5) includes the following components.
Parameters h ₁₃ and h ₂₃ : translational motion components.
Parameters h ₁₁ , h ₁₂ , h ₂₁ , h ₂₂ : Motion components of rotation, zooming and shearing.
Parameters h ₃₁ and h ₃₂ : Movement components of tilt.
In the case where the motion component of the tilt is targeted for the image stabilization control, since all eight parameters must be estimated, it is necessary to use the least square method with 8 degrees of freedom. Further, in the case where even the rotational motion component is the target of the image stabilization control, it is not necessary to estimate the parameters h ₃₁ and h ₃₂ representing the tilt motion component. For this reason, estimation of the other six parameters, that is, estimation by the least square method with six degrees of freedom may be performed. In addition, when only translational motion components are targeted for image stabilization, only the parameters h ₁₃ and h ₂₃ need be estimated.

When a gyro sensor is used for the posture change detection unit 108, rotational shakes of γ, β, and α are detected in the yaw direction, pitch direction, and roll direction, respectively. In this case, the geometric deformation parameter obtained as the homography matrix H _g can be expressed by the following equation.

  In step S <b> 205, the variable estimation unit 109 calculates the driving amount of the correction lens 102 and outputs it to the driving control unit 116. The drive amount of the correction lens 102 is instructed to the drive control unit 116 to return the rotation angles in the yaw direction and the pitch direction obtained from the posture change detection unit 108. Let the driving amounts of the correction lens 102 in the horizontal direction and the vertical direction be δ and θ, respectively. The relationship between each rotation angle in the yaw direction and the pitch direction and the driving amounts δ and θ of the correction lens 102 is determined by the structure of the mechanism part of the device (correction optical system), and is a low-order function such as a linear function. Can be expressed by

と表すことができる。
このとき、補正レンズ１０２で生じる偏心歪曲収差は補正レンズ１０２の駆動量に応じて発生しており、駆動量が大きくなるにつれて偏心歪曲収差量が増大する。偏心歪曲収差のない状態での、図５（Ａ）に示す格子画像５０１は、偏心歪曲収差により変形する。例えば、図５（Ｃ）の画像５０３に誇張して示すように被写体像が歪んで結像することになる。 In step S206 after step S205, the variable estimation unit 109 calculates an eccentric distortion correction parameter. From the driving amounts δ and θ of the correction lens 102 in the horizontal and vertical directions, the geometric deformation amount of the eccentric distortion is

It can be expressed as.
At this time, the eccentric distortion generated in the correction lens 102 is generated according to the drive amount of the correction lens 102, and the eccentric distortion amount increases as the drive amount increases. The lattice image 501 shown in FIG. 5A in the state without the eccentric distortion is deformed by the eccentric distortion. For example, the subject image is distorted and formed as shown exaggeratedly in the image 503 in FIG.

  In step S <b> 207, the variable estimation unit 109 acquires focal length information of the photographing optical system 101. This information is information that is held and used by a control unit such as a CPU (central processing unit) in the imaging apparatus as control information for the imaging optical system 101. The method for detecting the zoom position is known and will not be described. In the next step S208, the coordinate transformation control unit 110 determines the composition ratio of each geometric deformation parameter based on the geometric deformation parameters output from the variable generation unit 107 and the variable estimation unit 109, respectively, and the focal length information of the photographing optical system 101. To do. For example, when the focal length is short, the photographic optical system 101 is likely to generate barrel-shaped non-decentration distortion (see FIG. 5B), but as the focal length increases, the non-eccentric distortion disappears or is slightly pinched. Designed to be a mold. The coordinate conversion control unit 110 outputs a control gain value so that the non-eccentric distortion correction parameter becomes weaker than the design value as the focal length becomes longer. This is illustrated in FIG. In terms of design, the correction characteristic indicated by the graph line A is multiplied by the control gain. As a result, the correction characteristic is changed to that indicated by the graph line B, so that the correction effect can be weakened. The method for changing the correction characteristic may be selected as appropriate by holding the correction characteristic in the memory as a plurality of data.

With reference to FIG. 4B, an example of control characteristics when the weight of non-eccentric distortion correction is increased as the focal length becomes shorter will be described. In FIG. 4B, the horizontal axis indicates the focal length, and the vertical axis indicates the correction gain, that is, the gain coefficient related to non-eccentric distortion correction. The characteristic shown in the graph line A of FIG. 4A (horizontal axis X = 0 and vertical axis Y = 1.0) is multiplied by the correction gain of FIG. 4B.
FIG. 4C shows an example of control characteristics when the decentration distortion correction weight is increased as the focal length is increased. In FIG. 4C, the horizontal axis indicates the focal length, and the vertical axis indicates the correction gain, that is, the gain coefficient related to the eccentric distortion correction. Each drive amount δ, θ in (Expression 7) is multiplied by the correction gain in FIG.
The control characteristics shown in FIGS. 4B and 4C are defined so that the correction gain becomes 1.0 in a state where two coordinate conversion processes are normally performed. Controls according to these characteristics are executed simultaneously, and the total value of correction gains (gain coefficient values) is controlled to 1.0 or less. Thus, the coordinate conversion can be controlled so that the non-eccentric distortion correction composition ratio is high when the focal distance is short, and the decentration distortion correction composition ratio is high when the focal distance is long. The composition ratio represents a ratio set when combining a plurality of corrections. When the focal length is short, non-eccentric distortion correction occupies a relatively large ratio, and when the focal distance is long, decentration distortion correction occupies a relatively large ratio. Furthermore, by providing a period in which the total value of the correction gains is zero, it is possible to have a transition period in which no coordinate conversion is performed, and to relieve the uncomfortable feeling at the time of switching.

  Further, the coordinate conversion control unit 110 outputs a control gain value so that the geometric deformation correction parameter based on the driving amounts δ and θ of the correction lens 102 is weakened with respect to the lens driving amount as the focal length becomes longer. This can be realized by multiplying the gains so that the values of the driving amounts δ and θ in (Equation 7) become smaller. A method of changing the geometric deformation correction parameter of the correction lens 102 may be appropriately selected by having a plurality of correction parameter control functions.

合成されたホモグラフィ行列Ｈ_mを逆行列(Ｈ_m)^-1に変換することにより、像歪みが補正される。 When the parallel processing of the four systems shown in FIG. 2 is finished, the process proceeds to S209. In step S <b> 209, the geometric deformation amount control unit 111 calculates the parameters of each geometric deformation based on the composition ratio of the respective geometric deformations according to the instruction from the coordinate conversion control unit 110. For example, a combination ratio for correcting the posture change of the imaging apparatus and correcting the eccentric distortion of the photographing optical system is determined, and the respective geometric deformation parameters are calculated.
In the correction processing by coordinate transformation, the coordinate calculation unit 112 can correct image distortion correctly by executing calculation in the order of correction of camera posture change, correction of non-eccentric distortion, and correction of decentration distortion. Both the correction of the camera posture change and the correction of the eccentric distortion can be realized by projective transformation. That is, the following homography matrix H _m is calculated by multiplying the two homography matrices shown in (Expression 6) and (Expression 7).

Image distortion is corrected by converting the synthesized homography matrix H _m into an inverse matrix (H _m ) ⁻¹ .

  The coordinate conversion control unit 110 performs control so that decentration distortion is not corrected when the focal length of the imaging optical system 101 is short, and control is performed so that non-decentration distortion is not corrected when the focal length is long. To do. The projective transformation only needs to be performed once, but the geometric deformation correction is performed correctly. In S210, the geometric deformation processing unit 113 performs a geometric deformation process by coordinate transformation in accordance with the instruction of the coordinate calculation unit 112 obtained in S209.

As described above, in the present embodiment, when correcting the decentration distortion aberration of the photographing optical system 101, the posture change of the imaging apparatus, and the eccentric distortion aberration by the correction lens 102, the correction amount of the non-eccentric distortion aberration according to the focal length. And the weighting related to the correction amount of the eccentric distortion are controlled. That is, the control characteristics are determined by the correction gain multiplication processing described above. In addition, a composite ratio between the coordinate conversion related to the correction of the posture change of the imaging apparatus and the coordinate conversion related to the correction of the eccentric distortion of the photographing optical system is determined. Thereby, while substantially maintaining the correction effect, the processing load of the correction parameter setting and the correction processing itself can be reduced, and the power consumption can be suppressed by reducing the circuit resources.
Note that the continuity of the geometric deformation correction can be maintained by controlling the non-eccentric distortion correction amount and the decentration distortion correction amount so as to be zero with respect to the focal length without distortion. In addition, by providing a focal length region in which neither the decentration distortion correction nor the eccentric distortion correction is performed, the continuity of the geometric deformation correction can be maintained.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. The difference in control between the second embodiment and the first embodiment lies in the weighting related to the decentration distortion correction and the decentration distortion correction according to the amount of decentering distortion generated by the correction lens 102. Therefore, in the present embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed descriptions thereof will be omitted, and differences will be described.

With reference to the flowchart of FIG. 3, the control in this embodiment is demonstrated in detail. In the present embodiment, instead of the absence of S207 in FIG. 2, the processes of S206 and S208 are executed in parallel after S205. Therefore, only the process of S208 will be described.
In S208, the coordinate transformation control unit 110 acquires the geometric deformation parameters output from the variable generation unit 107 and the variable estimation unit 109, respectively. Then, the coordinate conversion control unit 110 determines the composition ratio of the geometric deformation parameters based on the horizontal and vertical drive amounts δ and θ of the correction lens 102 calculated in S205.
When the drive amounts δ and θ of the correction lens 102 are small, the correction lens 102 hardly generates decentration distortion, but the decentration distortion increases as the drive amounts δ and θ increase. As the driving amounts δ and θ increase, the coordinate conversion control unit 110 weakens the non-eccentric distortion correction parameter to a design value or more and outputs a control gain value so that the eccentric distortion can be corrected as designed. At that time, the magnitude of the drive amount may be controlled by either δ or θ, whichever is larger, or by using the synthesized magnitude, such as the mean square value of two angles. May be.

With reference to FIG. 4D, an example of control characteristics in the case where the weight of non-eccentric distortion correction is reduced as the amount of decentering distortion increases will be described. The horizontal axis of FIG. 4D indicates the driving amount δ or θ of the correction lens 102. The amount of decentering distortion can be defined by the driving amounts δ and θ of the correction lens 102. The vertical axis indicates a correction gain, that is, a gain coefficient related to non-eccentric distortion correction. The characteristic shown in FIG. 4A is multiplied by the correction gain shown in FIG.
FIG. 4E shows an example of control characteristics when the weight of decentration distortion correction is increased as the amount of decentration distortion increases. The horizontal axis of FIG. 4E indicates the driving amount δ or θ of the correction lens 102. The vertical axis represents a correction gain, that is, a gain coefficient related to decentration distortion correction. Each drive amount δ, θ in (Expression 7) is multiplied by the correction gain in FIG.
The control characteristics shown in FIGS. 4D and 4E are defined so that the correction gain is 1.0 in a state where two coordinate conversion processes are normally performed. Controls according to these control characteristics are executed simultaneously, and the total value of the correction gains is controlled to 1.0 or less. Thereby, when the amount of decentration distortion is small, the composition ratio of non-eccentric distortion correction is high, and when the amount of decentration distortion is large, the composition ratio of decentration distortion correction is high. As described above, as the amount of decentering distortion increases, as a relative result of reducing the weight of non-decentering distortion correction, coordinates are set so that the combined ratio of decentration distortion correction and camera posture change correction increases. You can control the conversion.
As for the method of changing the non-eccentric distortion correction characteristic, the correction characteristic may be held in a memory with a plurality of data and switched as appropriate. The method for changing the geometric deformation correction parameter of the correction lens 102 may be appropriately selected by having a plurality of correction parameter control functions.

  In this embodiment, since it is not necessary to obtain the focal length information of the photographing optical system 101 and calculate the composition ratio, the processing is simplified. That is, the composition ratio of each geometric deformation parameter is determined based on the driving amount of the correction lens 102. According to the present embodiment, it is possible to reduce the processing load of the correction parameter setting and the correction process itself while substantially maintaining the correction effect, and to suppress power consumption by reducing circuit resources. It should be noted that control may be performed so as to maintain the continuity of the geometric deformation correction by providing a drive amount region of the correction lens 102 that does not perform either the decentration distortion correction or the decentration distortion correction.

DESCRIPTION OF SYMBOLS 101 ... Shooting optical system 102 ... Correction lens 103 ... Imaging element 106 ... Aberration information storage part 107 ... Aberration correction parameter generation part 108 ... Attitude change detection part 109 ... Geometric deformation Parameter estimation unit 110 ... coordinate conversion control unit 111 ... geometric deformation amount control unit 112 ... coordinate calculation unit 113 ... geometric deformation processing unit 116 ... drive control unit

Claims (11)

An image capturing apparatus that performs image correction by synthesizing coordinate transformation by a plurality of geometric deformations and performing coordinate transformation for each pixel on an image acquired by an imaging optical system and an imaging element,
Generating means for generating geometric deformation information from distortion amount data that varies depending on optical parameters of the photographing optical system;
Detecting means for detecting a change in posture of the imaging device;
Calculating means for obtaining detection information by the detecting means and calculating geometric deformation information;
Coordinate transformation control means for obtaining the geometric deformation information from the generation means and the calculation means and determining a composition ratio of the respective coordinate transformations;
Geometric deformation amount control means for outputting geometric deformation parameters in accordance with the composition ratio indicated by the coordinate transformation control means;
Coordinate calculation means for obtaining the geometric deformation parameters and outputting coordinates after geometric transformation from coordinates before geometric transformation;
An imaging apparatus comprising: geometric deformation processing means for performing geometric deformation processing by coordinate transformation instructed by the coordinate calculation means on an image acquired by the photographing optical system and the imaging element. Before SL calculation means may calculates and outputs the driving amount of the correction member used in image blur correction to the drive control means of the correcting member, said generating means and said calculating means non decentering distortion of the photographing optical system and Each of the correction parameters for the eccentric distortion is calculated and output to the coordinate conversion control means,
The geometric transformation processing means performs geometric deformation processing in accordance with an instruction from the coordinate calculating unit, the correction of a change in the posture of the imaging device, the correction of the non-decentering distortion of the photographing optical system, executes the correction of decentering distortion The imaging apparatus according to claim 1, wherein:   The coordinate conversion control means determines a composite ratio of coordinate conversion related to correction of posture change of the imaging apparatus and coordinate conversion related to correction of decentration distortion of the photographing optical system. The imaging device described.   The imaging apparatus according to claim 2, wherein the coordinate conversion control unit controls weighting between correction of non-eccentric distortion and correction of decentration distortion.   5. The coordinate conversion control unit acquires information on a focal length of the photographing optical system, and increases a gain coefficient related to correction of non-eccentric distortion as the focal length becomes shorter. Imaging device.   The coordinate conversion control means acquires information on a focal length of the imaging optical system, and increases a gain coefficient for correcting decentration distortion as the focal length increases. The imaging device described. It said coordinate conversion control means, with a ratio of the correction of non-decentering distortion with respect to correction of decentering distortion when the focal length is short, relative to the correction of the non-decentering distortion when the focal length is long The imaging apparatus according to claim 6, wherein control is performed to increase a correction ratio of decentration distortion.   The imaging apparatus according to claim 4, wherein the coordinate conversion control unit decreases a gain coefficient related to correction of non-eccentric distortion as the amount of decentering distortion increases.   9. The imaging apparatus according to claim 4, wherein the coordinate conversion control unit increases a gain coefficient related to correction of the eccentric distortion as the amount of the eccentric distortion increases.   The coordinate conversion control means increases the ratio of correction of non-eccentric distortion to correction of decentration distortion when the amount of decentration distortion is small, and corrects non-eccentric distortion when the amount of decentration distortion is large. The image pickup apparatus according to claim 9, wherein control is performed to increase a decentration distortion correction ratio. An image correction method for performing image correction by synthesizing coordinate transformations by a plurality of geometric deformations and performing coordinate transformation for each pixel on an image acquired by an imaging optical system and an imaging element,
A generating step for generating geometric deformation information from distortion amount data that varies depending on optical parameters of the photographing optical system;
A detection step of detecting a change in posture of the imaging device;
A calculation step of obtaining detection information in the detection step and calculating geometric deformation information;
A coordinate transformation control step of obtaining the geometric deformation information generated in the generating step and the geometric deformation information calculated in the calculating step and determining a composition ratio of the respective coordinate transformations;
A geometric deformation amount control step for outputting a geometric deformation parameter in accordance with the composition ratio determined in the coordinate transformation control step;
A coordinate calculation step of obtaining the geometric deformation parameters and outputting coordinates after geometric transformation from coordinates before geometric transformation;
An image correction method comprising: a geometric deformation processing step of performing a geometric deformation process by coordinate transformation on an image acquired by the photographing optical system and the image sensor in accordance with an output in the coordinate calculation step.

Images