JP2006059270A - Method for correcting distortion of image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily and accurately perform distortion correction of an image resulting from aberration of a lens of a camera even in case that the optical axis of the lens is not arranged vertically to a deviation correction drawing. <P>SOLUTION: The distortion correction drawing having a plurality of sets of auxiliary points consisting of three or more linearly arranged auxiliary points on different straight lines is photographed by the camera to acquire an actual distortion correction image, a correction expression for giving an ideal center-of-gravity position of auxiliary points such that actual auxiliary points constituting each actual auxiliary point set are arranged substantially on a straight line in the image, and the distortion of the image is corrected by use of the correction expression. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a method for correcting distortion of an image photographed by a camera, and particularly to correction of distortion caused by distortion aberration of a camera lens. Note that the image in the present invention refers to a digital still image or a digital moving image formed by arranging a plurality of pixels on a two-dimensional plane.

  2. Description of the Related Art In recent years, techniques for measuring the size of a subject or collating an object or a person by processing and analyzing data of an image taken by a digital camera or the like have been developed. If it is an analysis based on images, it is possible to safely and quickly measure the length of an object that cannot be directly measured, and in the technical field such as collation, conventionally, there is no choice but to rely on human power, Since the collation work which required time and effort is automated, the result can be obtained in a very short time.

  However, when the above analysis is performed using an image photographed by a camera, distortion existing in the camera lens becomes a problem. If there is distortion in the lens, the image taken through the lens will not be similar to the actual subject and will have distortion. The main cause of this distortion is the distortion of the lens, also called distortion, which tends to be greater at the periphery of the lens than at the center of the lens. If the lens has distortion, the image of the subject in the captured image is different from the shape of the actual image of the subject, so that it is impossible to obtain a numerical value such as a length or a correct analysis result. In particular, when image matching is performed on a certain target, even if there is only a slight degree of image distortion or pixel position deviation that can be clearly judged to be the same or different by the human eye. This is inconvenient when the computer automatically performs the identification of the difference. Therefore, it is desirable that this distortion is corrected and removed as much as possible.

  In order to remove the distortion, a lens having a small distortion may be used. However, such a lens has a problem that it is expensive. Alternatively, it is possible to correct distortion by using a plurality of lens configurations. However, this is not only expensive as compared with a single lens camera, but also increases the size and weight.

  In view of this, a technique has been developed in which distortion is corrected by performing image processing so that an image captured and acquired through a lens becomes an ideal image having no distortion and similar to an actual subject. This distortion correction by image processing is generally performed by obtaining a distortion aberration correction expression representing distortion characteristics specific to a lens based on a plurality of points such as a plurality of lattice points and polygonal vertices. When obtaining a distortion correction formula based on the coordinate values of the lattice points and polygonal vertices, it is important to set the corrected ideal coordinate values so that no distortion exists. If this ideal coordinate value is appropriately determined, a correction formula capable of correcting the distortion of the entire image can be obtained by moving each pixel constituting the image having distortion to an ideal position. . As an example of such a correction formula, Non-Patent Document 1 proposes a correction formula that uses a pinhole camera model and obtains distortion correction as geometric transformation.

  As an example of a conventional correction method, Patent Document 1 discloses a technique related to a distortion aberration correction method for the purpose of improving measurement accuracy in image measurement. A calibration board on which a plurality of target marks with known coordinate values are displayed is photographed with a digital camera, and a multidimensional correction formula is obtained based on the measured coordinate values of the obtained photographed image and the known coordinate values, Make corrections.

  In Patent Document 1, it is assumed that the coordinate value of the target mark is known. However, after shooting the target mark, when comparing the image height measurement value on the imaging screen and the image height coordinate value based on the actual measurement, these values can only be compared in the same coordinate system, The image height measurement value on the photographing screen obtained in pixel units must be converted into the actual length or vice versa. This conversion is performed based on, for example, the relationship between the number of pixels constituting the target mark on the shooting screen and the known coordinate value. Since there is distortion on the shooting screen, the coordinate value of the shot image is changed. An error will be included when converting to the actual coordinate system.

  As described above, various image distortion correction methods have been proposed in the past. However, there are problems in that setting of ideal coordinate values is complicated and correction accuracy is low. Therefore, the inventor of the present application has disclosed a technique for effectively correcting by solving both of these problems in Patent Document 2 filed prior to the present application. This technology takes a distortion correction diagram having a plurality of lattice points arranged at equal intervals and a circle arranged at the center, acquires a distortion correction image as digital data, and uses the diameter of the center circle as a reference. Thus, the ideal positions of the lattice points arranged at equal intervals are appropriately determined, and distortion correction is performed. By using this technique, the position of an appropriate ideal lattice point can be easily set, and image distortion correction can be performed with high accuracy.

JP 2001-133223 A ([0008]) Japanese Patent Application No. 2004-191078 Katsumi Kankai and two others, “Distortion correction and panoramic images by image processing”, “Ricoh Technical Report No.23, SEPTEMBER, 1997”, pp.47-49.

  In the invention described in Patent Document 2, in order to maintain the constraint that the lattice points of the distortion correction diagram are arranged at equal intervals, the optical axis of the camera lens and the front of the camera are taken when the distortion correction diagram is photographed. Adjustments are made so that the normals set from the center of the distortion correction diagram arranged in FIG. However, since this adjustment must be performed strictly, a certain amount of time and labor are required for the work.

  Therefore, the problem to be solved by the present invention is to perform highly accurate distortion correction without strictly adjusting the optical axis of the lens and the normal of the distortion correction diagram when photographing the distortion correction diagram. It is an object to provide a distortion correction method for an image that enables the above.

  In order to solve the above problem, the inventor of the present application has corrected the distortion of the auxiliary points drawn on a straight line in the distortion correction diagram even if the distortion correction diagram is photographed with an inclination with respect to the optical axis of the lens. It was conceived that the ideal position of each auxiliary point is appropriately determined on the condition that the images are arranged on a straight line, and distortion correction is performed with high accuracy.

  This condition will be described with reference to FIG. Consider a case where a diagram in which a plurality of straight lines are drawn as shown in FIG. 6 (A) is photographed using an ideal lens without distortion. When the normal line set from the center of this figure is arranged so as to coincide with the optical axis of the lens, an image as shown in FIG. 6A is acquired. On the other hand, when the normal line set from the center of the figure does not coincide with the optical axis of the lens, that is, when the figure is arranged at an inclination, an image such as that shown in FIG. 6B is acquired. In this example, the upper side of the figure is inclined toward the front (that is, the lower side is at the back), and the left side of the figure is inclined toward the front (that is, the right side is at the back). However, each straight line in FIG. 6B has a change in slope as compared with the straight line depicted in FIG. 6A, but none of the straight lines is linear and maintains linearity. As a result, even if the distortion correction diagram is not arranged perpendicular to the optical axis and is photographed in an inclined state, the distortion correction diagram that has linearity is an ideal image without distortion. It can also be seen that the linearity is maintained.

  The image distortion correction method according to the present invention based on the above consideration is a distortion correction diagram having a plurality of auxiliary point sets each including three or more auxiliary points arranged on a straight line on different straight lines. A live-action distortion correction image is obtained by photographing with a camera, and in the live-action distortion correction image, a gravity center position of each live-action auxiliary point which is a point on the image corresponding to each auxiliary point is acquired, and each auxiliary point set A correction formula that gives the center of gravity position of the ideal auxiliary point such that the actual shooting auxiliary points constituting each set of actual shooting auxiliary point pairs corresponding to is arranged on a substantially straight line is obtained.

  In the distortion correction according to the present invention, it is assumed that the lens is axially symmetric with respect to the optical axis, and similarly, the distortion is also symmetric with respect to the optical axis. That is, the distortion aberration depends only on the distance from the center. Since the correction formula according to the present invention is unique to the lens, once the correction formula is calculated, the correction formula can always be applied thereafter.

  According to the image distortion correction method of the present invention, even if the distortion correction diagram is photographed in a state where the distortion correction diagram is not arranged perpendicular to the lens optical axis, an appropriate ideal auxiliary point in the distortion correction image is obtained. The position can be easily set, and the distortion correction accuracy of the image hardly decreases. Therefore, since an operation requiring strict adjustment can be omitted, it is possible to obtain an effect that time and labor can be greatly reduced as compared with the conventional correction method.

  Hereinafter, an image distortion correction method according to the present invention will be described. In the present invention, an image obtained by photographing a distortion correction diagram is referred to as a live-action distortion correction image, and an auxiliary point in the real-life distortion correction image is referred to as a live-action auxiliary point. An auxiliary point in an ideal image without distortion after correction is referred to as an ideal auxiliary point. Further, a set of live-action auxiliary points in the live-action distortion corrected image corresponding to the auxiliary point set in the distortion correction diagram is referred to as a live-action auxiliary point set.

First, the distortion correction diagram will be described. A plurality of auxiliary points are drawn in the distortion correction diagram. Then, the distortion correction diagram is configured such that an auxiliary point set including at least three auxiliary points is formed on each of a plurality of straight lines not drawn in the distortion correction diagram. One auxiliary point may belong to a plurality of auxiliary point sets. FIG. 1 is an example of a distortion correction diagram according to the present invention. In this distortion correction FIG. 1, a plurality of auxiliary points P ₁₁ , P ₁₂ ,... P ₆₇ are arranged in the form of lattice points, and a plurality of auxiliary point sets are formed in the horizontal and vertical directions. In this distortion correction FIG 1, P _11, P ₁₂ is a plurality of auxiliary point set disposed on a straight line in the horizontal direction, ..., P _{17 and, P 21, P 22, ...} , the horizontal direction P ₂₇ etc. A plurality of auxiliary point sets P ₁₁ , P ₂₁ ,..., P _61, etc. arranged on a straight line in the vertical direction are set as auxiliary point sets in the vertical direction. That is, each auxiliary point belongs to both the horizontal direction auxiliary point set and the vertical direction auxiliary point set. Distortion correction The horizontal and vertical spacing ratio of each auxiliary point in FIG. 1 is a: b, but it is not necessary that all the auxiliary points have a constant spacing ratio, and the auxiliary point spacing is constant. It doesn't have to be.

  Any auxiliary point can be used, such as the intersection of line segments, as long as the position of each auxiliary point can be specified in the distortion corrected image. It can be a point having a predetermined area as configured. In this case, in order to treat such an auxiliary point as a point, the center of gravity of a plurality of pixels forming the auxiliary point is set as the position of the auxiliary point. In addition, since the auxiliary point includes a large amount of distortion when the auxiliary point becomes large, the size of the auxiliary point is set to such a size that the center-of-gravity position can be acquired in the live-action distortion corrected image. For example, when the size of the entire image is 480 × 640 pixels (that is, 307200 pixels), it is preferable that the actual shooting auxiliary point is composed of 60 to 100 pixels. In addition, the accuracy of correction increases as the number of auxiliary points included in the distortion correction diagram increases, but at the same time, the calculation time increases.

  The distortion correction diagram configured as described above is taken with a camera. The distortion correction diagram does not need to be perpendicular to the lens optical axis of the camera, and the distortion correction diagram may have an inclination in any direction. The lens optical axis may not pass through the center of the distortion correction diagram.

  The center of the optical design of the lens may not match the center of the light receiving element such as a CCD. In order to perform this correction, a center circle having a predetermined area is drawn at the center of the distortion correction diagram, and the optical axis of the camera lens passes through the center of the distortion correction diagram (that is, the center of the center circle). Make adjustments as follows, and then shoot. This central circle may be one of the auxiliary points. Then, in the real image distortion correction image, the coordinates of the real image distortion correction image are corrected by translating the entire image so that the center of gravity of the real image center circle becomes the center of the light receiving element, that is, the origin of the coordinate system.

In this way, the center-of-gravity coordinates of the live-action auxiliary points D ₁₁ , D ₁₂ ,... Are acquired in the live-action distortion corrected image in which the coordinate system is appropriately corrected.

FIG. 2 shows distortion correction. FIG. 2 shows a live-action auxiliary point D ₁₁ in the real-image distortion correction image 2 when the right-hand side is tilted to the back side in the optical axis direction, rather than perpendicular to the optical axis of the camera. D ₁₂ ,... Are superimposed on ideal auxiliary points Q ₁₁ , Q ₁₂ ,... (The straight line connecting the auxiliary points in FIG. 2 is not a component of the real image distortion corrected image and the ideal auxiliary point.) .

The straight line drawn in the distortion correction diagram becomes a straight line in the ideal image after the distortion is corrected, regardless of the direction in which the distortion correction diagram is tilted during shooting. Based on this condition, the ideal coordinates of ideal auxiliary points Q ₁₁ , Q ₁₂ ,... Where the actual shooting auxiliary points D ₁₁ , D ₁₂ ,. If an expression that gives a value is obtained, the ideal coordinates of each pixel from which distortion aberration has been removed can be obtained from the photographed image based on the expression, and an image with corrected distortion can be created.

ここで、
F：レンズの焦点距離
θ：レンズの中心点と補正前の画素の座標を結ぶ直線が光軸となす角度
θ₀：レンズの中心点と補正後の画素の座標を結ぶ直線が光軸となす角度
A、B：パラメータ In the present invention, the following formulas 1 to 4 proposed in Non-Patent Document 1 can be suitably used as the distortion correction formula.

here,
F: Lens focal length θ: Angle that the straight line connecting the center point of the lens and the coordinates of the pixel before correction is the optical axis θ ₀ : Straight line connecting the center point of the lens and the coordinates of the corrected pixel is the optical axis angle
A, B: Parameter

Formulas 1 to 4 will be described with reference to FIG. 4 which is a model diagram of a pinhole camera. C is a pinhole point of the pinhole camera, and P is an object point in a three-dimensional space. P _d and P ₀ are the image forming points of P when there is a real image and there is distortion and when it is ideal and there is no distortion, and the angles formed by these and the optical axis (Z axis) are θ and θ ₀ . To do. Then, the distortion aberration of the lens is modeled by the polynomial expression 1 to obtain a distortion correction formula. Equations 2 to 4 are relational expressions clearly established by the geometric features of the model diagram depicted in FIG. That is, in Equations 1 to 4, if parameters A and B in Equation 1 are appropriately determined, a distortion correction equation can be obtained. A method for obtaining these two parameters will be described later.

Using Equations 1 to 4, image distortion correction is performed as follows. First, in an image having distortion, when the coordinates of a certain pixel are (x, y), these x and y are substituted into Equation 2 to obtain θ. If this θ is substituted into Equation 1, the value of θ ₀ can be obtained. Then, by substituting the values of θ ₀ and x and y into Equations 3 and 4, respectively, the corrected coordinates (x ₀ , y ₀ ) are determined. By performing this operation for all the pixels of the image, an image with corrected distortion can be obtained. When the ideal coordinate value is not an integer, the pixel complement is performed by the nearest neighbor method, the bilinear method, the bicubic method, or the like, thereby determining the luminance of each pixel. When the image is a color image, interpolation is performed for each pixel including color information.

Hereinafter, a method for obtaining the correction formula will be specifically described as an example. Distortion correction is performed using Equations 2 to 4 determined based on the pinhole camera model and Equation 1 modeling distortion. An example of a method for determining the parameters A and B in Expression 1 will be described. In this example, it is assumed that the distortion correction FIG. 1 in which 7 × 6 auxiliary points are drawn in the horizontal direction × vertical direction is used. In the present embodiment below, P _11, P _12, which are arranged in alignment in the horizontal direction in the distortion correction FIG 1, ..., P ₆₇ and P _21, P _22, ..., an auxiliary point set such P ₂₇ referred to as the horizontal auxiliary point set, P _11, P ₂₁ which vertically are disposed on a straight line, ..., P ₆₁ and P _12, P _22, ..., the auxiliary point set vertical auxiliary point set such P ₆₂ Call it. FIG. 5 shows a flowchart of a method for determining A and B.

First, in the live-action distortion corrected image 2, the center-of-gravity coordinates of the live-action auxiliary points D ₁₁ , D ₁₂ ,..., D ₆₇ are acquired (step S10).

  Thereafter, the number of executions of the calculation to be performed is set (step S11). Increasing the number of executions increases the possibility that more appropriate values of A and B can be obtained. Therefore, it is desirable to set as many as possible.

A set of parameters A and B having appropriate values is set for the number of executions set in step S11, and is substituted into Equation 1 (step S12). These parameter values may be set automatically, or an operator can set arbitrary values. The focal length F of the lens is known, and the x and y values of the barycentric coordinates of the actual shooting auxiliary points D ₁₁ , D ₁₂ ,..., D ₆₇ are known in step S 10. , Θ can be obtained. By substituting this θ into Equation 1, θ ₀ is obtained, and the coordinate values of R ₁₁ , R ₁₂ ,..., R ₆₇ are all shooting assistance points corrected using Equations 3 and 4 (hereinafter referred to as correction assistance points). x ₀ and y ₀ are calculated (step S13).

  A regression line is obtained for each horizontal auxiliary point set and each vertical auxiliary point set of the correction auxiliary points obtained in S13 (step S14). Then, the sum of the squares of the distances between each correction auxiliary point and its regression line is obtained in each group. Next, the sum of the squares of the distances of all the sets is calculated, and the sum is stored (step S15).

An example of the processing of S14 and S15 will be described. First, with respect to R ₁₁ , R ₁₂ ,..., R ₁₇ which is one of the horizontal correction auxiliary point sets, a regression line M ₁ of these correction auxiliary points is obtained (FIG. 3A). Here, the least square method is used as a method of calculating the regression line. The distances of the perpendiculars dropped from the respective points R ₁₁ , R ₁₂ ,..., R ₁₇ to the regression line M ₁ are obtained, respectively, and dM ₁ that is the sum of the squares of these distances is calculated. M _{2 to} M ₆ are also obtained for each of the remaining five horizontal direction correction auxiliary point sets, and sums dM _{2 to} dM ₆ of distance squares are obtained based on these. Similarly in the vertical direction, first, a regression line L ₁ of R ₁₁ , R ₂₁ ,..., R ₆₁ which is one of the vertical correction auxiliary point sets is obtained (FIG. 3 (B)), R ₁₁ , R ₂₁ , ..., find dL ₁ which is the sum of the squares of the distances of the perpendiculars dropped from R ₆₁ to L ₁ . For each of the remaining six vertical correction auxiliary point set also determine the L ₂ ~L _7, the sum dL ₂ through DL ₇ of the square of the distance based on these. The sum total of dM ₁ to dM ₆ and dL _{1 to} dL ₇ thus determined is calculated and stored.

  Until the set number of times is reached, a new set of parameters A and B are given to Equation 2 and the same processing is repeated (step S16).

After the above process is executed a set number of times, a set of A and B giving the minimum value among the sum of squares of the distances of the number of execution times stored is adopted as the parameters A and B of Equation 1 (step S17). ). The correction auxiliary points R ₁₁ , R ₁₂ ,..., R _{67 at} this time become ideal auxiliary points Q ₁₁ , Q _{12 ,.} Then, as described above, by applying Equations 1 to 4 that are correction equations to each pixel constituting the photographed image and moving the position of each pixel to the ideal pixel position, an image in which distortion is corrected Can be obtained.

  For the purpose of increasing the accuracy of the correction formula, a formula obtained by increasing the order of Formula 1 in the correction formula can also be used. In this case, the number of parameters increases, but the parameter value can be calculated using the same determination method as described above.

An example of the distortion correction figure which concerns on this invention. An example of the real image distortion correction image which concerns on this invention. An example of the regression line of the (A) horizontal direction correction auxiliary point group and (B) vertical direction correction auxiliary point which concern on this invention. Model of pinhole camera. 6 is a flowchart of a method for determining parameters A and B of a correction formula according to the present invention. An example of an ideal image when a distortion correction figure is arrange | positioned with respect to the optical axis of a lens (A) perpendicular | vertical, and (B) inclining.

Explanation of symbols

1 ... Distortion correction Fig. 2 Real-life distortion correction image

Claims (4)

By capturing a distortion correction diagram having a plurality of auxiliary point sets each including three or more auxiliary points arranged on a straight line on different straight lines, a live-action distortion correction image is obtained,
In the live-action distortion corrected image,
Obtaining the position of each live-action auxiliary point that is a point on the image corresponding to each auxiliary point;
An image distortion correction method characterized by obtaining a correction expression that gives a position of an ideal auxiliary point such that the actual shooting auxiliary points constituting each actual shooting auxiliary point group corresponding to each of the auxiliary point sets are arranged on a substantially straight line . In the distortion correction diagram, the auxiliary point is a point having a predetermined area,
The image distortion correction method according to claim 1, wherein, in the live-action distortion corrected image, the position of the live-action auxiliary point is the center of gravity of a plurality of pixels forming the real-life auxiliary point. 3. The image distortion correction method according to claim 1, wherein the auxiliary points are arranged in a grid pattern in the distortion correction diagram. By determining the parameters A and B in the following formulas 1 to 4, the correction formula is obtained.
The distortion of the image is corrected by obtaining the coordinates (x ₀ , y ₀ ) after distortion correction from the coordinates (x, y) of the pixels in the image before distortion correction using the correction equation. The image distortion correction method according to any one of 1 to 3.

here,
F: Lens focal length θ: Angle that the straight line connecting the center point of the lens and the coordinates of the pixel before correction is the optical axis θ ₀ : Straight line connecting the center point of the lens and the coordinates of the corrected pixel is the optical axis angle
A, B: Parameter

Images