JP2006010613A - Correcting method of image distortion - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a correcting method of image distortion capable of performing skewness aberration correction with high precision by establishing appropriately ideal coordinate values without the use of complicated devices. <P>SOLUTION: A photographed skewness correction image of digital data is acquired through camera-filming of skewness correction pictures having a central circle placed at the center and grid points arranged at an equal spacing, the positions of ideal grid points are decided from the ratio of the diameter of circle in the image concerned to the spacing of grid points, correction formula is derived on the basis of the positions of each photographed grid points and the positions of ideal grid points, and finally skewness aberration can be corrected with correction formula concerned. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a method of correcting distortion of an image photographed by a camera and converted into digital data, and more particularly to correction of distortion caused by distortion aberration of a camera lens. Note that the image in the present invention is not only a digital data image obtained by photographing with a digital still camera and a digital video camera, but also an image in which photographs and videos taken by an optical still camera and an optical video camera are converted into digital data. Shall also be included.

  2. Description of the Related Art In recent years, techniques for measuring the size of a subject or collating an object or a person by processing and analyzing data of an image captured by a digital camera have been developed. Using a digital camera, it is possible to safely and quickly measure the length of an object that cannot be directly measured, and in the technical field such as collation, it has traditionally relied on human power and required labor. Since the matching operation is automated, a result can be obtained in a very short time.

  However, when the above analysis is performed using an image photographed by a camera, distortion existing in the camera lens becomes a problem. If there is distortion in the lens, the image taken through the lens will not be similar to the actual subject and will have distortion. The main cause of this distortion is the distortion of the lens, also called distortion, which tends to be greater at the periphery of the lens than at the center of the lens. If the lens has distortion, the image of the subject in the captured image is different from the shape of the actual image of the subject, so that it is impossible to obtain a numerical value such as a length or a correct analysis result. In particular, when image matching is performed on a certain target, it is assumed that image distortion and pixel position deviation exist only to a very small extent that can be clearly judged to be the same or different by the human eye. However, it is inconvenient if the computer automatically identifies the difference. Therefore, it is desirable that this distortion is corrected and removed as much as possible.

  In order to remove the distortion, a lens having a small distortion may be used. However, such a lens has a problem that it is expensive. Alternatively, it is possible to correct distortion by using a plurality of lens configurations. However, this is not only expensive as compared with a single lens camera, but also increases the size and weight.

  Therefore, a technology has been developed that corrects distortion by performing image processing so that an image captured by a camera and acquired as digital data becomes an ideal image that is similar to an actual subject without distortion. I came. This distortion correction by image processing is generally performed by obtaining a distortion aberration correction expression representing distortion characteristics specific to a lens based on a plurality of points such as a plurality of lattice points and polygonal vertices. As an example of such a correction formula, Non-Patent Document 1 proposes a correction formula that uses a pinhole camera model and obtains distortion correction as geometric transformation.

  As an example of a conventional correction method, Patent Document 1 discloses a technique related to a distortion aberration correction method for the purpose of improving measurement accuracy in image measurement. A calibration board on which a plurality of target marks with known coordinate values are displayed is photographed with a digital camera, and a multidimensional correction formula is obtained based on the measured coordinate values of the photographed image and the known coordinate values, and correction is performed. Do.

JP 2001-133223 A ([0008]) Katsumi Kankai and two others, “Distortion correction and panoramic images by image processing”, “Ricoh Technical Report No.23, SEPTEMBER, 1997”, pp.47-49.

  When obtaining a distortion correction formula based on coordinate values such as lattice points and polygon vertices, it is important to set an ideal coordinate value after correction that there is no distortion. If this ideal coordinate value is appropriately determined, an appropriate correction formula can be obtained.

  In Patent Document 1, it is assumed that the coordinate value of the target mark is known. However, after shooting the target mark, when comparing the image height measurement value on the imaging screen and the image height coordinate value based on the actual measurement, these values can only be compared in the same coordinate system, The image height measurement value on the photographing screen obtained in pixel units must be converted into the actual length or vice versa. This conversion is performed based on, for example, the relationship between the number of pixels constituting the target mark on the shooting screen and the known coordinate value. Since there is distortion on the shooting screen, the coordinate value of the shot image is changed. An error will be included when converting to the actual coordinate system.

  Therefore, the problem to be solved by the present invention is to provide an image distortion correction method capable of appropriately setting ideal coordinate values and performing distortion aberration correction with high accuracy without using a complicated apparatus. That is.

  In order to solve the above-mentioned problem, the inventor of the present application has examined the characteristics of distortion aberration in detail, and as a result, the diameter of the center circle arranged at the center of the distortion correction diagram, which is the smallest distortion portion in the captured image. As a reference, it was conceived that the ideal position of each lattice point arranged at equal intervals is appropriately determined, and distortion correction is performed with high accuracy.

  An image distortion correction method according to the present invention made based on such considerations includes a distortion correction diagram having a plurality of grid points arranged at equal intervals and a center circle arranged at the center and larger than the grid points. A live-action distortion correction image is obtained by photographing with a camera, and in the real-life distortion correction image, the diameter of the real-shot center circle and the barycentric position of each real-shot grid point are obtained, and the diameter of the center circle and the lattice point interval are obtained. An ideal centroid position of each grid point is determined based on the ratio, a correction formula is obtained based on the centroid position and the ideal centroid position, and image distortion is corrected based on the correction formula.

  In this case, the area of the center circle may be obtained in the above-described real-image distortion corrected image, and the diameter of the center circle may be obtained from this area. In this case, the diameter of the circle can be obtained with higher accuracy.

ここで、
F：レンズの焦点距離
θ：レンズの中心点と補正前の画素の座標を結ぶ直線が光軸となす角度
θ₀：レンズの中心点と補正後の画素の座標を結ぶ直線が光軸となす角度
A、B：パラメータ Further, in the present invention, based on the following formulas 1 to 4 proposed in Non-Patent Document 1, the coordinates (x ₀ , Image distortion correction can be performed by obtaining y ₀ ) by the following formulas 1 to 4.

here,
F: Lens focal length θ: Angle that the straight line connecting the center point of the lens and the coordinates of the pixel before correction is the optical axis θ ₀ : Straight line connecting the center point of the lens and the coordinates of the corrected pixel is the optical axis angle
A, B: Parameter

Formulas 1 to 4 will be described with reference to FIG. 3 which is a model diagram of a pinhole camera. C is a pinhole point of the pinhole camera, and P is an object point in a three-dimensional space. P _d and P ₀ are the image forming points of P when there is a real image and there is distortion and when it is ideal and there is no distortion, and the angles formed by these and the optical axis (Z axis) are θ and θ ₀ . To do. Then, the distortion aberration of the lens is modeled by the polynomial expression 1 to obtain a distortion correction formula. Equations 2 to 4 are relational expressions clearly established by the geometric features of the model diagram depicted in FIG. In other words, in Equations 1 to 4, if parameters A and B in Equation 1 are determined, a distortion correction equation can be obtained. A method for obtaining these two parameters will be described later.

Using Equations 1 to 4, image distortion correction is performed as follows. First, in an image having distortion, when the coordinates of a certain pixel are (x, y), these x and y are substituted into Equation 2 to obtain θ. If this θ is substituted into Equation 1, the value of θ ₀ can be obtained. Then, by substituting the values of θ ₀ and x and y into Equations 3 and 4, respectively, the corrected coordinates (x ₀ , y ₀ ) are determined. By performing this operation for all the pixels of the image, an image with corrected distortion can be obtained.

ここで、H：実写中心円の中心と実写格子点との距離（以下、実写格子点高とする）、H₀：理想中心円の中心と理想格子点との距離（以下、理想格子点高とする）である。 In the present invention, a multidimensional formula determined based on the optical distortions T and H ₀ given by Formula 5 can also be used as the correction formula.

Here, H: distance between the center of the live-action center circle and the live-action grid point (hereinafter referred to as the live-action grid point height), H ₀ : distance between the center of the ideal center circle and the ideal grid point (hereinafter referred to as the ideal grid-point height) ).

  In the distortion correction according to the present invention, it is assumed that the lens is axially symmetric with respect to the optical axis, and the distortion is also symmetric with respect to the optical axis. That is, the distortion aberration depends on the distance from the center.

  According to the image distortion correction method of the present invention, it is possible to easily set an appropriate ideal lattice point position, and to perform image distortion correction with high accuracy.

  Hereinafter, an image distortion correction method according to the present invention will be described. In the present invention, an image, which is data obtained by photographing a distortion correction diagram, is referred to as a live-action distortion correction image, a center circle in the real-life distortion correction image is referred to as a live-action center circle, and a lattice point is referred to as a live-action lattice point. A center circle in an ideal image without distortion after correction is referred to as an ideal center circle, and a lattice point is referred to as an ideal lattice point.

First, the distortion correction diagram will be described. FIG. 1 is an example of a distortion correction diagram according to the present invention. In this distortion correction FIG. 1, a plurality of lattice points P ₁₁ , P ₁₂ ,... Arranged at equal intervals and one central circle P ₀ arranged at the center of the entire figure are drawn. Further, the diameter d _p of the center circle P ₀ and the lattice point interval connecting the centers of the lattice points have a predetermined ratio. In the distortion correction FIG. 1 of FIG. 1, the diameter d _p of the center circle P ₀ is a, the lattice point interval is 2a, and the ratio thereof is 1: 2.

  A lattice point is a point having a predetermined area. That is, each point of the live-action grid point captured as data is composed of a plurality of pixels having a certain area. Therefore, in the present invention, in order to treat such a lattice point as a point, the center of gravity of a plurality of pixels forming the lattice point is set as the position of the lattice point. In addition, since the lattice point includes a lot of distortion as it becomes larger, the lattice point is set to a size that allows the position of the center of gravity to be acquired in the actual image distortion correction image. For example, when the size of the entire image is 480 × 640 pixels, it is preferable that the actual shooting grid point is composed of 60 to 100 pixels. Further, although the accuracy of correction increases as the number of lattice points included in the distortion correction diagram increases, the calculation time increases at the same time.

The center circle P ₀ is a circle larger than the lattice point. However, if the center circle P ₀ is too small, the accuracy of obtaining the diameter of the actual photograph center circle is lowered. However, on the contrary, the distortion increases and increases from the center to the periphery of the lens. For this reason, the center circle P ₀ has a size that can accurately measure the diameter and area of the real-image distortion-corrected image and that hardly includes distortion inside.

In the case of digital image values of a pixel is binary, in order to properly determine the size of the center circle P ₀ described above, it is possible to use, for example, the following method. First, a digital circle which is a pseudo circle on a digital image is considered, and the number of pixels from the pixel closest to the center to the outermost pixel in the digital circle is defined as a digital radius. On the other hand, the number of pixels constituting the circle is defined as the digital area of the circle, and the radius is calculated backward based on the digital area. Then, an absolute value of a difference between these two radii is obtained to obtain an error. As an example, for a digital circle with a digital radius of 3, the digital area is 29. Dividing the value of this digital area by π and taking the positive square root gives the radius of about 3.038. Therefore, the error is 0.038. When errors are obtained for various digital radii in this way and each value is replaced with a conversion error that is a relative error with respect to a certain number of pixels, the conversion error increases the diameter (ie, area) of the digital circle. It gradually becomes smaller as it goes. This means that the area of the circle on the actual image distortion corrected image is increased, and the accuracy of the diameter value acquired based on the area is increased. When the conversion error is less than 1.0, since the pixel value does not change on the digital image, the error is not detected. Therefore, the size of the center circle P _{0 in} the distortion correction diagram is set to a size having a number of pixels such that the conversion error is less than 1.0 in the actual image distortion correction image. For example, when the entire image is 480 × 640 pixels, it is preferable that the center circle P ₀ is composed of 3000 pixels (radius is about 30 pixels).

Further, since the correction according to the present invention is based on the premise that the distortion is axially symmetric with respect to the optical axis, the distortion at the same distance from the center of the lens is equal. Therefore, in the distortion correction diagram, when the center of the center circle P ₀ is arranged at a position that is the center of rotational symmetry of all the lattice points, the lattice points having different distortion aberrations in the distortion correction image are within one quadrant and the quadrant thereof. Is limited to only the lattice points having different distances from the center of the central circle P ₀ . Therefore, in order to analyze more lattice points among the lattice points drawn in the distortion correction diagram, the center of the central circle P ₀ is not the center of rotational symmetry of all lattice points, that is, All the lattice points are translated so that the centers of all the lattice points and the center of the center circle P ₀ do not coincide.

And the distortion correction figure comprised as mentioned above is image | photographed with a camera. At this time, if the distortion correction diagram is not perpendicular to the camera, the arrangement of the lattice points in the live-action distortion correction image is not evenly spaced, so the optical axis of the camera lens and the distortion correction diagram arranged in front of the camera Adjustment is performed so that the normals set from the center (that is, the center of the center circle P ₀ ) coincide with each other, and then photographing is performed. As an example of this adjustment method, a camera is installed on a rail arranged perpendicular to the distortion correction diagram, and the center circle is photographed multiple times while moving the camera back and forth with respect to the distortion correction diagram. It is possible to use a distortion correction diagram or a method of adjusting the angle of the rail so that the ratio of the length in the vertical direction and the length in the horizontal direction in the image is always kept at 1: 1.

  Furthermore, since the center of the optical design of the lens and the center of the light receiving element such as a CCD may not coincide with each other, the entire image is translated in such a way that the center of gravity of the live-action center circle becomes the origin of the coordinate system, Correct the coordinates of the real image distortion correction image.

In this way, in the live-action distortion corrected image in which the coordinate system is appropriately corrected, the diameter d _D of the live-action center circle D ₀ and the barycentric coordinates of the live-action grid points P ₁₁ , P ₁₂ ,. This diameter may be calculated based on the area of the live-action center circle D ₀ . The diameter of the circle can be obtained by taking the positive square root of the value obtained by dividing the area of the live-action central circle D ₀ by the circumference and doubling it. In this case, the error included in the diameter value is 1 / square root of the error when the diameter value is directly acquired from the image, so that a more accurate diameter value can be obtained.

Then, since the distortion of the image based on the distortion of the lens having the minimum at the center of the image, photographed in the center circle D ₀ based on the fact that hardly contained strain, a live-action center circle D ₀ ideal central circle Q _0, and the diameter d _D of live-action center circle D ₀ and diameter d _Q of the ideal center circle Q _0. In FIG. 1, since there is a predetermined ratio between the diameter d _p of the center circle P ₀ and the interval between the lattice points, the diameter d _Q of the ideal center circle Q ₀ and the interval between the ideal lattice points The same ratio must exist between the two. By using this condition, the interval between the ideal lattice points is determined by the diameter d _D of the actual shooting center circle D ₀ , and the coordinate values of the ideal lattice points Q ₁₁ , Q ₁₂ ,.

FIG. 2 shows an image obtained by superimposing the ideal grid points Q ₁₁ , Q ₁₂ ,... On the live-action grid points P ₁₁ , P ₁₂ ,. It is not a component of the real image distortion correction image and the ideal lattice point). If a correction formula is obtained such that the coordinate values of the centroids of the live-action grid points P ₁₁ , P ₁₂ ,... Become the coordinate values of the centroids of the ideal grid points Q ₁₁ , Q ₁₂ ,. From the image, the ideal coordinates of each pixel from which distortion has been removed can be obtained, and an image with corrected distortion can be created. If the ideal coordinate value does not become an integer, the luminance of each pixel is determined by performing pixel complementation by the nearest neighbor method, the bilinear method, the bicubic method, or the like. When the image is a color image, interpolation is performed for each pixel including color information.

  Hereinafter, a distortion correction method using a real-life distortion correction image will be specifically described as an example. In the first embodiment, a correction method using a pinhole camera model will be described. In the second embodiment, a correction method using optical distortion will be described.

  In this embodiment, distortion aberration correction is performed using Equations 2 to 4 determined based on the pinhole camera model and Equation 1 modeling distortion. Here, an example of a method for determining the parameters A and B of Equation 1 will be described. FIG. 4 shows a flowchart of a method for determining A and B.

First, the photographed distortion correction image 2, Stock Center Circle D ₀ diameter d _D and Stock grid points D _11, D _12, ..., and acquires the barycentric coordinates of D ₄₅ (step S10). The live-action center circle D ₀ diameter d each ideal lattice points in the case of the diameter d _Q of ideal central circle Q ₀ the _{D Q 11, Q 12, ...} , each coordinate of Q _45, the center of distortion correction FIG. 1 It determined based on the ratio of a circle having a diameter d _p and spacing of the grid points (step S11).

  Next, the number of executions of the calculation performed thereafter is set (step S12). Increasing the number of executions increases the possibility that more appropriate values of A and B can be obtained. Therefore, it is desirable to set as many as possible.

Here, parameters A and B having appropriate values are arbitrarily set and substituted into Equation 1 (step S13). Since the focal length F of the lens is known, and the x and y values of the barycentric coordinates of the live-action grid points D ₁₁ , D ₁₂ ,..., D ₄₅ are known in step S 10, by substituting these into equation (2) , Θ can be obtained. This θ is substituted into Equation 1 to obtain θ _0, and the coordinate values x ₀ and y ₀ of the corrected grid points are calculated using Equations 3 and 4 (Step S14). The distance between the corrected grid point and the corresponding ideal grid point is obtained for all points, the sum of the distances is calculated, and the sum is stored (step S15).

  Until the set number of times is reached, the values of parameters A and B that are newly set arbitrarily are given to Equation 2, and the same processing is repeated (step S16).

  After the above process is executed a set number of times, a set of A and B that gives the minimum value among the sums of the stored execution times is used as the parameters A and B of Equation 1 (step S17). Then, as described above, the correction formulas 1 to 4 are applied to each pixel of the photographed image, and the image in which the distortion is corrected is obtained by moving the position of each pixel to the ideal pixel position. Obtainable.

  For the purpose of increasing the accuracy of the correction formula, a formula obtained by increasing the order of Formula 1 in the correction formula can also be used. In this case, the number of parameters increases, but the parameter value can be calculated using the same determination method as described above.

  As another embodiment according to the present invention, a distortion correction method based on optical distortion and ideal lattice point height will be described.

For this correction, a correction formula is obtained by the following procedure.
(1) In the photographed distortion correction image 2, Stock Center Circle D ₀ diameter d _D and Stock grid points D _11, D _12, ..., and acquires the barycentric coordinates of D _45.
(2) The diameter d _D of the live-action central circle D ₀ is defined as the diameter d _Q of the ideal central circle Q ₀ , and the ideal lattice points Q ₁₁ , Q _{12 ,.} Determine the coordinates.
The procedures 1 and 2 so far are the same as steps S10 and S11 of the first embodiment.

(3) The real grid point height and the ideal grid point height of the real grid point are obtained, and based on these, the optical distortion T is obtained from Equation 5 for each grid point.
(4) As shown in FIG. 5, the points obtained in the procedure 3 are plotted with the ideal grid point height (unit: pixel) on the vertical axis and the optical distortion (unit:%) on the horizontal axis. In FIG. 5, since 20 lattice points are targeted, 21 points are plotted including an ideal central circle where the ideal lattice point height and the optical distortion are both zero.
(5) Find a multi-order regression curve that best approximates the curve connecting these points. For example, if the regression equation is a quintic equation, the parameters a, b, c, d, and e of the equation H ₀ = aT ⁵ + bT ⁴ + cT ³ + dT ² + eT are determined using a least square method or the like. Further, although the calculation amount increases, a higher-order regression curve may be set in order to improve accuracy.

  If the relational expression between the optical distortion and the ideal lattice point height is obtained as described above, the ideal lattice point height can be obtained from the actual captured lattice point height by using Equation 5, and the ideal height of all pixels in the captured image can be obtained. By obtaining the position, distortion aberration correction can be performed.

An example of the distortion correction figure which concerns on this invention. An example of the real image distortion correction image which concerns on this invention. Model of pinhole camera. 6 is a flowchart of a method for determining parameters A and B of a correction formula according to the present invention. The graph showing the relationship between the ideal lattice point height of the lattice point and the optical distortion in the real image distortion corrected image.

Explanation of symbols

1 ... Distortion correction Fig. 2 Real-life distortion correction image

Claims (5)

A photographed distortion correction image is obtained by photographing with a camera a plurality of lattice points arranged at equal intervals, and a distortion correction diagram having a central circle arranged at the center and larger than the lattice point,
In the live-action distortion corrected image, obtain the diameter of the live-action center circle and the gravity center position of each live-action grid point,
Determining the ideal center-of-gravity position of each grid point based on the ratio of the diameter of the central circle and the grid point spacing;
Obtain a correction formula based on the center of gravity position and the ideal center of gravity position,
An image distortion correction method comprising correcting an image distortion based on the correction formula.   2. The image distortion correction method according to claim 1, wherein an area of the central circle is acquired in the real-image distortion correction image, and a diameter of the central circle is determined based on the area. The distortion of the image is corrected by obtaining the coordinates (x ₀ , y ₀ ) after distortion correction from the coordinates (x, y) of the pixel in the image before distortion correction by the following formulas 1 to 4. The image distortion correction method according to claim 1 or 2.

here,
F: Lens focal length θ: Angle that the straight line connecting the center point of the lens and the coordinates of the pixel before correction is the optical axis θ ₀ : Straight line connecting the center point of the lens and the coordinates of the corrected pixel is the optical axis angle
A, B: Parameter 3. The image distortion correction according to claim 1, wherein the image distortion is corrected using a multidimensional expression determined based on the optical distortion T and H ₀ given by Expression 5 as the correction expression. Method.

here,
H: Distance between the center of the live-action center circle and the live-action grid point
H ₀ : Distance between the center of the ideal center circle and the ideal grid point It is a distortion correction diagram used for image distortion correction,
The distortion correction figure characterized by having a plurality of lattice points arranged at equal intervals and a center circle arranged at the center and larger than the lattice points.

Images