JP4872890B2 - Image distortion correction method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image distortion correcting method for correcting distortion about an image photographed by a camera using a lens with a distortion aberration. <P>SOLUTION: In a step S2, it is determined whether a certain pixel position (u<SB>di</SB>, v<SB>dj</SB>) of the image photographed by the camera using a lens with a distortion aberration is included in a circle Cd including an area where point group data on an image surface are present. In a step S3, an image coordinate (u<SB>i</SB>, v<SB>j</SB>) is obtained by applying a distortion correction function by polynomial approximation about the pixel position (u<SB>di</SB>, v<SB>dj</SB>) determined to be included in the circle Cd. In a step S4, an image coordinate (u<SB>i</SB>, v<SB>j</SB>) is obtained by applying a distortion correction function by logarithmic approximation about the pixel position (u<SB>di</SB>, v<SB>dj</SB>) determined not to be included in the circle Cd. <P>COPYRIGHT: (C)2009,JPO&amp;INPIT

Description

  The present invention relates to an image distortion correction method for correcting distortion of an image taken by a camera using a lens having distortion aberration.

  In general, an image captured by a camera using a wide-angle lens is distorted, and thus image processing for correcting the distortion is necessary to accurately grasp the photographic subject. For example, various devices for displaying the field of view of the back and sides of an automobile in the passenger compartment have been proposed and are commercially available. In such an apparatus, various measures are taken to make the display image easy to see. When a wide-angle lens is used in the camera, the display image is distorted, and measures for this need to be taken. For example, regarding a parking assistance device, a so-called back guide monitor is commercially available that displays a predicted travel locus of a host vehicle superimposed on a captured image for parking guidance. Among these devices, there is also a device that processes the position and shape of the predicted travel locus drawn on the display device so as to be intentionally distorted in accordance with the distortion characteristics of the lens.

  In relation to such an apparatus, Patent Document 1 discloses a predicted vehicle trajectory display apparatus, and a method for correcting distortion by a wide-angle lens is proposed as fish-eye conversion.

  Patent Document 2 discloses a method for correcting image distortion, and an n-th order Taylor expansion, that is, a method by polynomial approximation is proposed as a distortion correction function.

  Further, Patent Document 3 discloses a method for correcting distortion of an image taken by a camera using a wide-angle lens, and an MMF model (Morgan-Mercer) including an acronym of Morgan, Mercer, and Flodin as a distortion correction function. -Flodin model) has been proposed.

The coefficient of the distortion correction function based on the polynomial approximation is obtained by photographing a space in which a target (teaching point) having a known actual size (three-dimensional coordinate value) is arranged with a camera, and the two-dimensional coordinate value of the target projected on the photographed image. And is estimated by a non-linear optimization method based on the point cloud data in which the three-dimensional coordinate value and the two-dimensional coordinate value are associated with each other.
JP-A 64-14700 JP 2001-133223 A JP 2006-127083 A

  In Patent Document 1, the distortion characteristics of a wide-angle lens are modeled by an exponential function, but there is no mention of accurately correcting the characteristics of a lens made of an aspheric surface. Also, there is no disclosure of a method for accurately correcting image distortion with respect to actual lens distortion characteristics.

  In Patent Documents 2 and 3, a nonlinear optimization method is used based on point cloud data of a space in which a target having a known three-dimensional coordinate value is placed and a target two-dimensional coordinate value extracted from an image of the space. It is used to estimate the coefficient of the distortion correction function.

  However, since a camera using a wide-angle lens has a wide viewing angle, it is difficult to shoot an image by arranging a target in the entire shooting space. That is, in an image taken with a camera using a wide-angle lens, the space where the target exists is biased toward the center of the image, and the point cloud data used for estimating the coefficient of the distortion correction function is also biased toward the center of the image. In addition, even if you can shoot an image with the target placed in the entire space to obtain point cloud data at the periphery of the image, the wide-angle lens has a large distortion aberration, so the coefficient of the distortion correction function is estimated. In some cases, the convergence of the solution cannot be obtained. Then, the point cloud data used for estimating the coefficient of the distortion correction function is biased toward the center of the image.

  Patent Documents 2 and 3 disclose a method for improving the accuracy of interpolation within a region where point group data used for estimating the coefficients of a polynomial exists, that is, the accuracy of interpolation, for a distortion correction function by polynomial approximation. It is not possible to improve the correction of the area where the point cloud data used for the estimation of the coefficient does not exist, that is, the accuracy of extrapolation, and no correction method for this area is disclosed.

  For example, the central part of the image of the camera used for the back guide monitor is an image with less distortion that makes it easy for the driver to grasp the sense of distance from the viewpoint of vehicle alignment and collision prevention. In addition, the peripheral portion of the image is a distorted image that can ensure a wide field of view so that it is easy to grasp the state of other vehicles and people in the vicinity. In order to obtain such an image, the lens used in the camera of the back guide monitor has a small distortion at the center and a larger distortion toward the periphery.

  With respect to an image shot with such a lens, the conventional image processing method and its apparatus estimate each coefficient of a polynomial that is a distortion correction function using point cloud data at the center of the image. The distortion cannot be corrected sufficiently.

  An object of the present invention is to provide an image distortion correction method that appropriately performs distortion correction on an image captured by a camera using a lens with distortion.

  In order to solve the above problems, in the present invention, an image captured by a camera using a distorted lens is nonlinear in a region included in a circle centered at the intersection of the optical axis of the camera and the image plane. Distortion images are corrected by applying a distortion correction function based on polynomial approximation for which a coefficient is obtained by the least square method, and distortion correction functions based on logarithmic approximation for which a coefficient is obtained by a nonlinear least square method are applied to the area not included in the circle. Thus, a distortion correction method for an image is provided, wherein the distortion image is corrected.

  ADVANTAGE OF THE INVENTION According to this invention, the distortion correction method of the image which performs distortion correction appropriately about the image image | photographed with the camera using the lens with a distortion can be provided.

  An image distortion correction method according to the present invention will be described with reference to the accompanying drawings.

  First, a coordinate system that describes the relationship between the position of a point in the three-dimensional space photographed by the camera and the position on the two-dimensional image plane where the point is projected on the image plane will be described.

  FIG. 1 is a diagram for explaining the geometric relationship between an image photographed by a camera and a space.

A perspective projection by a camera using an ideal lens without distortion can be modeled as shown in FIG. A three-dimensional coordinate system representing a space is used in which the optical center of the camera is the origin, the Z axis is aligned with the optical axis direction of the camera, and the X and Y axes are parallel to the horizontal and vertical directions of the image. . Such a coordinate system is called a camera coordinate system. On the other hand, as a coordinate system representing the position on the image plane, the intersection point where the optical axis of the camera and the image plane intersect is set as the origin, and the x axis and the y axis are determined in the horizontal direction and the vertical direction of the plane image, respectively. At this time, the relationship of [Equation 1] is established between the position (X, Y, Z) in the space and the position (x, y) on the image. Here, the focal length is normalized to 1.
[Equation 1]
x = X / Z
y = Y / Z

Next, the relationship between the space and the image plane is more generalized. In general, a coordinate system representing a space uses a world coordinate system fixed at an appropriate position in the space. If the position of the space expressed in the world coordinate system is (Xw, Yw, Zw), the relationship of [Equation 2] between the same point and the position (X, Y, Z) expressed in the camera coordinate system There is.

On the other hand, as for the position on the image plane, generally, the origin is set at a certain appropriate position on the image plane, and a pixel is used as a unit of length. Such general-purpose coordinates representing a position on the image plane are set as image coordinates (u, v), and coordinates (x, y) are normalized image coordinates (normalized image coordinates) to be distinguished from the image coordinates (u, v). Call it. At this time, the relationship between the normalized image coordinates and the image coordinates is expressed by [Equation 3].

From [Equation 1] to [Equation 3], as in [Equation 4], the relationship between general world coordinates (Xw, Yw, Zw) and image coordinates (u, v) is derived.

[Equation 4] can represent the relationship between the position of an arbitrary space and the position on the image plane when it is projected onto the image plane. Here, the focal length f of the camera, the physical interval δu in the horizontal direction, the physical interval δv in the vertical direction, and the image center (c _u , c _v ) are constants determined by the camera. It is called an internal parameter K (internal parameter, intrinsic parameter). On the other hand, R and t are constants determined by the position and orientation of the camera with respect to the world coordinates, and are called external parameters (external parameters, extrinsic parameters) of the camera. The camera internal parameters and external parameters are collectively referred to as camera parameters.

  2A and 2B are diagrams for explaining lens distortion, FIG. 2A is a diagram for explaining radial distortion of the lens, FIG. 2B is a diagram showing actual spatial information, and FIGS. D) is a diagram showing an example of distortion.

  In a wide-angle lens, it is necessary to consider distortion in the radial direction and tangential direction of the lens. As shown in FIG. 2A, the radial distortion occurs when the light beam emitted from the object A passes through the optical center at the incident angle θ and the output angle θ ′ is not equal to the incident angle θ. Tangential distortion occurs due to a shift or inclination of the center position between a plurality of lenses constituting the lens. The tangential distortion is smaller than the radial distortion and is often negligible in practice, and is omitted in the modeling for the sake of simplicity.

The distortion aberration can be modeled as [Equation 5].

When a certain point in a three-dimensional space is photographed to obtain image coordinates, an ideal lens without distortion is projected at a position (u, v) represented by the image coordinates, and a lens having distortion aberration is represented by the distortion image coordinates. Projected to the represented position (u _d , v _d ). κ ₁ and κ ₂ are coefficients representing lens distortion. Also, the position (u, v) projected on the image coordinates with an ideal lens without distortion aberration is the predicted position, and the position (u _d , v _d ) projected on the distortion image coordinates with a lens with distortion aberration is the observation position. Call it.

  FIGS. 2C and 2D show examples of images obtained by photographing the square lattice shown in FIG. 2B with a camera using a lens having radial distortion.

  An image taken with a camera using a lens with distortion is called a distorted image.

A perspective projection of a camera using a lens having a distortion aberration as shown in FIGS. 2C and 2D is modeled as shown in [Equation 5], and the predicted position of the target and the actual observation position based on this are modeled. The parameters were estimated so as to minimize the sum of squares of the geometric distance (reprojection error), and the actual spatial information as shown in FIG. 2B was used using an ideal lens without distortion. The image is corrected to the perspective projected by the camera. In the estimation of this parameter, the estimation by the nonlinear least square method expressed by [Equation 6] is called bundle adjustment. The parameters are composed of camera parameters and κ ₁ and κ ₂ which are coefficients representing lens distortion.

  Here, the point group data in which the coefficient of the distortion correction function is estimated, that is, the three-dimensional coordinate value used for bundle adjustment and the two-dimensional coordinate value are associated with each other is obtained by photographing a target having a known three-dimensional coordinate value. It is obtained by extracting the two-dimensional coordinate value of the target projected above and associating the coordinates of the three-dimensional coordinate value and the two-dimensional coordinate value.

  First, as a target having a known three-dimensional coordinate value, for example, a 500 mm square cross carpet is placed on the floor surface of the shooting space, and the space is shot with a camera to obtain a distorted image. Next, a plurality of these two-dimensional coordinate values are extracted from the distorted image with the corners (grid points) of the lattice patterns of the cross carpet as targets. Next, the origin in the three-dimensional space where the photographing space exists is determined, for example, as the lattice point of the cross carpet in front of the center on the image plane, and the three-dimensional coordinate value of each lattice point is obtained. From the three-dimensional coordinate value and the two-dimensional coordinate value, point group data in which corresponding lattice points are associated with each other is obtained. Further, by appropriately changing the distance between the camera and the floor, point cloud data in which the corresponding grid points are associated with each other is obtained from the three-dimensional coordinate values and the two-dimensional coordinate values of the respective grid points in the vertical direction of the image. .

  Based on this point cloud data, the camera internal parameters, external parameters, and coefficients representing lens distortion are estimated by a nonlinear optimization method.

  Here, an image plane onto which a target with a known three-dimensional coordinate value is projected is projected by a region of a circle C including a region where the target projected on the normalized image coordinates exists and a lens having distortion aberration. Consider an area of a circle Cd that includes an area where a target projected onto the distorted image coordinates exists. The region where the target exists is a region where point cloud data that can be adopted in consideration of the convergence of the solution of the nonlinear least square method in estimating the coefficient of the polynomial in the distortion correction function by the polynomial approximation exists.

The radius Rmax of the circle C including the region where the target projected on the normalized image coordinates exists is expressed in the camera coordinate system by [Expression 2] from the position (Xw, Yw, Zw) expressed in the world coordinate system. Radius (x, y) obtained from the position (x, y) of the target converted to the normalized image coordinates by [Equation 1] from this position (X, Y, Z). ² + y ² ) ^1/2 maximum value. That is, on the distorted image plane projected by the distortion aberration lens, the target projected on the image coordinates obtained by transforming [Equation 3] from the target position (x, y) projected on the normalized image coordinates. The position (u, v) of the image is obtained, and the radius {(u _d −c _u ) obtained from the position (u _d , v _d ) of the distorted image coordinates obtained from [5] from this position (u, v). ² + (v _d −c _v ) ² } ^1/2 maximum value. Radius {(u _d −c _u ) ² + (v _d −c _v ) ² } ^{1 / including} the region where the target projected on the distorted image coordinates projected by the lens having this distortion aberration exists ² is called a radius Rdmax.

Further, the radius (x ² + y ² ) ^1/2 obtained from the target position (x, y) projected on the normalized image coordinates is called the image height D on the normalized image coordinates, and is projected by a distorted lens. The radius {(u _d −c _u ) ² + (v _d −c _v ) ² } ^1/2 obtained from the position (u _d , v _d ) of the distorted image coordinates obtained is set as the image height Dd on the distorted image coordinates. Call it.

  Here, an example of estimating the distortion correction function represented by [Equation 6] by the nonlinear least square method will be described.

  FIG. 3 is a diagram illustrating an example of a distorted image.

  FIG. 4 is a diagram for explaining an example of an image obtained by correcting the distorted image shown in FIG. 3 using a distortion correction function based on polynomial approximation.

  FIG. 5 is a diagram for explaining the relationship between the distorted image shown in FIG. 3 and a circle including a region where the target exists.

  FIG. 6 is a diagram for explaining the relationship between a distortion correction function based on polynomial approximation and a circle including a region where a target projected on normalized image coordinates exists.

  As shown in FIG. 3, in a distorted image, when a three-dimensional coordinate value is known as a target, for example, when a 500 mm square cross carpet is placed on the floor surface of the shooting space and the image is acquired, The edges are distorted and projected in a curved line. This distortion of the image is greatly observed at the peripheral portion of the image, and particularly noticeable at the left and right lower ends of the image.

  As shown in FIG. 4, in the central portion of the image obtained by estimating the coefficient of the distortion correction function by the bundle adjustment represented by [Equation 6] and correcting the distortion image, the distortion of each side of the substantially straight line of each lattice is as shown in FIG. A natural image in which is corrected is obtained. However, in the distorted image shown in FIG. 3, it can be observed that the distortion of the substantially linear side of the lattice is not sufficiently corrected on the lower left and right sides of the image in which the distortion is significant.

  An image obtained by correcting a distorted image using a distortion correction function is called a corrected image.

  As shown in FIG. 5, the lattice points + used as point cloud data in estimating the coefficients of the distortion correction function in FIG. 4 are concentrated in the central portion of the distorted image and used as targets in the peripheral portion of the distorted image. There are no grid points that can be created. The area including the area where the grid point + exists is a circle Cd having a radius Rdmax, and the radius of the circle C projected on the normalized image coordinates is the radius Rmax.

  As shown in FIG. 6, the target exists in a circle C having a radius Rmax in normalized image coordinates. When the image height D obtained from the two-dimensional coordinate values on the normalized image coordinates on which the target ◇ is projected is represented by a broken line T, the target ◇ is projected on the broken line T. A solid line A represents a distortion correction function based on a polynomial. The distortion correction function based on the polynomial appropriately corrects the distortion image in the area of the radius Rmax where the target ◇ exists, and the distortion correction function based on the polynomial in the area where the target does not exist. However, it is observed that the distorted image cannot be corrected appropriately.

In FIG. 6, the horizontal axis is a value of radius (x ² + y ² ) ^½ with the origin of the intersection of the normalized coordinate and the optical axis, that is, the image height D, and the vertical axis is expressed by [Equation 7]. The rate of change h between the image height Dd at the distorted image coordinates and the image height D in the corrected image.
[Equation 7]
Change rate _{^{h = [{(u d -c}} u) 2 + (v d -c v) 2} 1/2 - {(u-c u) 2 + (v-c v) 2} 1/2] / {(U−c _u ) ² + (v−c _v ) ² } ^1/2
= Κ ₁ × r ² + κ ₂ × r ⁴

  FIG. 7 is a flowchart for explaining an image distortion correction method according to the present invention.

As shown in FIG. 7, in step S1 of the image distortion correction method according to the present embodiment, pixel positions (u _di, v _dj ) for performing distortion correction from an image (distortion image) captured by a camera using a wide-angle lens. ) Designates the position (u _d1, v _d1 ) of the start pixel.

Next, in step S2, it is determined whether or not the pixel position (u _di, v _dj ) is included in a circle Cd having a radius Rdmax that includes an area where point cloud data exists on the distorted image plane. When the pixel position (u _di, v _dj ) is included in the circle Cd having the radius Rdmax, the process proceeds to step S3. In other cases, the process proceeds to step S4.

Next, in step S3, a distortion correction function by polynomial approximation is applied to the pixel position (u _di, v _dj ) determined to be included in the circle Cd having the radius Rdmax in step S2, and image coordinates (u _i , v _j ) is obtained.

Next, in step S4, a distortion correction function based on logarithmic approximation is applied to the pixel position (u _di, v _dj ) determined not to be included in the circle Cd having the radius Rdmax in step S2, and image coordinates (u _i , v _j ) is obtained.

Next, in step S5, it is determined whether the pixel position (u _di, v _dj ) is the end pixel position (u _dimax, v _djmax ). When the pixel position (u _di, v _dj ) is the end pixel position (u _dimax, v _djmax ), the correction of the distorted image is finished to obtain a corrected image. In other cases, the process proceeds to step S6.

Next, in step S6, the pixel position (u _di, v _dj ) is advanced by one pixel in the u-axis direction or the v-axis direction, and the pixel position (u _{di + 1,} v _dj ) or pixel position (u _di, v _{dj + 1).} ). For example, when the image size is horizontal 640 pixels × vertical 480 pixels, imax = 640 and jmax = 480, and (u _d1, v _d2 ), (u _d1, v _d3 ) are sequentially executed every time processing is performed in step S6. ), (U _d1, v _d4 ),..., (U _d640, v _d478 ), (u _d640, v _d479 ), (u _d640, v _d480 ), and so on.

  The estimation of the coefficient of the expression approximating the distortion correction function is generally obtained from the point cloud data by the Levenberg-Marquardt method which is a nonlinear optimization method.

The coefficients of the polynomial are the focal length f of the camera represented by [Equation 2], [Equation 3], and [Equation 5], the physical interval δu of the pixels in the horizontal direction, and the physical distance of the pixels in the vertical direction. The interval δv, the coordinates (c _u , c _v ) in the image coordinate system at the center of the image, the 3 × 3 rotation matrix R, the three-dimensional translation vector t, and the coefficients κ ₁ and κ ₂ representing lens distortion. And is obtained by bundle adjustment represented by [Equation 6].

On the other hand, the coefficients of the logarithm are as follows: the focal length f of the camera represented by [Equation 2] and [Equation 3], the physical interval δu of the horizontal pixels, and the physical interval δv of the vertical pixels. Along with the coordinates (c _u , c _v ) in the image coordinate system at the center of the image, the 3 × 3 rotation matrix R, and the three-dimensional translation vector t, the distortion of the lens represented by [Equation 8] is expressed. consists factor kappa _ln1, kappa _ln2 Prefecture, determined by bundle adjustment.

The distortion of the wide-angle lens can be modeled as [Equation 8] by logarithmic expression.

In the distortion correction function based on the logarithmic approximation, the perspective projection of the camera using the lens having distortion aberration is modeled by the logarithmic expression as in [Equation 8], and the predicted position of the target based on this and the actual observation position are calculated. geometrical distance coefficient so as to minimize the sum of squares of (reprojection error) kappa _ln1, estimates the kappa _ln2. About this parameter estimation, estimation by the least squares method represented by [Equation 9] is performed.

  FIG. 8 shows the threshold radius value Rth of the region where the point cloud data is used for the estimation of the coefficient of the logarithmic expression in the distortion correction function by logarithmic approximation in the image distortion correction method according to the present invention, and the coefficient of the logarithmic expression. It is a flowchart explaining the method to estimate.

As shown in FIG. 8, in step S11, the coefficient of the polynomial is estimated by the nonlinear least square method expressed by [Expression 6], and expressed by [Expression 2], [Expression 3], and [Expression 5]. The focal length f, which is a coefficient of the polynomial, the physical distance δu between the horizontal pixels, the physical distance δv between the vertical pixels, and the coordinates (c _u , c _v ) in the image coordinate system at the image center A 3 × 3 rotation matrix R, a three-dimensional translation vector t, and coefficients κ ₁ and κ ₂ representing lens distortion are obtained.

  Next, in step S12, the threshold radius value Rth is set to zero.

Next, in step S13, the coefficient of the logarithm is estimated by the least square method represented by [Equation 9] using the point cloud data existing in the region from the threshold radius value Rth to the radius Rmax, and [Equation 8]. logarithmically coefficient kappa _ln1 represented by, determining the kappa _ln2. Note that the focal length f, the physical interval δu between the horizontal pixels, the physical interval δv between the vertical pixels, the coordinates (c _u , c _v ) in the image coordinate system at the center of the image, and 3 × The rotation matrix R of 3 and the three-dimensional translation vector t are obtained by the nonlinear least square method represented by [Equation 6] in step S11.

Next, in step S14, the determination coefficient ε ² of the least square method and the threshold value ε ² th are compared. If the coefficient of determination epsilon ² is less than or equal to the threshold epsilon ² th, the coefficient pairs formula in threshold radius value Rth at this time kappa _ln1, and ends the adoption and processing kappa _ln2. In other cases, the process proceeds to step S15. The threshold value ε ² th of the determination coefficient ε ² is set in consideration of the quality of the corrected image by the distortion correction function. For example, the threshold value ε ² th is set to about 0.001.

  Next, in step S15, an arbitrary value ΔRth is added to the threshold radius value Rth, and the process returns to step S13 again. The arbitrary value ΔRth is preferably greater than 0 and equal to or less than 1 which is the focal length on the normalized image coordinates, and is preferably about 0.2 to 0.3, for example.

  FIGS. 9 to 14 show the threshold radius value Rth of the point cloud data used for estimating the coefficient of the distortion correction function by logarithmic approximation in the image distortion correction method according to the present invention, the corrected image, It is a figure explaining the specific example of these relationships.

  9A to 14A are diagrams for explaining the relationship between the distortion correction function by logarithmic approximation and the threshold radius value Rth of the point cloud data used for estimating the coefficient of the logarithmic equation. ) Is a diagram showing an image corrected by a distortion correction function estimated for each threshold radius value Rth, and (C) is an overhead view obtained by converting (B).

9A to 14A, the horizontal axis is a value of radius (x ² + y ² ) ^1/2 with the origin at the intersection of the normalized coordinates and the optical axis, that is, the image height D, and the vertical axis Is the rate of change h between the image height Dd at the distorted image coordinates represented by [Equation 7] and [Equation 10] and the image height D in the corrected image.
[Equation 10]
Rate of change h = κ _ln1 × ln (r ² ) + κ _ln2

  As shown in FIG. 9A to FIG. 14A, the coefficient of the logarithmic formula is estimated by changing the threshold radius value Rth from 0 to 1.25 every 0.25. FIG. 9A is a diagram showing the relationship between the approximation by the logarithmic expression at the threshold radius value Rth = 0.0 and the targets ◇ and □, and FIG. 10A is the threshold radius value Rth = 0. .25 is a diagram showing the relationship between the approximation by the logarithmic expression and the targets ◇ and □, and FIG. 11A shows the approximation by the logarithmic expression at the threshold radius value Rth = 0.50 and the targets ◇ and □. FIG. 12 (A) is a diagram showing the relationship between approximation by the logarithmic expression at the threshold radius value Rth = 0.75 and the targets ◇ and □, and FIG. 13 (A) is a diagram showing the relationship. FIG. 14A is a diagram showing the relationship between the approximation by the logarithmic expression at the threshold radius value Rth = 1.00 and the targets ◇ and □, and FIG. 14A is the logarithmic expression at the threshold radius value Rth = 1.25. It is the figure which showed the relation between approximation and target ◇ and □The target □ is point cloud data used for estimating the coefficient of the logarithmic expression, and the image height D has a relationship of threshold radius value Rth ≦ image height D ≦ radius Rmax.

As shown in FIG. 13A, the determination coefficient ε ² = 0.0003 when the threshold radius value Rth = 1.00, and the threshold value ε ² th = 0.001 or less. At this time, the coefficient of the logarithmic type kappa _ln1, kappa _ln2 are respectively -0.171, a -0.1931. The logarithmic expression obtained by this coefficient corrects a distorted image outside the region of the image height D = 0 to 2.35 where the targets ◇ and □ exist, that is, in the range of the image height D = 2.35 to 5. The corrected image is shown in FIGS. 9 to 14B. As shown in (C) of FIG. 9 to FIG. 14, when comparing the overhead views obtained by converting from the corrected image, in the left and right lower parts of the image, that is, in the left and right spaces near the camera in the captured space, the coefficient of determination is compared with the corrected image obtained by the threshold epsilon ² th is greater than the distortion correction function, better coefficient of determination is obtained corrected image following distortion correction function threshold epsilon ² th is more ideal It can be seen that this is close to the perspective projection obtained with a simple lens.

  In the nonlinear optimization, if the peripheral portion of the distorted image is included in the point cloud data, the optimization does not converge. For this reason, the area | region which extracts point cloud data is limited to the center part of a distorted image. The limitation of this area is an empirical value, and in this example, the point up to the image height D = 2.35 in the normalized image coordinates is given. That is, Rmax = 2.35. Since this value varies depending on the focal length and the degree of distortion of the lens used, it is not uniquely determined.

  In other words, the image distortion correction method according to the present embodiment is included in a circle Cd including a region where a target whose three-dimensional coordinate values used for estimating the coefficient of the distortion correction function are known is projected onto the distortion image. In a region, that is, a region included in a circle Cd having a radius Rdmax centered on the intersection of the optical axis of the camera and the distorted image plane, a distortion correction function by polynomial approximation obtained by a nonlinear least square method is applied. In a region where the distorted image is corrected and not included in the circle Cd, point cloud data in which the image height D of the coordinates where the target is projected onto the normalized image coordinates is within the range of threshold radius value Rth ≦ image height D ≦ radius Rmax A corrected image is obtained by correcting a distorted image by applying a distortion correction function by logarithmic approximation obtained by a least square method based on the above.

  FIG. 15A is a view showing an example of a corrected image by the image distortion correction method according to the present invention, FIG. 15B is an overhead view obtained by converting the corrected image of FIG. (B) is an overhead view in which the optical axis of the overhead view is biased to the left side of the image.

  FIG. 16A is a diagram illustrating an example of a corrected image by a distortion correction method using a polynomial distortion correction function, and FIG. 16B is an overhead view obtained by converting the corrected image of FIG. C) is an overhead view in which the optical axis of the overhead view of (B) is biased to the left side of the image.

  15A and 15B are compared with FIGS. 16A and 16B, according to the image distortion correction method according to the present embodiment, the periphery of the distorted image, particularly the lower left and right sides of the image. In this area, it can be clearly observed that the accuracy of correction is improved. For example, by using the distortion correction method according to the present embodiment for a back guide monitor, a vehicle parked in a white line of a parking lot adjacent to the left or right side of the own vehicle or an adjacent lane is corrected for distortion. Can be confirmed with images.

  Further, when comparing FIG. 15C and FIG. 16C, when the camera used for the back guide monitor cannot be placed in the center of the own vehicle and is mounted shifted in either the left or right direction, for example, Even if it is attached to the rear right side of the vehicle, it is parked on the white line or adjacent lane of the parking lot near the vehicle on the rear left side of the vehicle far from the camera, that is, the lower right side of the image. The vehicle can be confirmed with a natural image with distortion corrected in a wider range.

  According to the image distortion correction method according to the present embodiment, distortion correction can be appropriately performed on an image captured by a camera using a lens with distortion.

  FIG. 17 is a block diagram illustrating a configuration of an image distortion correction apparatus according to an embodiment of the present invention.

  As shown in FIG. 17, the image distortion correction apparatus 1 according to the present embodiment includes an imaging unit 2 that captures an image and an image processing unit 3 that corrects distortion of the image captured by the imaging unit 2. The image distortion correction apparatus 1 can also include display means 4 for displaying an image whose distortion has been corrected by the image processing means 3. That is, the display unit 4 displays an image whose distortion has been corrected by the image processing unit 3. Instead of the display means 4, it may be configured so as to be provided to the processing means in the next stage without displaying an image whose distortion has been corrected.

  The image processing unit 3 includes a distortion correction unit 5. The distortion correction unit 5 is a distortion correction method for an image according to the present embodiment, that is, a target having a known three-dimensional coordinate value used for estimating a coefficient of a distortion correction function for an image captured by the imaging unit 2 is distorted. In the region included in the circle Cd including the region projected on the image, that is, in the region included in the circle Cd having the radius Rdmax centering on the intersection of the optical axis of the imaging unit 2 and the distorted image plane, the nonlinear minimum In a region that is not included in the circle Cd by applying a distortion correction function by polynomial approximation obtained by a square method, the image height D of the coordinates at which the target is projected on the normalized image coordinates is the threshold. Distortion of an image by applying a distortion correction function by logarithmic approximation obtained by a least square method based on point cloud data in a range of radius value Rth ≦ image height D ≦ radius Rmax It is corrected.

  15A and 15B are compared with FIGS. 16A and 16B, according to the distortion correction apparatus 1 according to the present embodiment, the peripheral portion of the distorted image, particularly the lower left and right sides of the image is displayed. In the region, it can be clearly observed that the accuracy of correction is improved. For example, by using the distortion correction method according to the present embodiment for a back guide monitor, a vehicle parked in a white line of a parking lot adjacent to the left or right side of the own vehicle or an adjacent lane is corrected for distortion. Can be confirmed with images.

  Further, when comparing FIG. 15C and FIG. 16C, when the camera used for the back guide monitor cannot be placed in the center of the own vehicle and is mounted shifted in either the left or right direction, for example, Even if it is attached to the rear right side of the vehicle, it is parked on the white line or adjacent lane of the parking lot near the vehicle on the rear left side of the vehicle far from the camera, that is, the lower right side of the image. The vehicle can be confirmed with a natural image with distortion corrected in a wider range.

  According to the image distortion correction apparatus according to the present embodiment, distortion correction can be appropriately performed on an image captured by a camera using a lens with distortion.

  The image distortion correction method and the distortion correction apparatus according to the present invention are not limited to apparatuses mounted on a moving body such as a vehicle, but are applied to various image processing apparatuses using a wide-angle lens camera. be able to.

The figure explaining the geometric relationship between the image image | photographed with the camera, and space. It is a figure explaining the distortion aberration of a lens, (A) is a figure explaining the radial direction distortion aberration of a lens, (B) is a figure which shows actual spatial information, (C) and (D) are examples of distortion aberration. FIG. The figure explaining an example of a distorted image. The figure explaining an example of the image which correct | amended the distortion image shown by FIG. 3 by the distortion correction function by polynomial approximation. The figure explaining the relationship between the distorted image shown by FIG. 3, and the circle | round | yen which include the area | region where a target exists. The figure explaining the relationship between the distortion correction function by polynomial approximation, and the circle | round | yen which include the area | region where the target projected on the normalized image coordinate exists. 6 is a flowchart illustrating an image distortion correction method according to the present invention. Among the image distortion correction methods according to the present invention, a method for estimating a threshold radius value Rth of a region in which point cloud data is used for estimating a coefficient of a logarithmic expression in a distortion correction function by logarithmic approximation and a coefficient of the logarithmic expression FIG. In the image distortion correction method according to the present invention, a specific example of the relationship between the threshold value Rth of the point cloud data used for estimating the coefficient of the distortion correction function by logarithmic approximation and the corrected image will be described. (A) is a diagram for explaining the relationship between the distortion correction function by logarithmic approximation and the threshold radius value Rth = 0.00 of the point cloud data used for estimating the coefficient of the logarithmic equation; B is a view showing an image corrected by a distortion correction function estimated for each threshold radius value Rth, and (C) is an overhead view obtained by converting (B). In the image distortion correction method according to the present invention, a specific example of the relationship between the threshold value Rth of the point cloud data used for estimating the coefficient of the distortion correction function by logarithmic approximation and the corrected image will be described. (A) is a figure explaining the relationship between the distortion correction function by logarithmic approximation and the threshold radius value Rth = 0.25 of the point cloud data used in estimating the coefficient of the logarithm. B is a view showing an image corrected by a distortion correction function estimated for each threshold radius value Rth, and (C) is an overhead view obtained by converting (B). In the image distortion correction method according to the present invention, a specific example of the relationship between the threshold value Rth of the point cloud data used for estimating the coefficient of the distortion correction function by logarithmic approximation and the corrected image will be described. (A) is a diagram for explaining the relationship between the distortion correction function by logarithmic approximation and the threshold radius value Rth = 0.50 of the point cloud data used for estimating the coefficient of the logarithmic equation; B is a view showing an image corrected by a distortion correction function estimated for each threshold radius value Rth, and (C) is an overhead view obtained by converting (B). In the image distortion correction method according to the present invention, a specific example of the relationship between the threshold value Rth of the point cloud data used for estimating the coefficient of the distortion correction function by logarithmic approximation and the corrected image will be described. (A) is a diagram for explaining the relationship between the distortion correction function by logarithmic approximation and the threshold radius value Rth = 0.75 of the point cloud data used for estimating the coefficient of the logarithmic equation; B is a view showing an image corrected by a distortion correction function estimated for each threshold radius value Rth, and (C) is an overhead view obtained by converting (B). In the image distortion correction method according to the present invention, a specific example of the relationship between the threshold value Rth of the point cloud data used for estimating the coefficient of the distortion correction function by logarithmic approximation and the corrected image will be described. (A) is a diagram for explaining the relationship between the distortion correction function by logarithmic approximation and the threshold radius value Rth = 1.00 of the point cloud data used for estimating the coefficient of the logarithmic equation; B is a view showing an image corrected by a distortion correction function estimated for each threshold radius value Rth, and (C) is an overhead view obtained by converting (B). In the image distortion correction method according to the present invention, a specific example of the relationship between the threshold value Rth of the point cloud data used for estimating the coefficient of the distortion correction function by logarithmic approximation and the corrected image will be described. (A) is a diagram for explaining the relationship between the distortion correction function by logarithmic approximation and the threshold radius value Rth = 1.25 of the point cloud data used for estimating the coefficient of the logarithmic equation; B is a view showing an image corrected by a distortion correction function estimated for each threshold radius value Rth, and (C) is an overhead view obtained by converting (B). (A) is a figure which shows an example of the correction image by the distortion correction method of the image which concerns on this invention, (B) is an overhead view obtained by converting the correction image of (A), (C) is an overhead view of (B). The bird's-eye view which biased the optical axis of a figure to the left side of the image. (A) is a figure which shows an example of the correction image by the distortion correction method using the distortion correction function by a polynomial, (B) is an overhead view obtained by converting the correction image of (A), (C) is (B). FIG. 3 is an overhead view in which the optical axis of the overhead view is biased to the left side of the image. 1 is a block diagram showing the configuration of an image distortion correction apparatus according to an embodiment of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Distortion correction apparatus 2 Imaging | photography means 3 Image processing means 4 Display means 5 Distortion correction | amendment part

Claims (1)

In a distortion correction method for correcting distortion of an image shot by a camera using a lens having distortion,
In a region included in a circle whose center is the intersection of the optical axis of the camera and the image plane, the target used for estimating the coefficient of the distortion correction function includes the region projected on the image plane. Apply a distortion correction function by polynomial approximation to find the coefficient by multiplication to correct image distortion,
An image distortion correction method for correcting image distortion by applying a distortion correction function by logarithmic approximation obtained by a least square method in an area not included in the circle.

Images