JP4775541B2 - Distortion correction method for captured images - Google Patents

Description

  The present invention relates to a distortion aberration correction method for a captured image.

  In recent years, with the improvement in performance of photography equipment such as digital cameras, it has become relatively easy to obtain high-quality images. For example, a structure such as a bridge is photographed with a digital camera, and the structure is obtained from the photographed image. A system has been developed that can measure various dimensions such as the shape and size of the non-contact and with high accuracy.

However, in a system using an optical lens such as a digital camera, a measurement error due to lens distortion is included.
For this reason, a measurement method has been proposed in which a calibration plate on which a target mark is provided in advance is attached to a measurement object, and distortion is corrected by photographing the calibration plate (for example, Patent Document 1).
JP-A-5-110926

  As described above, according to the measurement method for correcting distortion by photographing the reference calibration plate attached to the measurement object, the measurement object that is a large structure such as a bridge, that is, the imaging object is It is unsuitable for a wide range.

  Therefore, an object of the present invention is to provide a distortion aberration correction method in a captured image that can correct distortion aberration without attaching a reference calibration plate in advance to an object to be imaged.

In order to solve the above-described problem, a distortion aberration correction method for a captured image according to claim 1 of the present invention includes:
Shifting at least two photographed images by shifting any one of the predetermined regions of the subject to be photographed through the lens so that the relative positions thereof are different from the lens;
Applying a digital image correlation method to the at least two photographed images to detect a deviation amount (u ′ _x , u ′ _y ) of a predetermined region on both photographed images;
The plane model considering the rotational movement and the parallel movement due to the deviation in the aberration model quantity (α _x , α _y ) representing the aberration represented by the following expressions (A) and (B) ( the following expressions (C) and (D) The displacement amount detected in the above step is substituted into the deformation amount calculation equation represented by the following equations (E) and (F) obtained by adding the displacement amount calculation equation represented by ( Determining a coefficient representing at least an aberration model amount of the deformation amount calculation formula using a square method;
And correcting the distortion using the coefficient of the determined aberration model quantity.
α _x = k ₁ x (x ² + y ² ) (A)
α _y = k ₁ y (x ² + y ² ) (B)
u _x = a ₁ x + a ₂ y + a ₃ (C)
u _y = a ₄ x + a ₅ y + a ₆ (D)
u ′ _x = a ₁ x + a ₂ y + a ₃ + k ₁ {x _d (x _d ² + y _d ² ) −x _u (x _u ² + _yu ² )} (E)
u ′ _y = a ₄ x + a ₅ y + a ₆ + k ₁ {y _d (x _d ² + y _d ² ) −y _u (x _u ² + _yu ² )} (F)
Where u ′ _x is the amount of movement in the x direction including aberration, u ′ _y is the amount of movement in the y direction including aberration, x _u is the x coordinate before movement (excluding aberration), and _yu is before movement. Y coordinate (not including aberration), x _d is the x coordinate after movement (not including aberration), y _d is the y coordinate after movement (not including aberration), and u _x is the amount of movement in the x direction (aberration). And u _y is the amount of movement in the y direction (excluding aberrations).

According to the above Kiibitsu songs aberration correcting method, the shift amount including the aberrations in the predetermined region in at least two photographed images taken with a position shifted a shooting target, the rotational movement and translational movement due to the deviation in the aberration model weight Using the amount of deviation obtained by the digital image correlation method for the coefficient of at least the aberration model amount of the equation representing the plane related to this deformation amount calculation formula. Thus, since the determination is made by the least square method, for example, it is possible to correct the distortion aberration using only the captured image without using a reference calibration plate.

[Embodiment]
Hereinafter, a distortion correction method for a captured image according to an embodiment of the present invention will be described with reference to the drawings.

  In the present embodiment, for example, a measurement object is photographed using a photographing device (hereinafter referred to as a camera) such as a digital camera, that is, through a lens, and a photographed image of the measurement object photographed by the camera is used. When measuring various dimensions related to the surface, such as the shape and size of the measurement object, the displacement due to the distortion of the lens is included as an error, so the displacement due to this distortion is removed. A distortion aberration correction method will be described.

  The distortion aberration correction method according to the present embodiment uses a predetermined area (hereinafter referred to as a calculation area, specifically, a range of several tens of pixels × several tens of pixels) of a measurement object, that is, a photographing object, as a camera. The digital image correlation method is applied to at least two shot images taken at a predetermined distance with respect to the image to determine the amount of deviation between the two images, and the amount of deviation is used to cause distortion by the lens. Is a method of correcting the above.

First, a method for measuring a deviation amount (hereinbelow referred to as a movement amount) of a photographing object using a digital image correlation method will be described.
In the digital image correlation method, the amount of movement of the calculation area is detected using the correlation of the luminance value distribution. This is based on the fact that the diffuse reflection image of the object surface moves with the object surface, and the characteristics are preserved before and after the movement. The correlation between the luminance values in the images before and after the movement is obtained by the following equation (1). .

ここで、Ｉ_ｙ（ｘ，ｙ）は移動前画像の座標（ｘ，ｙ）における輝度値、Ｉ_ｄ（ｘ^＊，ｙ^＊）は移動後画像の座標（ｘ^＊，ｙ^＊）における輝度値である。座標（ｘ，ｙ）と（ｘ^＊，ｙ^＊）との間には下記（２）式の関係がある。

Here, I _y (x, y) is the luminance value at the coordinates (x, y) of the image before movement, and I _d (x ^* , y ^* ) is the luminance value at the coordinates (x ^* , y ^* ) of the image after movement. It is. The relationship of the following formula (2) exists between the coordinates (x, y) and (x ^* , y ^* ).

ここで、ｕ_ｘおよびｕ_ｙはそれぞれ計算領域画像の中心におけるｘ方向およびｙ方向の移動量である。図１に示すように、移動前のＰ点が移動後にＰ′点に移動したとすると、Ｐ点の移動量がｕ_ｘ，ｕ_ｙになる。

Here, u _x and u _y are movement amounts in the x direction and y direction at the center of the calculation area image, respectively. As shown in FIG. 1, if the P point before the movement is moved to the P ′ point after the movement, the movement amounts of the P point are u _x and u _y .

Then, the movement amounts u _x and u _y that minimize S in the above equation (1) may be determined. When detecting displacement with a resolution of 1 pixel or less, a luminance value may be interpolated using a bilinear function or a cubic spline function. In order to reduce the calculation speed, Newton-Raphson method or the like is used as an actual correlation calculation method.

Next, distortion will be described.
Distortion refers to a phenomenon in which a straight object is photographed as a curve, for example, using the distance from the center of the aberration (in many cases the center of the image, but not always) Can be modeled.

For example, as shown in FIG. 2, it is assumed that a point that should originally appear at the position of the point A is reflected at the point B due to distortion.
The distance between the points A and B (the amount of aberration) can be decomposed into a radial direction (radial direction) distortion alpha _r and circumferential distortion alpha _theta.

The circumferential distortion aberration α _θ is sufficiently smaller than the radial distortion aberration α _r . For example, when the circumferential distortion aberration α _θ is ignored, the radial distortion aberration α _r is a distance from the center of the aberration. As a function of r, it is expressed by the following equation (3).

ここで、ｋ_１，ｋ_２，ｋ_３，・・・は収差の量を表す係数である。多くの場合、高次項を無視できるので、（３）式は下記（４）式にて近似することができる。

Here, k ₁ , k ₂ , k ₃ ,... Are coefficients representing the amount of aberration. In many cases, higher-order terms can be ignored, so equation (3) can be approximated by equation (4) below.

すなわち、画像上の歪曲収差の量（幾何学的なずれ量）は収差の中心からの距離ｒの３乗に比例し、その係数ｋ_１を決定することで歪曲収差の量を知ることができ、これに基づき補正を行うことができる。

That is, the amount of distortion of the image (geometric deviation amount) is proportional to the cube of the distance r from the center of the aberration, it is possible to know the amount of distortion by determining the coefficients k ₁ Based on this, correction can be performed.

When it is disassembled the radial distortion alpha _r in the x and y directions, i.e., the amount of distortion in the orthogonal coordinate system is expressed by the following equation (5).

ここで、θ＝arctan（ｘ，ｙ）である。

Here, θ = arctan (x, y).

  As described above, when the subject is moved as it is by the digital image correlation method, that is, from the taken images before and after the movement when the rigid body is displaced, the correlation of the luminance value in the image is used. The amount of movement can be detected. In other words, by using this method, the distribution of the rigid translational displacement (hereinafter referred to as parallel movement or rigid body parallel movement) of the object to be imaged can be obtained. Further, the distribution of rigid body rotational displacement (hereinafter referred to as rotational movement or rigid body rotational movement) can also be obtained.

  FIG. 3 shows a displacement distribution obtained by the digital image correlation method when the object to be photographed is moved in the x direction (horizontal direction). When there is no distortion in the lens, the distribution does not appear in the displacement and becomes a uniform value. On the other hand, when there is aberration, a displacement distribution as shown in FIG. 3 appears. Distortion is detected using this distribution. 3A shows displacement in the x direction, and FIG. 3B shows displacement in the y direction.

Here, if the coordinate value of the point including the aberration is x ′, y ′, the coordinate of the ideal point without aberration is x, y, the amount of aberration in the x direction and the y direction is α _x , α _y. Their relationship is expressed by the following equation (6).

デジタル画像相関法により得られる移動量、すなわち収差を含んだｘ方向およびｙ方向の移動量ｕ′_ｘ，ｕ′_ｙは下記（７）式で表される。

The amount of movement obtained by the digital image correlation method, that is, the amounts of movement u ′ _x , u ′ _y in the _x and y directions including aberration are expressed by the following equation (7).

ここで、ｘ′_ｕは移動前のｘ座標（収差を含む）、ｙ′_ｕは移動前のｙ座標（収差を含む）、ｘ′_ｄは移動後のｘ座標（収差を含む）、ｙ′_ｄは移動後のｙ座標（収差を含む）、α_ｘｕは点（ｘ_ｕ，ｙ_ｕ）におけるｘ方向収差、α_ｙｕは点（ｘ_ｕ，ｙ_ｕ）におけるｙ方向収差、α_ｘｄは点（ｘ_ｄ，ｙ_ｄ）におけるｘ方向収差、α_ｙｄは点（ｘ_ｄ，ｙ_ｄ）におけるｙ方向収差、ｘ_ｕは移動前のｘ座標（収差を含まず）、ｙ_ｕは移動前のｙ座標（収差を含まず）、ｘ_ｄは移動後のｘ座標（収差を含まず）、ｙ_ｄは移動後のｙ座標（収差を含まず）、ｕ_ｘはｘ方向移動量（収差を含まず）、ｕ_ｙはｙ方向移動量（収差を含まず）である。

Here, x ′ _u is the x coordinate before movement (including aberration), y ′ _u is the y coordinate before movement (including aberration), x ′ _d is the x coordinate after movement (including aberration), y ′ _d (including aberration) y-coordinate after movement, alpha _xu is the point _{(x u, y} u) x-direction aberration in, alpha _yu is the point _(x u, _{y u)} y-direction aberration in, alpha _xd is the point ( x direction aberration at x _d , y _d ), α _yd is the y direction aberration at point (x _d , y _d ), x _u is the x coordinate before movement (excluding aberration), and _yu is the y coordinate before movement. (Without aberration), x _d is the x coordinate after movement (without aberration), y _d is the y coordinate after movement (without aberration), and u _x is the amount of movement in the x direction (without aberration) , U _y are y-direction movement amounts (not including aberrations).

  By the way, considering the case where the object to be imaged is a rigid body parallel movement and a rigid body rotational movement (also a rigid body displacement), the movement amount (deviation amount) is approximated by an equation representing a plane (deviation amount calculation formula). The knowledge that it is possible was obtained.

When the object to be imaged is moved in parallel with the rigid body, the movement amounts u _x and u _y can be expressed by the following equation (8).

ここで、移動量ｕ_ｘおよびｕ_ｙは撮影画像上で一様の値（定数）となる。一方、撮影対象物が剛体回転移動した場合には、移動前座標と移動後座標との関係は回転行列を用いて、下記（９）式にて表される。

Here, the movement amounts u _x and u _y are uniform values (constants) on the captured image. On the other hand, when the object to be imaged is rotated and moved rigidly, the relationship between the coordinates before movement and the coordinates after movement is expressed by the following equation (9) using a rotation matrix.

したがって、撮影対象物が剛体回転移動した場合の移動量ｕ_ｘおよびｕ_ｙは、下記（１０）式で表すことができる。

Therefore, the movement amounts u _x and u _y when the object to be photographed is rotated and rotated rigidly can be expressed by the following equation (10).

ここでθは回転角度であるので、撮影した特定の画像に対して定数となる。

Here, since θ is a rotation angle, it is a constant for a specific image taken.

That is, when measured rigid body translational movement becomes the (7) u _x and u _y are constant values in the equation (measured area in one night). On the other hand, when measured both rigid translational movement and the rigid rotation movement, since u _x and u _y may be represented as a two-dimensional plane, the movement amount of deformation amount calculation represented by the following formula (11) was measured It is expressed by a formula.

収差のモデルは収差を含まない座標の関数として表されているが、測定結果は収差を含む座標の関数として得られる。

Although the aberration model is expressed as a function of coordinates including no aberration, the measurement result is obtained as a function of coordinates including aberration.

  By the way, the movement amount (deviation amount) obtained by the digital image correlation method is obtained as a function of the point (x ′, y ′) including the aberration, but the aberration model of the equation (11) does not include the aberration. It is given as a function of (x, y). Therefore, each coefficient cannot be determined directly. Therefore, each coefficient is determined by combining the least square method and the iterative calculation.

  First, coordinate values (x ′, y ′) including aberrations are substituted into (x, y) as initial values, and approximate values of the respective coefficients are determined by a least square method. That is, it can be calculated by the following equation (12). Of course, when the least square method is used, calculation is performed for a plurality of pixels.

ここで、

here,

上記式中、下付添字の１・・・ｎはデータ点のインデックスを示す。以上の計算により得られた各係数の近似値を用い、収差を含まない座標（ｘ，ｙ）の近似値を計算する。その座標値（ｘ，ｙ）を用い、再び、同様の最小二乗法により各係数を計算する。これを各係数の値が収束するまで繰り返すことにより、各係数を決定することができる。

In the above formula, subscripts 1... N indicate data point indexes. Using the approximate value of each coefficient obtained by the above calculation, the approximate value of coordinates (x, y) not including aberration is calculated. Using the coordinate value (x, y), each coefficient is calculated again by the same least square method. By repeating this until the value of each coefficient converges, each coefficient can be determined.

If it is necessary to determine the center of aberration, the center coordinate of the aberration is set as an unknown and a nonlinear least square method may be used.
After determining the aberration model amount, that is, the aberration amount by the above method, the captured image is corrected using the value. When correcting the distortion of the captured image, for example, an image luminance value interpolation method or the like is used.

Here, a specific example will be described.
FIG. 3 described above shows the measurement result of the rigid body parallel movement amount, where (a) is the x direction and (b) is the y direction. From this measurement result, distortion was detected.

  That is, FIG. 4 shows the result of aberration correction performed on the movement amount distribution of FIG. 3 using the obtained coefficients. 4A shows the x direction, and FIG. 4B shows the y direction. That is, after the aberration correction, the value of the movement amount is almost uniform at each point on the image, and an appropriate result is obtained as the measurement of the rigid parallel movement.

Here, the distortion aberration correction method described above is described in a step format.
That is, this distortion aberration correction method includes a step of obtaining at least two photographed images by shifting a predetermined region of a subject to be photographed via a lens so that the relative position differs from that of the lens, and at least two of these. A step of detecting a deviation amount (u ′ _x , u ′ _y ) of a predetermined region on both the captured images by applying a digital image correlation method to the captured images of the lens, and an aberration model amount (α _x , α _y ) representing the aberration. ) To the deformation amount calculation formula shown by the following formulas (A) and (B) obtained by adding a shift amount calculation formula expressed by a plane formula considering rotational movement and parallel movement due to shift, Substituting the detected deviation amount and determining a coefficient representing at least an aberration model quantity of the deformation amount calculation formula using the least square method, and correcting distortion aberration using the coefficient of the determined aberration model quantity The Cormorants and steps are provided.

u ′ _x = a ₁ x + a ₂ y + a ₃ + α _x (A)
u ′ _y = a ₄ x + a ₅ y + a ₆ + α _y (B)
As described above, the displacement amount including the aberration for the calculation region in at least two photographed images photographed at the position where the photographing object is displaced is a plane representing rotational movement and parallel movement due to the displacement of the photographed image as the aberration model amount. Using the deviation amount obtained by the digital image correlation method, each coefficient of an equation representing a plane relating to the deformation amount calculation formula, at least a coefficient representing an aberration model amount, Since the determination is made by the least square method, for example, it is possible to correct distortion by using only the photographed image without using a reference calibration plate.

  By the way, in the said embodiment, although the case where the measurement target object was displaced rigidly was demonstrated, you may move a camera instead.

It is a figure which shows the positional relationship of the calculation area | region of the imaging | photography target object for demonstrating the distortion aberration correction method which concerns on embodiment of this invention. It is a figure which shows the distortion aberration for demonstrating the same distortion aberration correction method. It is a figure which shows the displacement distribution at the time of measuring the rigid body displacement in the same distortion correction method, (a) shows displacement in x direction, (b) shows displacement in y direction. It is a figure which shows the displacement distribution at the time of performing aberration correction with respect to the displacement distribution, (a) shows displacement in x direction, (b) shows displacement in y direction.

Claims (1)

Shifting at least two photographed images by shifting any one of the predetermined regions of the subject to be photographed through the lens so that the relative positions thereof are different from the lens;
Applying a digital image correlation method to the at least two photographed images to detect a deviation amount (u ′ _x , u ′ _y ) of a predetermined region on both photographed images;
The plane model considering the rotational movement and the parallel movement due to the deviation in the aberration model quantity (α _x , α _y ) representing the aberration represented by the following expressions (A) and (B) ( the following expressions (C) and (D) The displacement amount detected in the above step is substituted into the deformation amount calculation equation represented by the following equations (E) and (F) obtained by adding the displacement amount calculation equation represented by ( Determining a coefficient representing at least an aberration model amount of the deformation amount calculation formula using a square method;
And a step of correcting distortion using the coefficient of the determined aberration model quantity. A method of correcting distortion in a photographed image.
α _x = k ₁ x (x ² + y ² ) (A)
α _y = k ₁ y (x ² + y ² ) (B)
u _x = a ₁ x + a ₂ y + a ₃ (C)
u _y = a ₄ x + a ₅ y + a ₆ (D)
u ′ _x = a ₁ x + a ₂ y + a ₃ + k ₁ {x _d (x _d ² + y _d ² ) −x _u (x _u ² + _yu ² )} (E)
u ′ _y = a ₄ x + a ₅ y + a ₆ + k ₁ {y _d (x _d ² + y _d ² ) −y _u (x _u ² + _yu ² )} (F)
Where u ′ _x is the amount of movement in the x direction including aberration, u ′ _y is the amount of movement in the y direction including aberration, x _u is the x coordinate before movement (excluding aberration), and _yu is before movement. Y coordinate (not including aberration), x _d is the x coordinate after movement (not including aberration), y _d is the y coordinate after movement (not including aberration), and u _x is the amount of movement in the x direction (aberration). And u _y is the amount of movement in the y direction (excluding aberrations).

Images