KR20100091383A - Correction method of radial distortion based on a line-fitting - Google Patents

Abstract

PURPOSE: A correction method of radial distortion based on line-fitting is provided to correct a distorted image without place limit by removing the necessity of an additional target image. CONSTITUTION: Curves among photographed image are extracted, and the curves are fitted in a straight line. The ling between the center of the image and the point of the extracted curve is formed. The cross section where the line meets a fitted straight line is obtained. A distortion constant is extracted, and the distance between the central point of the each side and the center of the image is obtained. A distorted image is corrected.

Description

Correction Method of Radial Distortion Based on a Line-Fitting}

The present invention relates to a distortion image correction method for correcting a curved curve distorted toward a center point of an image in a camera having a wide-angle lens without losing the quality of the image by linear fitting.

In order to secure a wide field of view when stopping a vehicle, mounting of a TV camera employing an ultra wide-angle lens called a fisheye lens is gradually increasing. However, ultra wide-angle lenses generally generate geometric distortions in the circumferential direction from the center of the image. Accordingly, in order to widen the application range of the image captured by the wide-angle camera for recognition of lanes or obstacles, etc., it is necessary to correct geometric distortion in the circumferential direction. One of such distortion correction methods is disclosed in a prior domestic patent of registration number 10-0831462 (name of the invention: image distortion correction method using DLT algorithm). However, such a prior domestic patent necessarily requires a target with a lattice pattern in which a three-dimensional coordinate value of an intersection point is known on a plane to extract an image distortion constant. Therefore, since the method described in the prior patent requires a flat wide place to prepare a target, there are limitations on the place, and the inconvenience of having to accurately draw a lattice pattern with known coordinate values on the plane is difficult. Since it can be implemented only if it is acquired in advance, there are many problems in practical application. In addition, there is a problem that the processing speed is also slow because the distortion correction algorithm is complicated in the prior domestic patent.

The present invention is to solve these problems, the problem of the present invention is to provide a wide range of distortion image correction method that does not require a specific target and does not need to know the three-dimensional coordinates prior to correction because of the location limitation It is to.

In addition, another problem of the present invention is to provide a distortion image correction method for accurately reproducing a real image with a fast processing speed by suggesting a new algorithm for distortion correction.

In the method for solving the above problems in the circumferential distortion image correction method by the camera wide-angle lens for correcting the image having a curved curve distorted toward the center point by a linear fit by a camera having a wide-angle lens : A curve extraction step of automatically or manually extracting a portion of the captured image that was a straight line in the actual space turned into a curve in the image; A fitting step of fitting a straight line with respect to the curves extracted in the curve extraction step; A distortion constant extraction step of extracting a distortion constant optimized by an LM optimization algorithm; And a distortion image correction step of correcting the distortion image by the optimized distortion constant.

According to the present invention having the above-mentioned problems and solving means, it is possible to correct the distorted image without requiring a separate target image, without being limited in place. In addition, the present invention is to reproduce the image close to the actual image by extracting and correcting the optimal distortion constant. In addition, according to the present invention, accurate correction is possible regardless of whether the indoor image or the outdoor image.

Hereinafter, the configuration of the present invention according to the accompanying drawings will be described in detail.

를 매우 큰 수, 즉

로 설정한다. 2) 촬영된 영상 중 공간상에서 직선이었던 부분이 영상에서 곡선으로 변한 부분을 자동 또는 수동으로 추출하고, 곡선 각각을 직선으로 핏팅한다. 이 과정에서 추출한 곡선을 최초곡선이라 정의한다. 3) 영상의 중심(o)과 1), 2)단계에서 추출된 최초곡선상의 점들을 잇는 라인들을 정의하고, 이 라인들과 핏팅된 직선이 교차되는 점들을 구하고, 중심(o)에서 최초곡선상의 점들까지의 거리(

)와 핏팅된 직선상의 점들까지의 거리(

)를 추출하고, 이 두 거리 데이터를 최적화 알고리즘인 Levenberg-Marquardt (LM) 알고리즘에 입력한다. 4) LM 알고리즘 내에서는 본 발명에서 구하고자 하는 왜곡상수인

,

,

를 반복적으로 변화시켜 가면서 알고리즘내에 구축된 왜곡모델 에 의해

를 보정한다. 보정된

와 입력된

사이의 에러를 계산하여 이전 반복단계에서 얻은 에러와의 차이가 없다면 반복을 멈추고 이 때 얻은

,

,

그리고 에러와 보정된 거리값을 출력한다. 그러나 에러변화가 크면 절차 4)는 반복된다. 5) LM 알고리즘으로부터 출력된 에러를

라 하고,

와 이전 단계의 에러

의 차이를 계산한다. 6) 에러변화가 어느 임계값 이하이면 이때의

,

,

를 사용중인 광각렌즈에 의해 빚어진 영상의 기하학적인 왜곡을 보정할 왜곡상수로 선정한다. 그러나 에러 변화율이 크면

를

로 치환하고, 절차 4)에서 얻어진 보정된 점들을 대상으로 직선 핏팅을 하여 절차 3)~6)을 반복한다. The overall configuration of the present invention is made according to the following procedure as shown in FIG. 1) Initial distortion error

Very large number, i.e.

Set to. 2) Automatically or manually extract the portion of the captured image where the straight line in space becomes a curve in the image and fit each curve in a straight line. The curve extracted in this process is defined as the initial curve. 3) Define the lines connecting the center of the image (o) and the points on the initial curve extracted in steps 1) and 2), find the points where these lines and the fitted straight line intersect, and find the initial curve at the center (o). To points on the table (

) And the distance to the points on the fitted straight line (

) And input these two distance data into the Levenberg-Marquardt (LM) algorithm, which is an optimization algorithm. 4) In the LM algorithm, the distortion constant

,

,

Is repeatedly changed by the distortion model built into the algorithm.

Calibrate Calibrated

And input

If there is no difference from the error obtained in the previous iteration step by calculating the error between

,

,

The error and corrected distance values are then output. However, if the error change is large, the procedure 4) is repeated. 5) Error output from LM algorithm

,

And previous step errors

Calculate the difference. 6) If the error change is below a certain threshold,

,

,

Is selected as the distortion constant to correct the geometric distortion of the image formed by the wide-angle lens in use. However, if the error rate is large

To

Replace with, and repeat the process 3) ~ 6) by performing a linear fit on the corrected points obtained in the procedure 4).

Hereinafter, each step of the present invention will be described in detail.

1 . Distortion Constant Extraction and Distortion Correction

In the present invention, two iterations are included. One is done inside the LM algorithm, and the other is done outside the LM algorithm.

Hereinafter, the distortion constant extraction and the distortion correction will be described.

1-1. Curve Extraction from Distorted Image

The present invention proposes two methods of extracting a curve from an input distortion image, an automatic extraction method by an algorithm, and a extraction method using a mouse by hand.

(1) Automatic extraction by algorithm

To automatically extract curves from an image, follow the steps below. First, an edge is extracted from the input image and the extracted edge is thinned through non-local maximum suppression (NLMS). Second, a bucket value as shown in FIG. 2 is given to edge pixels remaining after NLMS according to the angle of the edge. Third, edge pixels having the same bucket value and connected to each other are collected in a blob unit using a blob coloring method. Fourth, it is determined whether edges divided into different blobs are connected by position coordinates and angle values of edge pixels at both ends of each blob. In other words, if the position difference and the angle difference are small, the two blobs are connected.

 There are several factors to consider in applying this method. 1) It is not possible to detect all the curves shown in the image. 2) Straight lines that are not curved can also be detected. 3) If the number of edge pixels belonging to the blob is small, it is excluded from the curve detection object. Due to item 1) above, a manual curve detection method will be added. In the case of item 2), a straight line is not necessary to correct the geometric distortion of the image occurring in the circumferential direction. Nevertheless, the straight part is extracted by the proposed method. The removal method will be described later.

(2) Extraction by hand

In order to accurately correct the distortion of the image, it is necessary to extract the distorted portion evenly in the wide area of the image as much as possible. The automatic extraction described above has the advantage of accurately extracting the pixels on the curve, but it is difficult to satisfy this condition. Manual curve extraction is a method of moving a mouse along a curve while watching an image, where the cursor points to the pixel on the curve to be extracted. This method is difficult to accurately extract the curved pixels, but has the advantage of uniformly extracting the curves throughout the image.

1-2. Line fitting

The linear fitting uses a regular equation as shown in equation (1).

(1)

(One)

를 가지고 식 (2)와 (3)을 통해 구한다. Here, the meanings of θ and ρ are as shown in FIG. 3. That is, θ is the angle formed by the x-axis and the fitted straight line, and ρ is the distance between the origin o and the fitted straight line. Θ and ρ in Equation (1) are the distortion points on the curve as shown in FIG.

And obtain through equations (2) and (3).

(2)

(2)

(3)

(3)

이며,

은 선택된 곡선상에 있는 점들의 수이고,

와

는 각각 점들의 x좌표 평균, y좌표 평균이다.From here

,

Is the number of points on the selected curve,

Wow

Is the x-coordinate average and y- coordinate average of each point.

를 잇는 라인을 긋고, 이 라인과 핏팅된 직선이 만나는 교점

를 구한다. 그러면 중 심(o)에서

까지의 거리

와

까지의 거리

를 얻을 수 있다. 추출된 곡선상의 모든 점에 대해 동일한 방법으로

와

를 얻는다. 이렇게 얻은

와

쌍들의 차이에 대한 분산을 구하여 이 분산이 어떤 정해진 값 이하면 이 곡선은 직선으로 간주하고 삭제한다. After the straight line fitting, the center of the image shown in FIG. 3 and the extracted curve point

Draw the line connecting the line and the intersection point of this line and the fitted straight line

. Then at the center (o)

Distance to

Wow

Distance to

Can be obtained. In the same way for all points on the extracted curve

Wow

Get So obtained

Wow

Find the variance of the difference between the pairs. If this variance is less than a certain value, the curve is considered a straight line and is deleted.

1-3. Distortion Constant Extraction

The distortion constant is extracted using Levenberg-Marquart (LM) method, the most widely used optimization algorithm. In the present invention, a polynomial model of Equation (4) and a field of view (FOV) model of Equation (5) are used as distortion models of the fisheye lens to be built in the LM algorithm.

(4)

(4)

(5)

(5)

,

,

는 왜곡상수들이다. 알고리즘에서

,

는 0으로 초기화하고,

는 0에 가까운 아주 작은 수로 초기화한다. From here

,

,

Are the distortion constants. In the algorithm

,

Initializes to 0,

Initializes to a very small number near zero.

와

쌍들이다. LM알고리즘은

,

,

를 갱신해가면서 입력된 모든

에 대해 수식 (4)와 (5)를 통해

을 계산하고,

를

의 쌍인

와 비교하여 에러

를 계산한다. 즉

이다. 여기에서 N은 LM알고리즘에 입력된

의 수량이고, 아래첨자 i는 반복 횟수를 의미한다. LM알고리즘에서 에러에 대한 초기값

는 매우 큰 수를 부여된다. Data input to the LM algorithm is shown in Figures 3 and 2

Wow

Pairs. LM algorithm

,

,

All the input while updating

Through formulas (4) and (5)

, And

To

Pair of

Error compared to

Calculate In other words

to be. Where N is the LM algorithm

And the subscript i means the number of repetitions. Initial value for error in LM algorithm

Is given a very large number.

,

,

의 최적값을 찾아내는데, 본 발명에서는 이전단계와 현단계의 에러의 변화,

가 아주 작은 수인

보다 작으면 반복을 종료한다. 그리고 현단계의 왜곡상수

,

,

,에러

및 식(5)의

를 출력한다. 만약 에러의 변화율이 크면

,

,

를 갱신하고 동일한 과정을 반복한다.The LM algorithm undergoes an iterative process as shown in FIG.

,

,

To find the optimum value of the present invention, in the present invention the change of the error of the previous step and the current step,

Very small number of people

If it is less than, end the iteration. And current distortion constant

,

,

,error

And of formula (5)

. If the error rate is large

,

,

Update it and repeat the same process.

1-5 Distortion Correction

을 가지고 입력영상에서 수동 또는 자동으로 추출된 곡선상의 점들을 보정한다. 도 3에 도시된 바와 같이 현재

와

는 알고 있다. 따라서

에 대응되는 점의 좌표만 구하면 된다. 이는 다음 식 (6)에 의해 간단히 계산된다.Output from LM algorithm

And correct the curve points extracted manually or automatically from the input image. As shown in Figure 3

Wow

Know. therefore

You only need to find the coordinates of the points corresponding to. This is simply calculated by the following equation (6).

(6)

(6)

가

로 변환되었는데 이 변환이 왜곡을 완전하게 보정했는지는 아직 알 수 없다. 본 발명에서는 에러의 변화율로서 왜곡상수 추출 및 왜곡보정과정을 계속 수행할 것인지 종료할 것인지 결정한다. 이를 위해 초기에러

를 매우 큰 값으로 지정해 놓는다. 그리고 현 단계에서의 에러는 LM알고리즘의 출력값인

를

로 놓고 다음 식(7)과 같은 에러변화율을 계산한다.Through equation (6)

end

It is not known if this conversion completely compensates for the distortion. In the present invention, as the rate of change of the error, it is determined whether to continue or end the distortion constant extraction and distortion correction process. Early error

Set the value to a very large value. The error at this stage is the output of the LM algorithm.

To

Calculate the error rate of change as shown in equation (7).

(7)

(7)

가 매우 적은 값인

보다 작으면 현단계에서 왜곡보정을 종료한다. 그리고 이 때의

,

,

가 사용중인 어안렌즈의 왜곡상수가 된다. 반면에

가

보다 크면 식(6)에 의해 얻어진 새로운 곡선상의 점군 (point set)을 대상으로 식 (1)을 이용하여 도 5에 보인 바와 같이 새로운 직선을 핏팅한다. 이어지는 과정은 도 3에서 보였던 바와 같이 영상 중심(o)과 최초에 자동 또는 수동으로 추출된 곡선상의 점

를 잇는 라인을 긋고, 이 라인과 새로 핏팅된 직선이 만나는 교점

를 구하여 중심(o) 에서

까지의 거리

를 얻는다. 그리고 다시 도 4에 보인 바와 같이

와

를 LM알고리즘에 입력하면 1-4절에서 설명된 동일한 연산이 수행되어 왜곡상수

,

,

,에러

및 식(5)의

이 출력되고, 식 (7)의 에러변화율에 의해 왜곡보정 종료 여부가 결정된다. 따라서 LM알고리즘에 입력되는

와

가운데

는 항상 동일한 값이 사용되고

만 바뀐다.

Is a very small value

If smaller, distortion correction is terminated at this stage. And at this time

,

,

Becomes the distortion constant of the fisheye lens in use. On the other hand

end

If larger, a new straight line is fitted as shown in FIG. 5 by using Equation (1) for a point group on the new curve obtained by Equation (6). The process that follows is shown in Figure 3 as the center of the image (o) and the point on the curve initially extracted automatically or manually

Intersect the line between the line and the newly fitted straight line

Find the from the center (o)

Distance to

Get And again as shown in Figure 4

Wow

Is entered into the LM algorithm, the same operation described in Sections 1-4 is performed and the distortion constant

,

,

,error

And of formula (5)

Is output, and it is determined whether or not distortion correction is to be ended by the error change rate in Equation (7). Therefore, input into LM algorithm

Wow

middle

Always uses the same value

Only change.

2. Distortion Correction by Extracted Distortion Constant

를 구해 식 (4)에 대입하고, 식(4)의 결과를 식 (5)에 대입하면 왜곡이 보정된

이 얻어진다. 그러면 식 (6)에 의해 왜곡이 보정된 좌표가 얻어지는데, 이는 도 6에서 p₁~p₄가 P₁~P₄로 변환되는 것이다. P₁~P₄로부터 왜곡보정된 영상의 크기를 도 6의 우측의 점선으로 표기한 사각형으로 결정할 수 있다. 2) 왜곡보정된 영상의 크기가 결정되면 이 영상의 각 픽셀,

의 밝기값 또는 색정보는 다음과 같이 왜곡영상으로부터 가져와 채운다. 보정된 영상의 중심(O)로부터 각 보정된 영상의 픽셀

까지의 거리를

라 했을 때 식 (5)의

대신에

를 사용하여 식 (5)와 (4)로부터 다음 식을 유도할 수 있다.If the distortion constant of the fisheye lens is known, the process of applying the curve extraction and the LM algorithm described above is no longer necessary to correct the distorted image. The input image can be corrected with a known distortion constant. The correction at this time is performed as follows. 1) The size of the distortion-corrected image is obtained as shown in FIG. The distance from the center of the image (o) to these points with respect to the midpoint of each side of the distorted image (p ₁ to p ₄ in FIG. 6)

Is obtained by substituting Eq. (4) and substituting the result of Eq. (4) into Eq. (5).

Is obtained. Then, a coordinate in which distortion is corrected by Equation (6) is obtained, which is converted from p ₁ to p ₄ into P ₁ to P ₄ in FIG. 6. The size of the distortion-corrected image from P ₁ to P ₄ may be determined as a rectangle indicated by a dotted line on the right side of FIG. 6. 2) Once the size of the distortion-corrected image is determined, each pixel of this image,

The brightness value or color information of is obtained from the distorted image as follows. Pixel of each corrected image from the center O of the corrected image

Distance to

When the formula (5)

Instead of

Can be derived from equations (5) and (4).

(8)

(8)

(9)

(9)

를 구하기 위해 뉴톤법을 사용한다. 뉴톤법에 의해

가 구해지면

에 대응되는 왜곡영상의 픽셀의 위치는 다음 식으로 구한다.In the present invention, from the formula (9)

Use Newton's method to find. By the Newton method

Is saved

The pixel position of the distorted image corresponding to is obtained by the following equation.

(10)

10

는 정수가 아니므로

주변의 4개의 픽셀과 이 4개 픽셀들의 밝기정보(입력영상이 흑백영상인 경우)나 색정보(입력영상이 칼라영상인 경우)를 가지고 바이리니어 인터폴레이션 (bilinear interpolation)을 통해

에 채울 정보를 얻는다.Coordinates obtained by equation (10)

Is not an integer

By bilinear interpolation with surrounding 4 pixels and brightness information (when input image is black and white image) or color information (when input image is color image) of these 4 pixels

Get information to fill in

3. Experimental Results

In order to accurately correct the distorted image, the straight portion in the real world should be selected evenly in a wide area of the image where the curved portion is distorted in the image. As shown in FIG. 7, it can be seen that the distortion is not properly performed when the distortion correction is performed without extracting a curve sufficient to reflect the distortion effect in the entire area of the input image. FIG. 7B is a result of extracting a curve based only on automatic extraction of the curve proposed in the present invention for the input image shown in FIG. 7A. FIG. 7C is a result of correcting the distortion with the curve of FIG. 7B. In detail, it can be seen that a portion which was a straight line in the real world still remains as a curve at the left side or the bottom of the image.

 FIG. 8 is a result of distortion correction when the distorted points of the same image used in FIG. 7 are extracted to be evenly distributed throughout the image. Comparing FIG. 7C with FIG. 8B, it can be seen that FIG. 8B is much better in correcting distortion.

 9 shows an example in which distortion correction is performed on an image photographed outdoors by the algorithm proposed in the present invention.

1 is a block flow diagram illustrating the overall configuration of the present invention.

2 is a diagram illustrating a bucketing process of edge angles for extracting an initial curve from an image in the present invention.

3 is a graph illustrating a straight line fitting method in the present invention.

4 is a flowchart illustrating a distortion constant extraction process in the LM algorithm according to the present invention.

5 is a graph illustrating a correction of a curve initially extracted in the present invention and a new straight line fitting.

6 is a graph illustrating a method for setting the size of a distortion-corrected image in the present invention.

7A is an image before correction photographed by the wide-angle lens camera of the present invention.

FIG. 7B is a result of extracting a curve based only on automatic extraction of the curve proposed in the present invention for the input image shown in FIG. 7A.

FIG. 7C is a corrected image having distortion corrected with the curve of FIG. 7B.

FIG. 8A is a result of extracting the curves evenly distributed for the same image as that of FIG. 7A.

FIG. 8B is a corrected image corrected for the distorted image by the extraction curve of FIG. 8A. 9A is an image photographed outdoors by a wide-angle lens camera.

FIG. 9B is an image in which distortion correction is performed on the same image of FIG. 9A by the algorithm proposed in the present invention.

Claims (4)

A method of correcting a circumferentially distorted image by a camera wide-angle lens in which an image having a curved line distorted toward a center point by a camera having a wide-angle lens is corrected by linear fitting: A curve extracting step of extracting a portion of the photographed image from the image, the portion of which is a straight line into a curve in the image; A fitting step of fitting a straight line with respect to the curves extracted in the curve extraction step by the following equation (1); Center of image (o) and extracted curve points

Draw the line connecting the line and the intersection point of this line and the fitted straight line

Find the, at the center (o)

Distance to

Wow

Distance to

Suppose, determined by the following equations (4) and (5)

, And

Wow

Distortion constant whose error is within a preset value

,

,

A distortion constant extraction step of extracting a; The distance between the center of each side of the distorted image and the center of the image (o)

Is determined by inputting the following equation (6)

And a distortion image correction step of correcting the distortion image by means of the circumferentially distorted image correcting method of the camera wide-angle lens.

Where θ is the angle between the x-axis and the fitted straight line, ρ is the distance between the origin O, the center of projection, and the fitted straight line, and θ and ρ in Equation (1) are the distortion points on the curve

It is determined by the following equations (2) and (3).

From here

,

Is the number of points on the selected curve,

Wow

Is the x-coordinate average and y- coordinate average of each point.

From here

,

,

Are distortion constants,

,

Initializes to 0,

Initializes to a very small number near zero.

The method of claim 1, wherein the curve extraction step Extracting an edge from the input image and thinning the extracted edge through local non-local maximum suppression (NLMS); Assigning a bucket value to the remaining edge pixels according to the angle of the edge after the thinning step; Collecting edge pixels having the same bucket value and connected to each other by a blob coloring method using a blob coloring method; And determining whether or not edges divided into different blobs are connected by position coordinates and angle values of edge pixels at both ends of each blob. The circumferential distortion image of the camera wide-angle lens according to claim 2, wherein the curve extraction step further includes a curve extraction step by manually extracting a curve by moving a mouse along the curve while viewing an image. Correction method. The method of claim 1, wherein the corrected image

The brightness value or the color information of may be determined by the following equation 8, 9, 10

Circumferentially distorted image correcting method using a camera wide-angle lens, characterized in that it employs a brightness value or color information.

Images