WO2012161431A9

WO2012161431A9 - Method for generating an image of the view around a vehicle

Info

Publication number: WO2012161431A9
Application number: PCT/KR2012/003482
Authority: WO
Inventors: 하수영; 나경진; 김진경
Original assignee: 아진산업(주)
Priority date: 2011-05-23
Filing date: 2012-05-03
Publication date: 2013-03-28
Also published as: KR101249791B1; KR20120130798A; WO2012161431A3; WO2012161431A2

Abstract

The present invention relates to a method for generating an image of the view around a vehicle, which involves estimating a polygon coordinate in a distortion image obtained through a wide-angle camera and polygon-mapping the estimated polygon coordinate into a standard image. To this end, the method for generating an image of the view around a vehicle according to the present invention involves correcting the radial distortion of an image obtained from a plurality of wide-angle cameras in order to generate a view image, and comprises the steps of: setting a standard coordinate value with respect to each intersection point of a standard grid pattern and generating a standard image divided by a plurality of standard polygons having standard vertices (A', B', C') with respect to a grid pattern image; capturing a checkerboard pattern using a plurality of wide-angle cameras to obtain each distortion image; detecting the edge of the checkerboard pattern in each distortion image; estimating a plurality of correction points by using brightness distribution change amounts in a vertical direction and in a parallel direction during line tracking along the detected edge area, and selecting a correction order for the estimated correction points; dividing each distortion image into a plurality of distortion polygons having correction vertices (A, B, C); mapping each pixel coordinate of the distortion image onto a standard coordinate in order to make a distortion polygon correspond to a standard polygon according to the correction order in order to obtain each correction image; and generating an image of the view around a vehicle by combining each correction image with a vehicle plane image.

Description

How to create vehicle around view image

The present invention relates to a method for generating a vehicle around view image, and more particularly, to a method for generating a vehicle around view image for estimating a polygon coordinate point in a distorted image obtained through a wide-angle camera and for performing a polygon mapping of the estimated polygon coordinate point to a reference image. It is about.

In many cases, cameras are installed at various locations, such as front and rear, for the purpose of providing convenience of driving and analyzing the cause of an accident when a traffic accident occurs. In particular, in recent years, a camera is installed on the rear side of the vehicle to provide a rear image when the vehicle is reversed, thereby providing convenience of parking.

To provide such a rear image, a wide-angle camera that can secure a wide field of view is mainly used. A wide-angle camera not only has a lower resolution toward the outer portion, but also a distortion, ie, radial distortion, that is contorted toward the outer portion. .

Therefore, there is a need for a process of correcting such a distorted image, which is generally corrected using a warping equation. The warping equation is expressed as a first-order, second-order, or third-order equation as shown in the following equation.

Equation 1

Equation 2

Equation 3

In the correction of the distortion image, a warping parameter is calculated using a grid-shaped standard grid image and then applied to all pixels. To apply this warping equation, the coordinate values of the intersection points of the standard grid image and the wide angle camera are used. It is necessary to know the coordinate values of the intersection points on the distortion grid image obtained by imaging.

However, in the related art, the coordinate values of the intersection points of the standard grid image and the coordinate values of the intersection points on the distorted grid image obtained by capturing with the wide-angle camera must be directly obtained by hand.

On the other hand, in the case of the rear camera of the vehicle, it is very helpful for rear parking, but since only the rear image of the vehicle is provided, it is impossible to grasp the situation in front of the vehicle or in the left and right directions, so the parking is not completely recognized without knowing all the surroundings. There is also a disadvantage.

In addition, the conventional distortion correction method has a disadvantage that the distortion is not completely corrected despite the use of the warping equation.

An object of the present invention for solving the disadvantages of the background technology is to divide the reference grid pattern into a plurality of reference polygons, and to divide the distortion image obtained by the wide-angle camera into a plurality of distortion polygons, and then to coordinate the coordinate points of each distortion polygon. Automatically estimates and corrects the distortion by mapping each coordinate point of the distortion polygon to the reference polygon so that the estimated distortion polygon corresponds to the reference polygon, and freely transforming and shifting the position of the distorted image to the position to be corrected. The present invention provides a method of generating a vehicle around view image.

Another object of the present invention is to provide a vehicle around view image generation method for generating and displaying a single around view image by combining external images taken from the front, rear, left and right of the vehicle corrected by the polygon mapping method. .

In the method of generating an around view image of a vehicle according to the present invention, a method of generating an around view image by correcting a radial distortion of an image obtained from a plurality of wide-angle cameras, the reference for each intersection point of a reference grid pattern Setting a coordinate value and generating a reference image obtained by dividing a plurality of reference polygons having reference vertices A ', B', and C 'with respect to the grid pattern image. Obtaining a distorted image of the image, detecting an edge of a check board pattern in each distorted image, and performing a line trek along the detected edge region, using a change in brightness distribution in a vertical direction and a horizontal direction. Estimating the corrected vertex (A, B, C) for each distortion image Dividing into a plurality of distortion polygons, acquiring each correction image by mapping each pixel coordinate of the distortion image to the reference coordinate so that the distortion polygon corresponds to the reference polygon according to the correction order, and obtaining each correction image and the plane of the vehicle Combining the images to generate an around view image.

Here, the step of generating an around view image may remove overlapping images of adjacent corner areas from a plurality of correction images, set reference line segments inclined inward from an outer edge portion of each of the correction images, and reference adjacent correction images. Overlapping images are removed by overlapping line segments.

In addition, the mapping of the distortion polygon and the reference polygon determines whether the pixel to be mapped is located inside the distortion polygon, and performs mapping only when the pixel to be mapped is located inside the distortion polygon.

The mapping of the pixel coordinates of the distorted image to the reference coordinates may include selecting a first reference vertex from any one of vertices A, B, and C, a first straight line passing through the first reference vertex and the mapping target pixel P, and Calculating a first coordinate of the intersection point of the line segment facing the first reference vertex, and selecting one of the vertices A, B, and C except for the first reference vertex as the second reference vertex, and A second intersection coordinate calculation step of calculating intersection coordinates between a second reference vertex and a second straight line passing through the mapping target pixel P and a line segment facing the second reference vertex, and a first intersection coordinate ratio on the first straight line and Calculating a second intersection coordinate ratio on the second straight line, and calculating a third intersection coordinate on the reference polygon by mapping the first intersection coordinate ratio to vertices A ', B', and C 'of the reference polygon, 2 crossing left Comparing the ratios to vertices A ', B', C 'of the reference polygon to calculate the fourth intersection coordinates on the reference polygon, and extracting third and fourth vertices facing each of the third and fourth intersection points. Extracting a third segment connecting the third vertex and the third intersection point and a fourth segment connecting the fourth vertex and the fourth vertex to calculate an intersection point P ′ of the third segment and the fourth segment; and And mapping the mapping target pixel P to the intersection point P '.

In addition, the first reference vertex selection may include calculating a first slope for calculating three slopes of the distorted polygon, calculating a distance between the mapping target pixel P and each vertex A, B, and C; Extracting three triangles by connecting the mapping target pixel P and each vertex, and extracting vertices far from the vertex and the mapping target pixel P, respectively, for each triangle; Calculating a second slope, calculating a difference between the first vertex and the second slope, extracting a line segment having the smallest difference between the vertices, and the mapping target pixel P; A vertex facing the smallest line segment is selected as the first reference vertex.

In addition, the second reference vertex is selected as a vertex closest to the distance as a result of calculating the distance to the mapping target pixel P.

The present invention has the advantage that the distortion can be completely corrected by freely transforming and shifting the position to a position to correct the selected region of the distorted image using the polygon mapping method.

In addition, the present invention has an advantage in that a plurality of distortion polygons in the distorted image are automatically estimated to the reference polygons of the reference image, thereby automatically correcting the distortion image.

In addition, the present invention has the advantage that the blind spots can be removed when parking by combining and generating a single around view image by combining the external images taken from the front, rear, left, right of the vehicle to enable smooth parking. .

1 is a flowchart sequentially illustrating a method of generating a vehicle around view image according to the present invention.

2 to 7 are reference views for explaining a vehicle around view image generation process according to the present invention.

8 is a view for explaining a process for determining whether a pixel to be mapped is located inside a distortion polygon according to the present invention.

9 is a view for explaining a method of mapping a distortion polygon to a reference polygon in accordance with the present invention.

FIG. 10 is a flowchart illustrating a process of selecting two reference vertices which are references when polygons are mapped in an embodiment of the present invention. FIG.

FIG. 11 is a reference diagram for explaining the method of FIG. 10; FIG.

12 is a view illustrating an around view image finally combined in a vehicle around view generating method according to the present invention;

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the present invention.

1 is a flowchart sequentially illustrating a method of generating a vehicle around view image according to the present invention, and FIGS. 2 to 7 are reference diagrams for explaining a process of generating a vehicle around view image according to the present invention. The following processing may be performed via a PC or may be processed by a process inside the vehicle.

First, the present invention generates a reference grid pattern corresponding to the reference check board pattern 10 shown in FIG. 2 (S10), and sets reference coordinate values for each intersection point of the grid pattern (S20). In operation S30, the reference image 20 in which the grid pattern image is divided into a plurality of reference polygons 30 is set. In this case, the plurality of reference polygons 30 are formed in a polygonal shape having vertices A ', B', and C '.

Next, a check board pattern is photographed by a plurality of wide-angle cameras, respectively, to obtain a distorted image 40 as shown in FIG. 3 (S40).

Next, the check board pattern of the distorted image 40 is recognized, and an edge of the check board pattern in the distorted image is detected as shown in FIG. 4 through morphology and robust operation (S50).

When the edge of the check board pattern is detected, a plurality of correction points are estimated and a correction order for the estimated correction points is selected (S60). At this time, the correction point estimation is estimated using the change in the brightness distribution in the vertical direction and the horizontal direction while performing line tracking along the detected edge region as shown in FIG. 5. That is, since the brightness variation is large at each intersection in the check board pattern, the point where the brightness variation is large during the line tracking process is estimated as a correction point and corrected according to the order estimated as the correction point as shown in FIG. 6. Select the order.

Subsequently, the distortion points are divided into a plurality of distortion polygons having correction points A, B, and C (S70).

As shown in FIG. 7, each pixel coordinate of the distorted image is mapped to the reference coordinate so that the distorted polygon corresponds to the reference polygon according to the selected correction order, thereby obtaining each corrected image (S80). That is, each pixel in the distortion polygon 50 is mapped to the reference polygon 30 of the reference image. In this case, it is determined whether the pixel to be mapped is located inside the distortion polygon. If the pixel to be mapped is located inside the distortion polygon, the coordinates of the pixel are mapped to the reference coordinate. If the pixel to be mapped is not inside the distortion polygon, the mapping is performed. Exclude from the target.

Hereinafter, a process of mapping the distortion polygon according to the present invention to the reference polygon will be described in more detail with reference to FIGS. 8 to 10.

8 is a view for explaining a process for determining whether a pixel to be mapped is located inside a distortion polygon according to the present invention. First, as shown in FIG. Check if it is located within the distortion polygon consisting of B and C. To this end, it is necessary to obtain the coordinates of the points D and E of the square including vertices A, B, and C as shown in FIG.

At this time, the coordinates of D and E can be obtained using the following equation.

If such a coordinate is obtained and a square range including the vertices A, B, and C of the pixel to be mapped P is specified, a part outside the coordinate may be defined as outside. At this time, the outside range condition is if (P _x <D _x ∥P _x > E _x ∥ P _y <D _y ∥P _y > E _y ), and if the outside range condition is not satisfied, the corresponding pixel to be mapped (P You can see that is located inside the rectangle.

In addition, as shown in FIG. 8C, in order to check whether the mapping target pixel P is located inside a polygon including vertices A, B, and C.

Wow

Find the coordinate of the point F where In this case, F can be obtained by the following equation.

F = cross (

,

)

In this way, when the coordinates of the point F are obtained, it is checked whether the mapping target pixel P is located outside the distortion polygon. Here, the condition for the pixel to be mapped (P) outside the distortion polygon is the outer range condition if (F _x <D _x ∥ F _x > E _x ∥ F _y <D _y ∥ F _y > E _y ), Point F is a line segment

This is a case where the mapping target pixel P is located in the OUT1 or OUT2 region.

Then, as shown in (d) of FIG. 8, the coordinates of the points G and H of the quadrangular points B and F, which are connected to the mapping target pixel P, are vertices are obtained.

At this time, the coordinates of G and H can be obtained using the following equation.

In this way, when the coordinates of G and H are obtained, it is checked whether the mapping target pixel P is located outside the rectangle BGFH. Here, the outside range condition for the mapping target pixel P to be located outside the distortion polygon is if (P _x <G _x ∥ P _x > H _x ∥P _y <G _y ∥P _y > H _y ). The case where the pixel P is located outside the rectangle BGFH corresponds to the case where the pixel to be mapped is located in the OUT3 region.

As a result of checking the outside range condition, if the outside range condition does not correspond, the pixel to be mapped P is located inside the distortion polygon.

If it is determined that the mapping target pixel P is located inside the distortion polygon, the corresponding pixel is mapped to correspond to the reference image.

FIG. 9 is a view for explaining a method of mapping a distorted polygon to a reference polygon according to the present invention. A first straight line passing through A) and the mapping target pixel P

) And the line segment facing the first reference vertex (A)

The coordinate of the 1st intersection point (a) which () intersects is computed. Then, one of the two except for the first reference vertex A is selected as the second reference vertex B, and a second straight line passing through the second reference vertex B and the mapping target pixel P (

) And the line segment facing the second reference vertex (B)

The coordinate of the 2nd intersection point (b) which () intersects is computed.

Here, the coordinates of the first intersection point a and the second intersection point b are obtained by the following equation.

a = CROSS (

,

)

b = CROSS (

,

)

Then, line segment

Coordinate ratio of b to

:

) And line segment

Coordinate ratio of a to

:

) And calculate the coordinate ratio of

:

) And the coordinate ratio of a (

:

) Corresponding to the vertices (A ', B', C ') of the reference polygon

And line segment at point a 'on Pinterest

Calculate the coordinates of point b 'on the phase.

Here, the coordinates of the points a and b are calculated by the following formula.

In this way, when the coordinates of points a 'and b' are calculated, the segment connecting A 'and a' (

) And the line connecting vertices B 'and b' (

The intersection point P 'of () is calculated.

Here, the coordinate of P 'is calculated by the following formula.

P '= CROSS (

,

)

When the coordinates of the intersection point P 'are calculated in this way, the mapping target pixel P is mapped in correspondence with the intersection point P'.

However, when two line segments passing through the mapping target pixel P have almost no difference in inclination close to any one line segment of the polygon, the program recognizes the two line segments almost horizontally and an error occurs. Therefore, in order to correct such an error, the reference vertex should be selected such that the slope of two line segments passing through the mapping target pixel P faces the line segment sufficiently larger than the slope of each line segment of the polygon.

FIG. 10 is a flowchart for explaining a process of selecting two reference vertices as reference points when mapping a polygon in an embodiment of the present invention, and FIG. 11 is a reference diagram for explaining the method of FIG. 10 and a point P in a triangle ABC. To map to the triangle A'B'C ', the coordinates of point P and the vertex ABC,

,

You need to select the line ratio of and choose the two best line ratios to map point P.

First, using the equation

Calculate the slope of (S100).

ab = SLOPE (A, B)

ac = SLOPE (A, C)

bc = SLOPE (B, C)

The distance between each vertex and P is compared, and the distance information between the vertex far from the point P and the point P is stored as dap (S200).

Specifically, compare the lengths of A and B at point p and store the long ones in dap, compare the lengths of A and C at point p, and store the long ones in dbp, and store points B and B, Compare the lengths of C and store the long one in dcp.

dap = DIST (A, B)

dbp = DIST (A, C)

dcp = DIST (B, C)

Subsequently, a vertex far from the point P is found to store information (S300).

Specifically, the length of the dap and the dbp is compared to the point P and the longest vertex is determined by comparing the length of the dap and the dbp as shown in the following formula, and stored in the ABP. Determine the farthest vertex and store it in the ACP, compare the dbp and dcp lengths to determine the longest point as the farthest vertex and store the longest in the BCP.

ABP = dap <dbp (= B) or dap> dbp (A)

ACP = dap <dcp (= C) or dap> dcp (A)

BCP = dbp <dcp (= C) or dbp> dcp (B)

Then, the slope between P and the vertex found in step S300 is calculated using the following formula (S400).

abp = SLOPE (P, ABP)

acp = SLOPE (P, ACP)

bcp = SLOPE (P, BCP)

Next, the difference between the slope calculated in step S100 and the slope calculated in step S400 is calculated by using the following formula (S500).

sdap = ABS (ab, abp)

sdbp = ABS (ab, abp)

sdap = ABS (ab, abp)

If the absolute value of the value calculated in this manner is taken, the slope difference can be calculated. As the value approaches 0, the line segment passing through the pixel to be mapped P is closer to each line segment of the distortion polygon.

Then, the line segment with the smallest difference in the calculated slopes (

), The corresponding segment (

) And vertex (B) facing each other is selected as the first reference vertex (S600), and the vertex (C) closest to the mapping target pixel (P) is selected as the second reference vertex (S700). Then, based on vertices B and C

,

Map using line segment ratio of. That is, a straight line passing through each selected vertex B and C and the mapping target pixel P

,

) And line segments facing each vertex (B, C)

,

The coordinates of the intersection points (b, c) of) are calculated (S800), the coordinate ratio of each intersection point (b, c) is calculated (S900), and mapped using the same.

12 is a diagram illustrating an around view image that is finally combined in a method of generating a vehicle around view according to the present invention. Each of the correction images generated by the above-described method is combined with a planar image of a vehicle to generate an around view image. (S90).

In this case, in combining a plurality of correction images, overlap images of corner areas adjacent to each other in the plurality of correction images should be removed. To this end, a reference line segment inclined inward from the outer edge portion of each of the corrected images is set, and adjacent correction images are overlapped so that the reference line lines coincide with each other, thereby eliminating duplicate images.

Next, after the overlapped image is removed, four calibrated images are arranged on the top, bottom, left, and right sides of the extracted plane image of the vehicle to generate one overall around view image.

Although the present invention has been described in connection with the above-mentioned preferred embodiments, it is possible to make various modifications or variations without departing from the spirit and scope of the invention. Accordingly, the appended claims will cover such modifications and variations as fall within the spirit of the invention.

Claims

In the method for generating an around view image by correcting the radiation distortion of the image obtained from a plurality of wide-angle camera,

Setting a reference coordinate value for each intersection point of a reference grid pattern and generating a reference image divided into a plurality of reference polygons having reference vertices A ', B', and C 'for the grid pattern image;

Capturing a check board pattern with a plurality of wide-angle cameras to obtain respective distortion images;

Detecting an edge of a check board pattern in each of the distorted images;

Estimating a plurality of correction points using the variation in brightness distribution in the vertical direction and the horizontal direction while performing line trekking along the detected edge region, and selecting a correction order for the estimated correction points;

Dividing each of the distorted images into a plurality of distorted polygons as correction vertices (A, B, and C);

Acquiring each corrected image by mapping respective pixel coordinates of the distorted image to reference coordinates so that the distorted polygon corresponds to the reference polygon according to the correction order; And

And generating an around view image by combining each of the corrected images and the planar image of the vehicle.
The method of claim 1,

The around view image generating step,

Remove overlapping images of adjacent corner regions from the plurality of correction images, and set reference line segments inclined inward from the outer edge portion of each of the correction images, and overlap the adjacent correction images so that the reference line segments coincide with each other. Vehicle around view image generation method, characterized in that for removing the image.
The method of claim 1,

The distortion polygon and the reference polygon mapping step,

And determining whether the mapping target pixel is located inside the distortion polygon, and performing mapping only when the mapping target pixel is located inside the distortion polygon.
The method of claim 3,

The mapping of pixel coordinates of the distorted image to reference coordinates may include:

The first reference vertex is selected from one of the vertices A, B, and C, and a coordinate of the intersection of the first reference vertex and the first straight line passing through the mapping target pixel P and the line segment facing the first reference vertex Calculating a first intersection point coordinate step;

One of the two vertices A, B, and C except for the first reference vertex is selected as a second reference vertex, and a second straight line passing through the second reference vertex and the mapping target pixel P and faces the second reference vertex. A second intersection coordinate calculating step of calculating intersection coordinates with the line segment to be viewed,

Calculating a first intersection coordinate ratio on the first straight line and a second intersection coordinate ratio on a second straight line;

The third intersection coordinate on the reference polygon is calculated by matching the first intersection coordinate ratio with the vertices A ', B', and C 'of the reference polygon, and the second intersection coordinate ratio is used as the vertex A', Calculating a fourth intersection coordinate on the reference polygon in correspondence with B ', C'),

Extract third and fourth vertices facing each of the third and fourth intersections, and extract the third segment connecting the third and third intersections and the fourth segment connecting the fourth and fourth vertices. Calculating an intersection point P ′ between the third line segment and the fourth line segment, and

And mapping the mapping target pixel (P) in correspondence with an intersection point (P ′).
The method of claim 4, wherein

The first reference vertex selection,

A first slope calculating step of calculating a slope of three segments of the distortion polygon,

Calculating a distance between the mapping target pixel P and each vertex A, B, or C;

Extracting three triangles by connecting the mapping target pixel P and each vertex, and extracting vertices far from the vertex and the mapping target pixel P with respect to each triangle;

A second slope calculation step of calculating vertices far from the mapping target pixel P, a mapping target pixel P, and a slope;

Calculating a difference between the first slope and the second slope and extracting a line segment having the smallest slope difference;

And selecting a vertex facing the line segment having the smallest difference in the slope as a first reference vertex.
The method of claim 5,

And the second reference vertex is selected as a vertex closest to the distance as a result of calculating the distance to the pixel to be mapped (P).