KR101239740B1 - An apparatus for generating around view image of vehicle using polygon mapping and multi look-up table - Google Patents

Abstract

PURPOSE: A vehicle around view image generation apparatus using polygon mapping and multiple lookup table is provided to correct more precise image distortion by correcting image distortion using a polygon mapping technique. CONSTITUTION: A image signal reception unit(20) receives a plurality of input image signals photographed from a plurality of wide-angle cameras(10). An image synthesis unit(30) synthesizes a plurality of the input images, a user interface image, and a parking support image by referring a first to a third LUT(Look Up Tables)(51-53). An image display unit(40) displays the synthesis image to one terminal output window. [Reference numerals] (10) Wide-angle camera; (20) Image signal reception unit; (30) Image synthesis unit; (40) Image display unit; (51) First look up table; (52) Second look up table; (53) Third look up table

Description

Vehicle around view image generator using polygon mapping and multi-lookup table {AN APPARATUS FOR GENERATING AROUND VIEW IMAGE OF VEHICLE USING POLYGON MAPPING AND MULTI LOOK-UP TABLE}

The present invention relates to an apparatus for generating an around view image of a vehicle using images obtained from a plurality of wide-angle cameras, and more particularly, to estimate a polygon coordinate point in a distorted image obtained through a wide-angle camera, and to estimate the estimated polygon coordinates. In the polygon mapping image distortion correction method for mapping a point to a reference image in a polygon, a wide angle camera using a multi-lookup table that stores a polygon coordinate correspondence and a coordinate correspondence between a user interface image and a parking assist image and a terminal output window Image distortion-corrected image, user interface image, and parking assist image can be displayed on one terminal output window, which can significantly reduce the computation time for image signal processing, As well as the surrounding footage of the vehicle In addition, the present invention relates to a technology for supporting parking more smoothly by simultaneously displaying parking assistance images.

Recently, with the development of IT technology, attempts to apply it to vehicles are increasing. For example, a black box device that mounts a camera in a vehicle to record driving conditions or surrounding conditions, or a parking assistance system that installs a camera in the rear of the vehicle to take a rearward image and output it to a display device inside the vehicle when the vehicle is reversed. This trend is reported to be increasing. Meanwhile, among these technologies, wide-angle cameras are installed in front, rear, left, and right sides of the vehicle, and the images captured from these cameras are reconstructed into images of the form directly above the vehicle, that is, viewed from above, and output to the display device of the vehicle. There has also been proposed a system for the driver's convenience.

Such a system is called a bird eye vie system or an AVM (around view monitoring) system in that it provides an image as if the bird is looking down from the sky. This technique uses a wide-angle camera with a fish eye lens to secure a wider viewing angle. When such a wide-angle camera is used, a distorted image is obtained as an initial image signal, thereby eliminating distortion. This requires a process to correct the image.

A process of correcting such a distorted image is generally performed by correcting a distorted image by using a warping equation, and the warping equation is expressed by a first, second and third order equation as shown in the following equation.

However, the distortion correction using the warping equation has a disadvantage that the distortion is not completely corrected because the image distortion of the outer portion remains because the correction is performed using the three-dimensional equation.

And, no matter what type of distortion image correction method is used, a complicated calculation process is required to correct the distortion image. Accordingly, the around view system according to the related art has a problem in that a computational process is complicated and needs to be processed in real time, thus requiring a large amount of computation and requiring high specification and expensive hardware equipment.

In order to solve this problem, Korean Patent No. 10-1077584 relates to an image processing apparatus and method for matching images obtained from a plurality of cameras, each of which constitutes a plurality of input image signals obtained from a plurality of cameras. Receive a plurality of input image signals from an image signal receiver using a look-up table that stores image mapping data for a relationship in which image pixels correspond to image pixels of a composite image signal, and correspond to each input image signal by referring to the look-up table. By generating and outputting a composite video signal, a technique has been proposed to enable a plurality of images acquired from a plurality of cameras to be quickly and efficiently configured into a single planarized image.

However, this conventional technique uses a single lookup table. As shown in FIG. 1, only a relationship between a photographed image of a camera and a composite image in which distortion is corrected is stored in a single lookup table. Since it is separately stored, and thus the around view image and the user interface image are generated in each window and output on the screen, there is a problem that image processing efficiency is lowered.

In addition, the conventional around view system displays only the around view image and does not display a parking assist image such as an image (rear sense signal image) that visualizes the detection signal of the rear sensor in the around view image so that satisfactory parking support is achieved. It is not supported.

The present invention has been made to solve the above problems, an object of the present invention is to correct the image distortion by using a polygon mapping method without using the warping equation, and to achieve a more accurate image distortion correction, and multi-lockup table By using this, the distortion correction of the wide-angle camera image, the user interface image, and the parking assist image can be displayed on one terminal output window, thereby significantly reducing the computation time for image signal processing.

Another object of the present invention relates to a technique for supporting smoother parking by enabling to simultaneously display not only the surrounding image of the vehicle but also the parking assistance image within the around view image.

According to a preferred embodiment of the present invention for achieving the above object, a plurality of wide-angle camera installed to photograph the four sides of the vehicle, an image signal receiving unit for receiving a plurality of input image signals taken from the plurality of wide-angle camera, By using a polygon mapping method for estimating polygon coordinate points in the distorted image obtained by the wide-angle camera and polygon mapping the estimated polygon coordinate points to the reference image, between the polygon coordinate points of the distorted image and the polygon coordinate points of the reference image. A first lookup table for storing first image mapping data for a correspondence relationship, and a second image mapping data for a relationship in which image pixels constituting a pre-generated user interface image correspond to image pixels on a terminal output window. Video Pixel Composing 2 Lookup Table, Parking Support Video Lookup table for storing third image mapping data of a relationship corresponding to an image pixel on a terminal output window, the plurality of input images, the user interface image, and the parking assist image with reference to the first to third lookup tables. There is provided an apparatus for generating a vehicle around view image using a polygon mapping and a multi-lookup table, comprising: an image synthesizer for synthesizing an image and an image display unit for displaying the synthesized image on one terminal output window.

The image display unit may include a user interface display unit, a rear camera image display unit, and an around view image display unit.

The user interface image may include at least one of an image constituting the user interface display unit and a vehicle image displayed at the center of the around view image display unit, and the parking assist image is displayed on the rear camera image display unit. The image may include at least one of a vehicle trajectory image and a rear sensing signal image displayed on the around view image display unit.

The rearward sensing signal image is represented by a plurality of arcs representing distances to the rear obstacles to be sensed, and a corresponding relationship between the distances of the rear obstacles and the arcs is stored in a third lookup table, The output time and output duty cycle information of the corresponding unit alarm sound are also preferably stored in the third lookup table.

The first image mapping data sets reference coordinate values for each intersection point of the reference grid pattern, and divides the first image mapping data into a plurality of reference polygons having reference vertices A ', B', and C 'for the grid pattern image. Generating a reference image, photographing a check board pattern with a plurality of wide-angle cameras to obtain respective distortion images, detecting edges of the check board patterns in each of the distortion images, and lines along the detected edge regions Estimating a plurality of correction points using the variation in brightness distribution in the vertical direction and the horizontal direction while performing trekking, and selecting a correction order for the estimated correction points, wherein the correction vertices A Dividing the plurality of distorted polygons into (B, C), so that the distorted polygon corresponds to the reference polygon according to the correction order. Mapping each pixel coordinate to a reference coordinate, determining whether the pixel to be mapped is located inside the distortion polygon, and matching coordinates of the points inside the distortion polygon at the same ratio as the coordinate inside the reference polygon. It is preferable to obtain through the step of calculating the image mapping data.

In addition, matching of the pixel coordinates of the distorted image to the reference coordinates may include selecting a first reference vertex from any one of the vertices A, B, and C, and passing the first reference vertex and the mapping target pixel P. A first intersection coordinate calculation step of calculating intersection coordinates of a straight line and a line segment facing the first reference vertex, and selecting one of the vertices A, B, and C except for the first reference vertex as the second reference vertex And a second intersection coordinate calculation step of calculating intersection coordinates of a second straight line passing through the second reference vertex and the pixel to be mapped P and the line segment facing the second reference vertex, and the first on the first straight line. Calculating an intersection coordinate ratio and a second intersection coordinate ratio on a second straight line, and matching the first intersection coordinate ratio to vertices A ', B', and C 'of the reference polygon to coordinate with the third intersection point on the reference polygon. To calculate Calculating a fourth intersection coordinate on the reference polygon by mapping the second intersection coordinate ratio to vertices A ', B', and C 'of the reference polygon, and facing each of the third and fourth intersection points. The third vertex and the fourth vertex are extracted, and the third segment connecting the third vertex and the third intersection and the fourth segment connecting the fourth intersection and the fourth vertex are extracted, and the intersection point of the third segment and the fourth segment ( Calculating P ′) and mapping the mapping target pixel P corresponding to the intersection point P ′.

The first reference vertex selection may include calculating a first slope of three lines of the distorted polygon, and calculating a distance between the mapping target pixel P and the vertices A, B, and C. And obtaining three triangles by connecting the mapping target pixel P and each vertex, and extracting vertices far from the vertex and the mapping target pixel P for each of the triangles; A second slope calculation step of calculating vertices far from the pixel P, a mapping target pixel P, and a slope; calculating a difference between the first slope and the second slope and extracting a line segment having the smallest difference in slope; And selecting a vertex facing the line segment having the smallest difference in the slope as a first reference vertex, and the second reference vertex has a distance calculated as a result of the distance from the pixel to be mapped (P). It can be selected as the closest vertex.

According to the present invention, by correcting the image distortion by using the polygon mapping method can be more accurate image distortion correction, and the program performance is improved not only by improving the time and computation required to generate the around view image, Compared to this, even a device with a lower specification can provide equivalent performance, which can lower the product cost and thereby significantly improve product competitiveness.

1 is a view illustrating a concept of displaying an around view image by an around view image generating apparatus using a conventional single lookup table.
2 is a block diagram illustrating an apparatus for generating a vehicle around view image using a multiple lookup table according to the present invention;
3 is a view illustrating a concept of displaying an around view image by a vehicle around view image generating apparatus using a multiple lookup table according to the present invention;
4 illustrates a process of generating a rear sensing signal image.
5 is a flowchart sequentially illustrating a method of generating a vehicle around view image according to the present invention.
6 to 11 are reference views for explaining a process of generating a vehicle around view image according to the present invention.
12 is a view for explaining a process for determining whether a pixel to be mapped is located inside a distortion polygon according to the present invention.
13 is a view for explaining a method of mapping a distortion polygon to a reference polygon in accordance with the present invention.
FIG. 14 is a flowchart illustrating a process of selecting two reference vertices which are references when polygons are mapped in an embodiment of the present invention. FIG.
FIG. 15 is a reference diagram for explaining the method of FIG. 14; FIG.

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the present invention.

2 is a block diagram illustrating an apparatus for generating a vehicle around view image using a multiple lookup table according to an embodiment of the present invention, and FIG. 3 is a concept in which an around view image is displayed by an apparatus for generating a vehicle around view image using a multiple lookup table according to the present invention. 4 illustrates a process of generating a rear sensing signal image.

Referring to FIG. 2, the apparatus for generating a vehicle around view image using a multi-lookup table according to the present invention includes a plurality of wide-angle cameras 10, image signal receivers 20, image synthesizers 30, and image display units 40. And a memory 50.

The wide-angle camera 10 is disposed to partially overlap the photographing area between adjacent wide-angle cameras, and is composed of at least two or more pieces, each of which photographs an image, converts the image into an electrical signal, and transmits the image signal to the image signal receiving unit 20. Preferably, the wide-angle camera 10 is installed in the front, rear, left, right of the vehicle so as to capture the front, rear, left, right image of the vehicle.

The image signal receiving unit 20 is a means for receiving a plurality of input image signals respectively obtained from the plurality of wide-angle cameras 10. The image signal receiving unit 20 pretransmits the received plurality of input image signals and transmits them to the image synthesizing unit 30. .

When the image synthesizing unit 30 receives a plurality of input image signals from the image signal receiving unit 20, the image synthesizing unit 30 performs a function of synthesizing the plurality of input images, the user interface image, and the parking assist image by referring to the lookup table of the memory 50. Perform.

The image display unit 40 displays a composite image on one terminal output window, and may be displayed in the form of the image on the right side of FIG. 3, which is a user interface display unit 61 displaying a user interface screen and a rear side. And a rear camera image display unit 62 displaying an image photographed by the wide-angle camera 10 installed in the display unit, and an around view image display unit 63 displaying surrounding images of the vehicle around the trajectory shape of the vehicle.

The memory 50 includes first to third lookup tables in which correspondences between the input image signal, the user interface image, the parking assist image, and the image pixel on the terminal output window are recorded.

The first lookup table 51 estimates polygon coordinate points in the distorted image obtained by the wide-angle camera 10 and uses a polygon mapping method to perform polygon mapping on the estimated polygon coordinate points to the reference image. The first image mapping data for the correspondence between the coordinate point and the polygon coordinate point of the reference image is stored.

The first image mapping data of the correspondence relationship between the captured polygon coordinate point and the polygon coordinate point of the reference image indicates the coordinates when the pixels of the distorted image are corrected. When the correspondence relationship between the coordinate information of each pixel is recorded in the first lookup table 51, the distortion image can be quickly corrected by referring to the first lookup table 51 without calculation for polygon mapping when generating the composite image. Will be. Such a polygon mapping method will be described in detail with reference to FIGS. 5 to 11.

The second lookup table 52 stores second image mapping data about a relationship in which image pixels constituting a pre-generated user interface image correspond to image pixels on a terminal output window. The user interface image includes an image constituting the user interface display unit and a vehicle image displayed at the center of the around view image display unit 63. The user interface image is generated in advance by a program maker, and records coordinate information when each pixel constituting the user interface image is displayed on the actual terminal output window as a lookup table.

The third lookup table 53 stores third image mapping data for a relationship in which image pixels constituting the parking assist image correspond to image pixels on the terminal output window. The parking assist image includes a vehicle trajectory image displayed on the rear camera image display unit and a rear sensing signal image displayed on the around view image display unit, and the rear sensing signal image is generated by the method as illustrated in FIG. 4. Coordinate information when each of the pixels constituting the subframe is displayed on the actual terminal output window is recorded as a lookup table. The rear sensing signal image is a sensing signal output when the rear obstacle is detected by the rear sensing sensor. The rear sensing signal image may be displayed as a plurality of arc images representing a distance from the rear obstacle to be detected. The relationship is stored in the third lookup table. Each arc image is displayed differently from the vehicle image according to the distance between the obstacle and the vehicle so that the user can easily determine the distance from the obstacle.

Preferably, the third lookup table 53 further stores output time and output duty cycle information of the unit alarm sound corresponding to each arc.

As shown in FIG. 3, the image synthesizing unit 40 refers to the first to third image mapping data regarding the correspondence relationship between the photographed distortion image, the user interface image, the parking assist image, and the terminal output window, and the image synthesizer 40 displays the image on the single window. User interface video, around view video and parking assist video can be displayed on the screen, greatly simplifying the image processing process.

Hereinafter, a method of generating a rear sensing signal image will be described with reference to FIG. 4.

First, as in (a), the coordinates of (x1, y1) and (x3, y3) are known coordinates when constructing the UI, and point A is the center of two coordinates. We can find A (x2, y2). That is, x2 = x3-x1 and y2 = y3-y1. The coordinates of x2 and x4 are the same.

For example, if the x-axis length of the actual vehicle is 1865 mm and the y-axis length is 4895 mm, the ratio of the pixel size to y = 2.62x is obtained. By specifying the X-axis of the vehicle image to be used, the length of the pixel on the Y-axis can be known. Since the coordinates of the point are the center points of the X-axis pixels of the vehicle image, the X-axis coordinates can be obtained.

The length of line segment AC is the pixel length of the vehicle image Y axis. This assumes three sensors and positions them. Since points B, C, and D are the positions of the rear sensors running on the actual vehicle, the ratio can be used to obtain coordinates to be located in the vehicle image.

As shown in (b), the coordinates of the midpoints of the line segments BC and the line segment CD are calculated to obtain the width of the rear sense signal.

의 수식을 이용하여 선분 AB의 길이를 구한다. 그리고, 선분 BB'와 선분 BB"의 길이가 같고,

이므로 B'와 B"의 좌표를 구할 수 있으며, 동일한 방법으로 점 D'와 점 D"의 좌표를 구한다.From (c), from the lengths of line segments AC and BC

Find the length of line segment AB using the equation And the length of line segment BB 'and line segment BB "is the same,

Therefore, the coordinates of B 'and B "can be obtained, and the coordinates of the points D' and D" are obtained in the same manner.

As shown in (d), the arc indicates a range in which the rear sensing signals appear, and when three rear sensing sensors are used, three signal range intervals can be determined.

As shown in (e), when the information on the distance from the initial position summarized above is arranged in the third lookup table 53 and the rear sensing signal image is displayed, the corresponding value is called, and the rear sensing signal image is ( As shown in f), it is displayed together with the around image so that the driver can easily recognize the obstacle behind the vehicle.

When the rear detection signal image is displayed, it is preferable that the alarm sound is output at the same time. The length and duty cycle of the alarm sound may vary depending on the distance from the obstacle.

That is, for example, as shown in Table 1 below, the distance between the vehicle and the obstacle can be divided into eight sections, and the length and duty cycle of the alarm sound can be varied for each section, and the closer the obstacle is to the closest region 1 The priority of the alarm may be increased so that the period of the alarm sound and the length of the alarm sound within one period may be increased.

Hereinafter, the principle of correcting distortion of an image through polygon mapping will be described in detail with reference to FIGS. 5 to 11.

5 is a flowchart sequentially illustrating a method of generating a vehicle around view image according to the present invention, and FIGS. 6 to 11 are reference views for explaining a process of generating a vehicle around view image according to the present invention. The following processing may be performed via a PC or may be processed by a process inside the vehicle.

First, the present invention generates a reference grid pattern corresponding to the reference check board pattern 10 shown in FIG. 6 (S10), and sets reference coordinate values for each intersection point of the grid pattern (S20). In operation S30, the reference image 20 in which the grid pattern image is divided into a plurality of reference polygons 30 is set. In this case, the plurality of reference polygons 30 are formed in a polygonal shape having vertices A ', B', and C '.

Next, a plurality of wide-angle cameras respectively photograph the check board patterns to obtain a distorted image 40 as shown in FIG. 7 (S40).

Subsequently, the check board pattern of the distorted image 40 is recognized, and an edge of the check board pattern in the distorted image is detected as shown in FIG. 8 through morphology and robust operation (S50).

When the edge of the check board pattern is detected, a plurality of correction points are estimated and a correction order for the estimated correction points is selected (S60). In this case, the correction point estimation is performed using the change in brightness distribution in the vertical direction and the horizontal direction while performing line tracking along the detected edge region as shown in FIG. 9. That is, since a large amount of change in brightness appears at each intersection in the check board pattern, the point where the change in brightness is large in the line tracking process is estimated as a correction point, and is corrected in the order estimated by the correction point as shown in FIG. 10. Select the order.

Subsequently, the distortion points are divided into a plurality of distortion polygons having correction points A, B, and C (S70).

As shown in FIG. 11, each corrected image is obtained by mapping each pixel coordinate of the distorted image to the reference coordinate so that the distorted polygon corresponds to the reference polygon in the selected correcting order (S80). That is, each pixel in the distortion polygon 50 is mapped to the reference polygon 30 of the reference image. In this case, it is determined whether the pixel to be mapped is located inside the distortion polygon. If the pixel to be mapped is located inside the distortion polygon, the coordinates of the pixel are mapped to the reference coordinate. If the pixel to be mapped is not inside the distortion polygon, the mapping is performed. Exclude from the target.

Hereinafter, a process of mapping a distortion polygon according to the present invention to a reference polygon will be described in more detail with reference to FIGS. 12 to 15.

12 is a view for explaining a process for determining whether a pixel to be mapped is located inside a distortion polygon according to the present invention. First, as shown in FIG. Check if it is located within the distortion polygon consisting of B and C. To this end, it is necessary to obtain the coordinates of the points D and E of the square including vertices A, B, and C as shown in FIG.

At this time, the coordinates of D and E can be obtained using the following equation.

If such a coordinate is obtained and a square range including the vertices A, B, and C of the pixel to be mapped P is specified, a part outside the coordinate may be defined as outside. At this time, the outside range condition is if (P _x <D _x ∥P _x > E _x ∥ P _y <D _y ∥P _y > E _y ). You can see that is located inside the rectangle.

와

가 만나는 점 F의 좌표를 찾는다. 이때, F는 아래의 수식에 의해 구할 수 있다.In addition, as shown in (c) of FIG. 12, in order to check whether the mapping target pixel P is located inside a polygon including vertices A, B, and C.

Wow

Find the coordinate of the point F where In this case, F can be obtained by the following equation.

,

)F = cross (

,

)

)를 벗어난 경우로서 매핑 대상 픽셀(P)이 OUT1 또는 OUT2 영역에 위치하는 경우에 해당한다. In this way, when the coordinates of the point F are obtained, it is checked whether the mapping target pixel P is located outside the distortion polygon. Here, the condition for the pixel to be mapped (P) outside the distortion polygon is the outer range condition if (F _x <D _x ∥ F _x > E _x ∥ F _y <D _y ∥ F _y > E _y ), Point F is a line segment

This is a case where the mapping target pixel P is located in the OUT1 or OUT2 region.

As shown in FIG. 12D, the coordinates of the points G and H of the quadrangle of which points B and F connected to the vertex of the pixel P to be mapped are obtained.

At this time, the coordinates of G and H can be obtained using the following equation.

In this way, when the coordinates of G and H are obtained, it is checked whether the mapping target pixel P is located outside the rectangle BGFH. Here, the outside range condition for the mapping target pixel P to be located outside the distortion polygon is if (P _x <G _x ∥ P _x > H _x ∥P _y <G _y ∥P _y > H _y ). The case where the pixel P is located outside the rectangle BGFH corresponds to the case where the pixel to be mapped is located in the OUT3 region.

As a result of checking the outside range condition, if the outside range condition does not correspond, the pixel to be mapped P is located inside the distortion polygon.

If it is determined that the mapping target pixel P is located inside the distortion polygon, the corresponding pixel is mapped to correspond to the reference image.

) 및 제 1 기준 꼭지점(A)과 마주보는 선분(

)이 교차하는 제 1 교차점(a)의 좌표를 산출한다. 그리고, 제 1 기준 꼭지점(A)을 제외한 둘 중 하나를 제 2 기준 꼭지점(B)으로 선정하고, 제 2 기준 꼭지점(B)과 매핑 대상 픽셀(P)을 지나는 제 2 직선(

) 및 제 2 기준 꼭지점(B)과 마주보는 선분(

)이 교차하는 제 2 교차점(b)의 좌표를 산출한다. FIG. 13 is a view for explaining a method of mapping a distortion polygon to a reference polygon according to the present invention. First, any one of vertices A, B, and C is selected as a first reference vertex A, and a first reference vertex ( A first straight line passing through A) and the mapping target pixel P

) And the line segment facing the first reference vertex (A)

The coordinate of the 1st intersection point (a) which () intersects is computed. Then, one of the two except for the first reference vertex A is selected as the second reference vertex B, and a second straight line passing through the second reference vertex B and the mapping target pixel P (

) And the line segment facing the second reference vertex (B)

The coordinate of the 2nd intersection point (b) which () intersects is computed.

Here, the coordinates of the first intersection point a and the second intersection point b are obtained by the following equation.

,

)a = CROSS (

,

)

,

)b = CROSS (

,

)

에 대한 b의 좌표비(

:

) 및 선분

에 대한 a의 좌표비(

:

)를 산출하고, 산출된 b의 좌표비(

:

) 및 a의 좌표비(

:

)를 기준 폴리곤의 꼭지점(A',B',C')에 대응시켜 선분

상의 점 a'의 좌표와 선분

상의 점 b '의 좌표를 산출한다.Then, line segment

Coordinate ratio of b to

:

) And line segment

Coordinate ratio of a to

:

) And calculate the coordinate ratio of

:

) And the coordinate ratio of a (

:

) Corresponding to the vertices (A ', B', C ') of the reference polygon

And line segment at point a 'on Pinterest

Calculate the coordinates of point b 'on the phase.

Here, the coordinates of the points a and b are calculated by the following formula.

)와 꼭지점 B'와 b'를 잇는 선분(

)의 교차점(P')를 산출한다. In this way, when the coordinates of points a 'and b' are calculated, the segment connecting A 'and a' (

) And the line connecting vertices B 'and b' (

The intersection point P 'of () is calculated.

Here, the coordinate of P 'is calculated by the following formula.

,

)P '= CROSS (

,

)

When the coordinates of the intersection point P 'are calculated in this way, the mapping target pixel P is mapped in correspondence with the intersection point P'.

However, when two line segments passing through the mapping target pixel P have almost no difference in inclination close to any one line segment of the polygon, the program recognizes the two line segments almost horizontally and an error occurs. Therefore, in order to correct such an error, the reference vertex should be selected such that the slope of two line segments passing through the mapping target pixel P faces the line segment sufficiently larger than the slope of each line segment of the polygon.

,

,

의 선분비가 필요하며, 점 P를 매핑하기 위해 최적의 선분비 2개를 선택해야 한다.FIG. 14 is a flowchart for explaining a process of selecting two reference vertices as reference points when mapping a polygon in an embodiment of the present invention, and FIG. 15 is a reference diagram for explaining the method of FIG. 14 and a point P in a triangle ABC. To map to the triangle A'B'C ', the coordinates of point P and the vertex ABC,

,

,

You need to select the line ratio of and choose the two best line ratios to map point P.

)의 기울기를 산출한다(S100). First, using the equation

Calculate the slope of (S100).

ab = SLOPE (A, B)

ac = SLOPE (A, C)

bc = SLOPE (B, C)

The distance between each vertex and P is compared, and the distance information between the vertex far from the point P and the point P is stored as dap (S200).

Specifically, compare the lengths of A and B at point p and store the long ones in dap, compare the lengths of A and C at point p, and store the long ones in dbp, and store points B and B, Compare the lengths of C and store the long one in dcp.

dap = DIST (A, B)

dbp = DIST (A, C)

dcp = DIST (B, C)

Subsequently, a vertex far from the point P is found to store information (S300).

Specifically, the length of the dap and the dbp is compared to the point P and the longest vertex is determined by comparing the length of the dap and the dbp as shown in the following formula, and stored in the ABP. Determine the farthest vertex and store it in the ACP, compare the dbp and dcp lengths to determine the longest point as the farthest vertex and store the longest in the BCP.

ABP = dap <dbp (= B) or dap> dbp (A)

ACP = dap <dcp (= C) or dap> dcp (A)

BCP = dbp <dcp (= C) or dbp> dcp (B)

Then, the slope between P and the vertex found in step S300 is calculated using the following formula (S400).

abp = SLOPE (P, ABP)

acp = SLOPE (P, ACP)

bcp = SLOPE (P, BCP)

Next, the difference between the slope calculated in step S100 and the slope calculated in step S400 is calculated by using the following formula (S500).

sdap = ABS (ab, abp)

sdbp = ABS (ab, abp)

sdap = ABS (ab, abp)

If the absolute value of the value calculated in this manner is taken, the slope difference can be calculated. As the value approaches 0, the line segment passing through the pixel to be mapped P is closer to each line segment of the distortion polygon.

)을 찾고, 해당 선분(

)과 마주보는 꼭지점(B)를 제 1 기준 꼭지점으로 선정하고(S600), 매핑 대상 픽셀(P)과 가장 가까운 꼭지점(C)을 제 2 기준 꼭지점으로 선정한다(S700). 이어서, 꼭지점 B,C를 기준으로 하는

,

의 선분비를 이용하여 매핑이 이루어진다. 즉, 선택된 각 꼭지점(B,C)과 매핑 대상 픽셀(P)를 지나는 직선(

,

)과, 각 꼭지점(B,C)과 마주보는 선분(

,

)의 교차점(b,c)의 좌표를 산출하고(S800), 각 교차점(b,c)의 좌표비를 산출하고(S900), 이를 이용하여 왜곡 폴리곤에 속하는 화소들과 기준 폴리곤 간에 속하는 화소들 간의 매핑이 이루어진다.Then, the line segment with the smallest difference in the calculated slopes (

), The corresponding segment (

) And vertex (B) facing each other is selected as the first reference vertex (S600), and the vertex (C) closest to the mapping target pixel (P) is selected as the second reference vertex (S700). Then, based on vertices B and C

,

The mapping is done using the line segment ratio of. That is, a straight line passing through each selected vertex B and C and the mapping target pixel P

,

) And line segments facing each vertex (B, C)

,

The coordinates of the intersection points (b, c) of () are calculated (S800), the coordinate ratio of each intersection point (b, c) is calculated (S900), and the pixels belonging between the pixels belonging to the distortion polygon and the reference polygon using the same. The mapping between the two takes place.

Therefore, when the mapping relationship between the pixels belonging to the distortion polygon and the pixels belonging to the reference polygon is stored in the first lookup table 51 as the first image mapping data, an around view image is obtained by using images captured from wide-angle cameras. At the time of generation, the composite image may be directly generated from the captured image by referring to the first lookup table 51 without a separate calculation process.

Although the present invention has been described in connection with the above-mentioned preferred embodiments, it is possible to make various modifications and variations without departing from the spirit and scope of the invention. Accordingly, the appended claims are intended to cover such modifications or changes as fall within the scope of the invention.

10: wide angle camera 20: video signal receiver
30: image synthesizer 40: image display unit
50: memory 51: the first lookup table
52: second lookup table 53: third lookup table
61: user interface display unit 62: rear camera image display unit
63: around view video display unit

Claims (9)

A vehicle around view image generating apparatus for generating and displaying an around view image by using images acquired from a plurality of wide-angle cameras.
A plurality of wide-angle cameras installed to photograph all sides of the vehicle;
An image signal receiver configured to receive a plurality of input image signals photographed from a plurality of wide angle cameras;
By using a polygon mapping method for estimating polygon coordinate points in the distorted image obtained by the wide-angle camera and polygon mapping the estimated polygon coordinate points to the reference image, between the polygon coordinate points of the distorted image and the polygon coordinate points of the reference image. A first lookup table that stores first image mapping data for a correspondence relationship;
A second lookup table configured to store second image mapping data of a relationship in which image pixels constituting a pre-generated user interface image correspond to image pixels on a terminal output window;
A third lookup table configured to store third image mapping data of a relationship between rear sensing signal image pixels displayed in the vehicle around view image corresponding to the image pixel on the terminal output window when the vehicle rear obstacle is detected;
An image synthesizer configured to synthesize the plurality of input images, a user interface image, and a parking assist image by referring to the first to third lookup tables; And
And an image display unit displaying the composite image on one terminal output window. 2. The method of claim 1,
And the image display unit comprises a user interface display unit, a rear camera image display unit, and an around view image display unit. The method of claim 2,
The user interface image may include at least one of an image constituting the user interface display unit and a vehicle image displayed at the center of the around view image display unit, and polygon mapping and vehicle around view image generation using the multi-lookup table. Device. The method of claim 2,
The parking assist image includes at least one of a vehicle trajectory image displayed on the rear camera image display unit and a rear sensing signal image displayed on the around view image display unit, and polygon mapping and vehicle around using the multiple lookup table. View image generator. The method of claim 4, wherein
The rear sensing signal image is represented by a plurality of arcs representing distances to the rear obstacles to be detected, and a corresponding relationship between the distances to the rear obstacles and the arcs is stored in a third lookup table. Polygon mapping and vehicle around view image generator using The method of claim 5, wherein
And the third lookup table further stores output time and output duty cycle information of a unit alarm sound corresponding to each arc, and includes a multiple lookup table. The method of claim 1,
The first image mapping data is
Setting a reference coordinate value for each intersection point of a reference grid pattern and generating a reference image divided into a plurality of reference polygons having reference vertices A ', B', and C 'with respect to the grid pattern image; Acquiring each distortion image by capturing a check board pattern with a wide-angle camera; detecting edges of the check board pattern in each distortion image; and performing line trekking along the detected edge region; Estimating a plurality of correction points by using the change in brightness distribution to and determining a correction order for the estimated correction points, and setting the correction points as vertices A, B, and C for the distorted images. Dividing into a plurality of distortion polygons, and mapping each pixel coordinate of the distorted image to reference coordinates such that the distortion polygon corresponds to the reference polygon according to the correction order Determining whether the pixel to be mapped in the distorted image is located in the distorted polygon and calculating coordinates of points in the distorted polygon at the same ratio as coordinates in a reference polygon to calculate the first image mapping data. Polygon mapping and vehicle around view image generating apparatus using a multi-lookup table, characterized in that obtained through the step. The method of claim 7, wherein
The mapping of each pixel coordinate of the distorted image to the reference coordinate so that the distortion polygon corresponds to the reference polygon according to the correction order
The first reference vertex is selected from one of the vertices A, B, and C, and a coordinate of the intersection of the first reference vertex and the first straight line passing through the mapping target pixel P and the line segment facing the first reference vertex A step of calculating a first intersection point to be calculated and selecting one of the vertices A, B, and C except for the first reference vertex as the second reference vertex, and passing the second reference vertex and the mapping target pixel P; Calculating a second intersection coordinate with a second straight line and a line segment facing the second reference vertex; and calculating a first intersection coordinate ratio on the first straight line and a second intersection coordinate ratio on a second straight line. And calculating a third intersection coordinate on the reference polygon by mapping the first intersection coordinate ratio to vertices A ', B', and C 'of the reference polygon, and using the second intersection coordinate ratio as a vertex of the reference polygon. A ', B', C ') Correspondingly calculating a fourth intersection coordinate on the reference polygon, extracting third and fourth vertices facing each of the third and fourth intersection points, and a third segment connecting the third and third intersection points. And extracting a fourth line segment connecting the fourth intersection point and the fourth vertex to calculate an intersection point P ′ of the third line segment and the fourth line segment, and converting the mapping target pixel P to the intersection point P ′. A vehicle around view image generating apparatus using polygon mapping and multiple lookup tables, characterized in that the mapping is performed in correspondence. The method of claim 8,
The first reference vertex selection may include calculating a first slope of three segments of the distorted polygon, calculating a distance between the pixel to be mapped P and the vertices A, B, and C; Obtaining three triangles by connecting the mapping target pixel P and each vertex, extracting vertices having the greatest distance between the vertex and the mapping target pixel P with respect to each of the triangles; A second slope calculation step of calculating vertices farthest from (P), the mapping target pixel P, and a slope; calculating a difference between the first slope and the second slope, and extracting a line segment having the smallest slope difference; And selecting a vertex facing the line segment having the smallest difference in the slope as a first reference vertex,
And the second reference vertex is selected as a vertex having the closest distance as a result of the distance calculation with the pixel to be mapped (P).

Images