WO2018050224A1

WO2018050224A1 - Method for determining a plane of a roadway, driver assistance system as well as motor vehicle

Info

Publication number: WO2018050224A1
Application number: PCT/EP2016/071744
Authority: WO
Inventors: Andreas WESTHUES; Ciáran HUGHES; Duong-Van NGUYEN; Jonathan Horgan
Original assignee: Connaught Electronics Ltd.
Priority date: 2016-09-15
Filing date: 2016-09-15
Publication date: 2018-03-22

Abstract

The invention relates to a method for determining a plane (13) of a roadway (12), in which an image (10) is captured by means of a camera (4) of a motor vehicle (1) located on the roadway (12), wherein the image (10) describes an environmental region (8) of the motor vehicle (1), which includes a part of the roadway (12), a plurality of ground points (p, p_0, p_1, p₂, p₃) is determined in the image (10), which are associated with the roadway (12) in the image (10), wherein a positional value describing a spatial position in the environmental region (8) is associated with each of the ground points (p, p_0, p_1, p₂, p₃), and the plane (13) of the roadway (12) is determined based on the ground points (p, p_0, p_1, p₂, p₃), wherein the ground points (p, p_0, p_1, p₂, p₃) are sorted based on their positional values depending on their distance (r) to the camera (4), a subset is iteratively formed from the ground points (p, p_0, p_1, p₂, p₃) depending on their height starting from the ground point (p0) closest to the camera (4), and the plane (13) of the roadway (12) is determined based on the subset of the ground points (p, p_0, p_1, p₂, p₃).

Description

Method for determining a plane of a roadway, driver assistance system as well as motor vehicle

The present invention relates to a method for determining a plane of a roadway, in which an image is captured by means of a camera of a motor vehicle located on the roadway, wherein the image describes an environmental region of the motor vehicle, which includes a part of the roadway, a plurality of ground points is determined in the image, which are associated with the roadway in the image, wherein a positional value describing a spatial position in the environmental region is associated with each of the ground points, and the plane of the roadway is determined based on the ground points. Moreover, the present invention relates to a driver assistance system for a motor vehicle. Finally, the present invention relates to a motor vehicle with such a driver assistance system.

Presently, the interest is in particular directed to driver assistance systems for motor vehicles, which serve for assisting the driver in driving the motor vehicle. Driver assistance systems are known from the prior art hereto, which for example include at least one sensor, by which an environmental region of the motor vehicle can be captured. In particular, it can be checked if an obstacle to the motor vehicle is located in the environment of the motor vehicle by the at least one sensor. For example, the sensor can be a camera, by means of which an image or an image sequence of the environmental region can be provided. With the aid of a corresponding object recognition algorithm, an obstacle in the environment of the motor vehicle can then be recognized.

In 3D object recognition algorithms used heretofore, it is usually assumed that the motor vehicle moves on a uniform plane of the roadway. In addition, it is taken into account that the camera is in a predetermined installation position. In the 3D object recognition algorithms, individual object points are recognized, and a positional value is then determined for each of the object points, which describes the spatial position of the environmental region. All of the points, the height of which falls below a predetermined threshold value, are interpreted as ground points associated with the ground or the roadway surface. The object points, the height of which is greater than the threshold value, are interpreted as objects or obstacles to the motor vehicle.

Hereto, DE 10 201 1 1 18 171 A1 describes a method for continuously estimating a roadway plane of a motor vehicle from image sequences of an image capturing device. Therein, current 3D points of the surroundings of the motor vehicle are determined from the current image of the image sequence. Further, a first selection of the current 3D points with respect to the horizontal component is performed for obtaining a preselected set. Thereafter, performing a second selection within the preselected set with respect to the vertical component for obtaining a selected set of current 3D points and the estimation of the current roadway plane at least based on the selected set of current 3D points are effected.

It is the object of the present invention to demonstrate a solution, how a plane of a roadway, on which the motor vehicle is located, can be more reliably determined.

According to the invention, this object is solved by a method, by a driver assistance system as well as by a motor vehicle having the features according to the respective independent claims. Advantageous implementations of the invention are the subject matter of the dependent claims, of the description and of the figures.

A method according to the invention serves for determining a plane of a roadway, in which an image is captured by means of a camera of a motor vehicle located on the roadway, wherein the image describes an environmental region of the motor vehicle, which includes a part of the roadway. A plurality of ground points is determined in the image, which are associated with the roadway in the image, wherein a positional value describing a spatial position in the environmental region is associated with each of the ground points. The plane of the roadway is determined based on the ground points. Further, the ground points are sorted based on their positional values depending on their distance to the camera, a subset is iteratively formed from the ground points depending on their height starting from the ground point closest to the camera, and the plane of the roadway is determined based on the subset of the ground points.

With the aid of the method, a plane of a roadway or a roadway surface is to be

determined, on which the motor vehicle is located or travels. In particular, an inclination or orientation of the plane of the roadway relative to the motor vehicle is to be determined. Basically, the roadway can be a part of the ground, on which the motor vehicle is located. For example, the roadway can be an asphalted road, a gravel path, a meadow or another terrain section. An image is captured or provided by at least one camera of the motor vehicle, which at least partially depicts the roadway. A plurality of ground points is determined in the image, which describe the roadway. Hereto, a corresponding object recognition algorithm can for example be used, by which objects or object points in the image can be recognized. In particular, a three-dimensional object recognition algorithm can be used, which additionally provides a positional value for each of the recognized object points, which describes the spatial position of the object point in the environmental region. Thus, those object points, the height of which falls below a predetermined threshold value, can for example be associated with the roadway or the ground and be designated as ground points.

According to the invention, it is now provided that the object points, which have already been identified as ground points, or the object points, which have been associated with the ground or the roadway surface, are examined in more detail. Hereto, the ground points are examined in more detail with respect to their positional values. The positional values can describe the position of the ground points with respect to a camera coordinate system. The respective positional values describe the distance of the ground points to the camera on the one hand and the height of the respective ground points on the other hand. The ground points are sorted depending on their distance to the camera. Subsequently, a subset is iteratively formed starting from the ground point closest to the camera. Thus, the individual ground points are successively examined starting from the ground point having the lowest distance to the camera. Therein, the height of the ground points is respectively determined. Depending on their height, the ground points are then associated or not associated with a subset. Subsequently, only those ground points associated with the subset are used to determine the plane of the roadway. This is based on the realization that the ground points disposed closer to the camera can be more accurately

characterized. In particular, the respective positional value of these ground points can be more accurately determined. Thus, starting from the ground point closest to the camera, it can be examined if the ground points, which first have been associated with the roadway surface, actually describe the roadway surface. Further, the camera presents a fixed reference quantity with its known installation position. In this manner, the plane of the roadway can be more reliably determined.

Preferably, starting from the ground point closest to the camera, the ground points are each successively associated with the subset depending on their height and depending on the ground points already contained in the subset. In other words, the individual ground points are successively checked with respect to their height starting from the ground point closest to the camera. If the respective height of the ground points is in a predetermined range, the ground point is associated with the subset. In the method, those ground points already associated with the subset are additionally taken into account. Here too, it is taken into account that those ground points disposed closer to the camera can be more reliably characterized with respect to their spatial position in the environmental region. The respective ground points are examined starting from the ground point closest to the camera, and it is checked whether or not the respective ground point can be associated with the subset. By this method, it can be iteratively determined if the ground points actually are to be associated with the roadway surface.

In an embodiment, for associating the ground points with the subset, a height value is determined for each of the ground points and a line of best fit is determined for the height values starting from the height value of the ground point closest to the camera. For each of the ground points, thus, a height value is determined based on the positional value, which describes the height of the ground point in the environmental region. For example, the height value can describe the height of the ground point in a camera coordinate system. For example, the respective ground points can be registered in a diagram, which describes the distance to the camera on the one hand and the height value on the other hand. From the ground point closest to the camera, a line of best fit can then be iteratively determined, wherein outliers, which cannot be associated with this line of best fit, are not included in the subset. In this manner, the subset and thereby also the plane of the roadway can be determined with low computational effort and within a short computing time.

Furthermore, it is advantageous if for determining the line of best fit, a center of gravity of the ground points already contained in the subset and the ground point next in the row is each iteratively determined and a slope of a connecting line between the center of gravity and the height value of the ground point next in the row is determined. For determining the line of best fit, those ground points already contained in the subset and the ground point next in the row are examined. At the beginning of the determination of the line of best fit, for example, only the ground point closest to the camera can be contained in the subset. Between the determined center of gravity and the height value of the ground point next in the row, a connecting line is then determined. This connecting line is then used to determine the line of best fit. Thus, the line of best fit can be simply determined with low computational effort.

Preferably, the ground point next in the row is associated with the subset if the determined slope falls below a predetermined limit value. As previously mentioned, a connecting line can be determined between the center of gravity and the height value of the ground point next tin the row. In addition, a slope of this connecting line can be examined. If the slope falls below a certain limit value, the next ground point in the row can be associated with the subset. If the slope of the connecting line exceeds the predetermined limit value, the ground point next in the row can be considered as an outlier and not be associated with the subset. Therein, it can also be provided that the slope of the connecting line previously determined in the iterative method is taken into account. This allows determining the subset of the ground points and thereby also the plane of the roadway within a short computing time.

Preferably, the ground points are associated with a plurality of predetermined areas depending on their positional value and the subset of the ground points is determined for each of the areas. In other words, a plurality of areas can be predetermined in the environmental region. The respective ground points can be associated with one of the predetermined areas depending on their positional value or their spatial position. The ground points can then be sorted depending on their distance to the camera for each of the areas. Subsequently, it can be examined if the ground points are to be associated with the subset for each of the areas starting from the ground point closest to the camera. The subsets determined for the respective areas can then be combined with each other to determine the plane of the roadway. If the motor vehicle includes multiple cameras, which are disposed distributed on the motor vehicle, a plurality of areas can be predetermined for each of the cameras. In this manner, the plane of the roadway can be reliably determined.

Preferably, the plurality of areas is predetermined such that the areas are each formed in the shape of a circular sector. In other words, an auxiliary plane can be defined starting from the camera, which in particular extends along the horizontal. This auxiliary plane can be divided into multiple circular sectors, wherein a respective center of the circular sectors is in the area of the camera. The sectors can encompass an area, which substantially corresponds to the capturing range of the camera. Thereby, the plane can be determined in simple and reliable manner.

Preferably, a plurality of images is provided by means of the camera, the plane of the roadway is determined based on each of the provided images, and an averaged plane is determined from the determined planes. In other words, the camera can provide a sequence of images or an image sequence. For each of the images, the plane of the roadway is then determined. From the individual planes having been determined in the respective images, an averaged or a smoothed plane can then be determined. Thus, measuring errors can be compensated for. In addition, the plane of the roadway can be consecutively determined in this manner. In an embodiment, the plane of the roadway is determined such that respective distances of the subset of the pixels to the plane with respect to a normal of the plane are minimal. This method can also be referred to as orthogonal approximation. Herein, the distances are measured along a normal vector of the plane. Thus, the plane can be simply and reliably determined based on the subset of the pixels.

According to an alternative embodiment, it is provided that the plane of the roadway is determined such that respective distances of the subset of the pixels to the plane with respect to the height are minimal. This method can also be referred to as planar approximation. Therein, it is in particular taken into account that the height value of the ground points, which is for example the position of the respective ground point in a z- direction depending on the position of the respective ground value in the other spatial directions, for example an x- and a y-direction. In this manner too, the plane of the roadway can be determined with high reliability.

Furthermore, it is advantageous if the plane of the roadway is determined based on the subset of the pixels by means of the least squares method. With the least squares method, the plane can be laid in the subset of the pixels such that the distances of the respective pixels to the plane are minimized. This allows determining the plane within a short computing time.

In an embodiment, an orientation of the plane of the roadway located in front of the motor vehicle in direction of travel is determined with respect to a longitudinal axis and/or a transverse axis and/or a vertical axis of the motor vehicle. In other words, the plane of the roadway can be determined with respect to a coordinate system of the motor vehicle or a coordinate system of the camera. Thus, the orientation of the plane with respect to the motor vehicle can be determined in reliable manner. Thereby, it can for example be determined if the roadway in front of the motor vehicle descends or ascends. This information can then for example be used in the object recognition.

A driver assistance system according to the invention for a motor vehicle is adapted to perform a method according to the invention. The driver assistance system includes at least one camera. It can also be provided that the driver assistance system includes a plurality of cameras, which are disposed distributed on the motor vehicle. Moreover, the driver assistance system includes a control device, by means of which the images of the at least one camera can be evaluated, and by means of which the plane of the roadway can be determined. The control device can for example be constituted by an electronic control unit of the motor vehicle.

A motor vehicle according to the invention includes a driver assistance system according to the invention. The motor vehicle is in particular formed as a passenger car.

The preferred embodiments presented with respect to the method according to the invention and the advantages thereof correspondingly apply to the driver assistance system according to the invention as well as to the motor vehicle according to the invention.

Further features of the invention are apparent from the claims, the figures and the description of figures. The features and feature combinations mentioned above in the description as well as the features and feature combinations mentioned below in the description of figures and/or shown in the figures alone are usable not only in the respectively specified combination, but also in other combinations or alone, without departing from the scope of the invention. Thus, implementations are also to be considered as encompassed and disclosed by the invention, which are not explicitly shown in the figures and explained, but arise from and can be generated by separated feature combinations from the explained implementations. Implementations and feature combinations are also to be considered as disclosed, which thus do not have all of the features of an originally formulated independent claim.

Now, the invention is explained in more detail based on preferred embodiments as well as with reference to the attached drawings.

There show:

Fig. 1 in schematic illustration a motor vehicle according to an embodiment of the present invention, which has a driver assistance system with four cameras;

Fig. 2 an image provided by one of the cameras and showing an environmental region of the motor vehicle; Fig. 3 the motor vehicle according to Fig. 1 , which is located on a roadway, wherein two objects are in front of the motor vehicle in direction of travel of the motor vehicle;

Fig. 4 the motor vehicle and the objects according to Fig. 3, wherein the roadway has an upgrade;

Fig. 5 the motor vehicle and the objects according to Fig. 3, wherein the roadway has a downgrade;

Fig. 6 a schematic flow diagram of a method for determining a plane of the

roadway;

Fig. 7 a plurality of ground points, which are associated with respective areas;

Fig. 8 a schematic flow diagram of a method for pre-filtering the ground points;

Fig. 9 multiple graphs describing the determination of a line of best fit for the ground points; and

Fig. 10 a schematic flow diagram of a method for determining the line of best fit.

In the figures, identical and functionally identical elements are provided with the same reference characters.

Fig. 1 shows a motor vehicle 1 according to an embodiment of the present invention in a plan view. In the present case, the motor vehicle 1 is formed as a passenger car. The motor vehicle 1 includes a driver assistance system 2, which serves for assisting the driver of the motor vehicle 1 in driving the motor vehicle 1 . The driver assistance system 2 includes a control device 3, which can for example be constituted by an electronic control unit of the motor vehicle 1 . Moreover, the driver assistance system 2 includes at least one camera 4.

In the present embodiment, the driver assistance system 2 includes four cameras 4, which are disposed distributed on the motor vehicle 1 . Herein, one of the cameras 4 is disposed in a front area 5, one of the cameras 4 is disposed in a rear area 6 and two of the cameras 4 are disposed in respective lateral areas 7, in particular in areas of the wing mirrors. At least one image 10 (see Fig. 2) of an environmental region 8 of the motor vehicle 1 can be captured by the respective cameras 4. The cameras 4 are connected to the control device 3 for data transmission. Presently, corresponding data lines are not illustrated for the sake of clarity. The respective images 10 of the cameras 4 can be evaluated by means of the control device 3.

With the aid of the driver assistance system 2, in particular objects 9 in the environmental region 8 of the motor vehicle 1 are to be recognized. Hereto, the control device 3 can use a corresponding object recognition algorithm to recognize the objects 9 in the images 10. Herein, a three-dimensional object recognition algorithm can in particular be used. Fig. 2 exemplarily shows an image 10, which is provided by one of the cameras 4. In the image

10, a plurality of object points 1 1 are recognized, which are associated with respective objects 9 in the image 10. In addition, a positional value is associated with each of the object points 1 1 , which describes the relative position of the respective object point 1 1 . Based on the positional value, it can thus be determined, where the object point 1 1 is located in the environmental region 8 of the motor vehicle 1 . Presently, those object points

1 1 , the height of which falls below a threshold value h, are associated with a roadway 13 or a roadway surface. These object points 1 1 associated with the roadway 12 are designated as ground points p.

Fig. 3 shows the motor vehicle 1 , which is located on the roadway 12. Two objects 9 are located in front of the motor vehicle 1 in direction of travel. Further, the threshold value h for the height of the objects 9 is drawn. This threshold value can for example be in a range between 10 cm and 25 cm. The object point 1 1 of the object 9, which is disposed closer to the motor vehicle 1 , has a height, which is less than the threshold value h. The object point 1 1 of the object 9, which is farther away from the motor vehicle 1 , has a height, which is greater than the threshold value h. The threshold value h can for example be selected such that the object 9 having a lower height than this threshold value h can be traversed by the motor vehicle 1 without damage to the motor vehicle 1 . Presently, a plane 13 of the roadway 12 is straight.

Compared hereto, Fig. 4 shows an example, in which the roadway 12 has an upgrade. Upon assumption of a straight plane 13, the heights of the object points 1 1 of the two objects 9 would exceed the threshold value h. Compared hereto, Fig. 5 shows an example, in which the roadway 12, on which the motor vehicle 1 is located, has a downgrade. Upon assumption of a straight plane 13, the object points 1 1 of the two objects 9 would be associated with the roadway 12 since their heights are lower than the threshold value h. In this case, collision with the rear object 9 could impend. For this reason, it is required to precisely determine the plane 13 of the roadway 12. In Fig. 4 and 5, such planes 13 are exemplarily drawn.

Fig. 6 shows a schematic flow diagram of a method for determining the plane 13 of the roadway 12. The method is coarsely organized in two blocks 14 and 15. In the block 14, pre-filtering of the ground points p for determining a subset of the ground points p is effected. In the block 15, the plane 13 is determined based on the subset of the ground points p. In a step S1 , the method is started. Subsequently, the ground points p in the image 10 are determined in a step S2, which are associated with the roadway 12. In a step S3, the ground points p are associated with a subset. In addition, the ground points p, which are associated with the subset, are output in a step S4. In addition, the ground points p are determined in a step S5, which do not belong to the subset. These ground points p, which are not associated with the subset, can also be designated as outliers. As explained in more detail below, the determination of the subset of the ground points p can be separately effected for respective areas 16. In a step S6, an accumulated matrix is determined based on the respective subsets of the ground points p. In a step S7, the plane 13 is then determined based on the subset of the ground points p by means of the least squares method. In a step S8, the determined plane 13 having been determined based on the subset of the ground points p of the current image 10 is then output.

Presently, it is provided that the plane 13 is determined based on the ground points p of multiple images 10 consecutive in time. Thereto, in a step S9, a history of the last n accumulated matrices is output. Subsequently, in a step S10, a combined accumulated matrix for the last n images 10 is output. In a step S1 1 , the plane 13 is determined for each image 10 by means of the least squares method. In addition, an averaged or smoothed plane is output based on the respective determined planes 13 in a step S12. Finally, the method is terminated in a step S13.

Fig. 7 shows the motor vehicle 1 in a plan view. Presently, the pixels p determined in the image 10 are transformed into a camera coordinate system. Presently, this is shown based on the camera 4, which is disposed in the right lateral area 7. In addition, a plurality of areas 16 is defined around the camera 4 in this coordinate system, which are each formed in the shape of a circular sector. The respective circular sectors are determined such that their center is disposed in the area of the camera 4. The respective ground points p are then associated with one of the areas 16 based on their positional value. Then, it is checked for each of the areas 16 if the ground points p are to be associated with a subset. This is effected during pre-filtering or in the block 14, which is explained in more detail in Fig. 8. Herein, the method is started in a step S14. In a step S15, the positions of the respective ground points p are transformed into the camera coordinate system. In a step S16, a local list of the ground points p is then output. In a step S17, the respective ground points p are associated with one of the areas 16 or the sectors based on their positional value. In a step S18, a list of the ground points p with respect to the areas 16 is then output. In a step S19, the ground points p are sorted depending on their distance r to the camera 4. In a step S20, a line of best fit 17 is determined starting from the ground point p closest to the camera 4. The line of best fit 17 is iteratively determined for the respective ground points p. Herein, outliers can also be determined. Subsequently, the method is continued for the next ground point p in the row in a step S21 . When all of the ground points p have been successively examined, the method is terminated in a step S22.

Fig. 9 shows six diagrams, based on which the determination of the line of best fit 17 is to be illustrated. The respective diagrams show a distance r to the camera 4 on the abscissa and a height value z on the ordinate. The height value z describes the height of the respective ground point p in the camera coordinate system. In the diagram identified by a, the respective ground points p are plotted for each of the areas 16 or sectors. In the diagram identified by b, it is started with that ground point p₀, which is closest to the camera 4. An existing slope a is set to the value of zero and a new slope β is set to the value of zero. In the diagram identified by c, the next ground point pi in the row is determined. For the points p₀ and pi , a center of gravity C is determined. In addition, a connecting line 18 between the center of gravity C and the point pi is determined.

Subsequently, a slope γ of the connecting line 18 is determined. Therein, it is checked if a magnitude of the difference between the slope γ and the last valid slope a is small enough. Presently, this is the case. Subsequently, the newly resulting slope β is determined. It is calculated according to the following formula: β = γ · 0.5 + α · 0.5 .

Subsequently, it is checked if the slope β is small enough. If this is the case, a is set equal to β. In the diagram identified by d, the next ground point p₂ in the row is examined. Here too, the center of gravity C is determined from the points p₀, pi and p₂. In addition, the connecting line 18 between the center of gravity C and the point p₂ is here determined. Subsequently, it is checked if the absolute value of the slope γ minus the slope a is small enough. Presently, this is not the case. Presently, the point p₂ is defined as an outlier. In the diagram identified by e, the method is analogously continued for the next point p₃. The diagram identified by f shows the line of best fit 17 through the ground points p. Presently, the outliers identified by the reference character 18 were not considered.

Fig. 10 shows a schematic flow diagram for determining the line of best fit 17. Therein, the method is started in a step S23. In a step S24, the last valid slope a is set to the value 0. In addition, the counter for the ground points p associated with the subset and the counter for the outliers are set to the value 0. Further, in a step S25, the list with the ground points 9 in the sector or the area 16 is provided. In a step S26, the next ground point p along the row starting from the camera 4 is examined. In a step S27, the distance r to the camera 4 and the height value z of the pixel p are provided. Subsequently, the center of gravity C of the ground point p and the ground points p already contained in the subset is determined in a step S28. Therein, the center of gravity C corresponds to the sum of the ground points p contained in the subset divided by the number of the ground points p of the subset added to the value 1 . Therein, the center of gravity C is determined both for the distance r and for the height value z (step S29). In a subsequent step S30, the slope γ is determined. The slope γ is determined between the center of gravity C and the ground point p (step S31 ).

In a step S32, it is checked if the absolute value of the difference between the slope γ and the last valid slope a is less than a predetermined threshold value. If this is the case, the method is continued in a step S33. Herein, the newly resulting slope β is determined. It results from the sum of the slope a multiplied by a weighting factor and the slope β multiplied by 1 minus the weighting factor. In a step S34, it is then checked if the newly resulting slope β is less than a limit value. If this is not the case, a is set equal to β and the ground point is included in the subset (step S35). If this is not the case, the ground point p is marked as an outlier in a step S36 and the counter of the outliers is incremented by one value (step S37).

In a step S38, it is further checked if the counter for the outliers is < 2. If the counter for the outliers is greater than 2, the three ground points are not included in the subset in a step S39. Otherwise, it is checked in a step S40 if all of the ground points p have been checked. If further ground points are no longer present (step S41 ), the line of best fit 17 is output in a step S42. Therein, the line of best fit 17 has the slope of the connecting line 18 through the last valid ground point p (step S43). Finally, the method is terminated in a step S44.

In this manner, the subset of the ground points p can be determined. The plane 13 of the roadway 12 can then be determined from this subset. The plane 13 can in particular be determined according to the least squares method. Herein, it is to be attempted to minimize the distances δ, of the ground points p. Thereto, two methods can be used. In a first method, which can also be referred to as orthogonal approximation, the orthogonal distances δ, of the ground points p are minimized. This means that the distances δ, along the normal vector

of the plane 13 are determined. A second method provides that the distances δ, with respect to the height values z or in z-direction are minimized. This method can also be referred to as planar approximation or as z approximation. For both methods, a plurality of ground points p,,..., p_n in the two-dimensional space are present. Overall, it is attempted to minimize the sum of the squares of the distances δ, for each of the points p. Hereto, a cost value Δ is determined. This can be expressed by the following formula:

In the orthogonal approximation, it is assumed that the plane 13 is given in the Hesse normal form. Therein, a point x on the plane 13 is defined via the normal vector ή and the point on the plane p₀:

If a point y is not on the plane, the value zero does not result for Thus, the

cost value Δ or a distance can be determined as follows:

In this equation, it is assumed that the ground point p₀ located on the plane 13 is known. If the points ρ,,...,ρ_η are centered by subtracting the center of gravity C, the value of p₀ becomes zero and the cost value Δ can be determined as follows:

wherein r(x, y, z) is the three-dimensional Gramian matrix, which includes the sum of the inner products of the points. Herein, r = r(x, ;y, z) is set and one obtains:

Therein, Γ is symmetrical and quadratic. For solving the approximation, the eigenvalue problem can be solved:

Since the Gramian matrix is symmetrical, it has three eigenvalues λ₁≤λ₂≤λ₃ . The corresponding and normalized eigenvectors are The minimization results as

follows:

This means that is the normal vector of the searched plane 13. The cost value Δ corresponds to

In the planar approximation, the distances in z-direction δ, of the points Ρί(χ,, y,, z,) are determined for the plane 13:

_p

Herein, it is attempted to determine has a minimum. Thereto,

is defined as follows:

a, b and d represent variables. Here too, it is attempted to center the ground points Pi,...,p_n by subtracting the center of gravity C such that d becomes zero. This can be defined for the cost value Δ as follows:

Therein, X, Y and Z are components of the vector for the ground points, wherein the

Euclidean norm is to be minimized. This can be effected in that the orthogonal projection (a^■ X + b^■ Y) is determined for Z in the two-dimensional sub-space L(X,Y). The solution hereto is:

with the determinants:

As a result, one obtains the plane equation:

After multiplication with G, one obtains:

G_x-x + G_y-y-G-z = 0.

The plane equation a-x + b-y-c-z = 0 can be transformed into the Hesse normal form: a- x + b- y + -c- z = 0.

(a,b,c)-(x,y,z) = 0

(a,b,c)- p = 0. The vector (a, b, c) is orthogonal to the approximated plane 13. Its norm is

If the orthogonal vector (a, b, c) is normalized to the length of one, the Hesse normal form results:

The accumulated matrix can be determined for both described methods. Therein, a 4x4 matrix σ, is determined for each point p,, which has the following form :

The addition of all σ, results in the matrix∑ :

Herein, it is apparent that the matrix∑ has the Gramian matrix Γ(χ, y, z) as a sub-matrix.

The outer rows or columns include the component vectors for the points and the number of the points.

For centralizing the data, both above described variants can be used. Therein, respective point clouds are centered in their own center of gravity S. Since the accumulated matrix ∑ only includes the data of the non-centered points, since they stem from the real world, first, a centered accumulated matrix has to be determined:

Herein, the elements of the matrix are to be determined from the elements of the

matrix∑ . Herein, the component vectors

become zero. On the example of ,

this is to be explained in the following. In the accumulated matrix, the component vector X results as x_l + x₂ + χ₃ + · · · χ_η and includes the sum of all of the x components of the ground points p,. Presently, the vector

is to be determined, which has the corrected x components and the x components of the center of gravity C on

. This can be described as follows:

This means that the sum of all x values of the points must be zero. For calculating the part of the Gramian matrix, below is considered as an example:

This does not apply to non-centered data. For determining the centered data for the

following calculation can be effected:

In the same manner, the other elements of the Gramian matrix can be determined.

Overall, the matrix results as:

In this manner, the plane 13 of the roadway 12 can be reliably determined with low computational effort. This allows reliably characterizing objects 9 in the environmental region 8 of the motor vehicle 1 with respect to their height. In this manner, obstacles to the motor vehicle 1 can be recognized with greater precision based on the images 10 of the cameras 4. Thereby, the driver assistance system 2 can be securely operated.

Claims

1 . Method for determining a plane (13) of a roadway (12), in which an image (10) is captured by means of a camera (4) of a motor vehicle (1 ) located on the roadway (12), wherein the image (10) describes an environmental region (8) of the motor vehicle (1 ), which includes a part of the roadway (12), a plurality of ground points (p, Po, p₁ , p₂, P₃) is determined in the image (10), which are associated with the roadway (12) in the image (10), wherein a positional value describing a spatial position in the environmental region (8) is associated with each of the ground points (p, po, p₁ , p₂, P₃), and the plane (13) of the roadway (12) is determined based on the ground points (p, p₀, pi , p₂, p₃),

characterized in that

the ground points (p, p₀, Pi , p₂, P3) are sorted based on their positional values depending on their distance (r) to the camera (4), a subset is iteratively formed from the ground points (p, p₀, Pi , p₂, p₃) depending on their height starting from the ground point (p₀) closest to the camera (4) and the plane (13) of the roadway (12) is determined based on the subset of the ground points (p, p₀, Pi , p₂, p₃).

2. Method according to claim 1 ,

characterized in that

starting from the ground point (p₀) closest to the camera (4), the ground points (p, Po, pi , p₂, p₃) are each successively associated with the subset depending on their height and depending on the ground points (p, p₀, pi , p₂, p₃) already contained in the subset.

3. Method according to claim 1 or 2,

characterized in that

for associating the ground points (p, p₀, pi , p₂, p₃) with the subset, a height value (z) is determined for each of the ground points (p, p₀, pi , p₂, p₃) and a line of best fit (17) is determined for the height values (z) starting from the height value (z) of the ground point (p₀) closest to the camera (4).

4. Method according to claim 3,

characterized in that for determining the line of best fit (17), a center of gravity (G) of the ground points (p, Po, p₁ , p₂, P₃) already contained in the subset and of the ground point (pi, p₂, p₃) next in the row is respectively iteratively determined, and a slope (γ) of a connecting line (18) between the center of gravity (C) and the height value (h) of the ground point (Pi , P2, P3) next in the row is determined.

5. Method according to claim 4,

characterized in that

the ground point (p_{1 ;} p₂, p₃) next in the row is associated with the subset if the determined slope (γ) falls below a predetermined limit value.

6. Method according to any one of the preceding claims,

characterized in that

the ground points (p, p₀, p₁ , p₂, P₃) are associated with a plurality of predetermined areas (16) depending on their positional values and the subset of the ground points (p, po, p₁ , p₂, P₃) is determined for each of the areas (16).

7. Method according to claim 6,

characterized in that

the plurality of areas (16) is predetermined such that the respective areas are formed in the shape of a circular sector.

8. Method according to any one of the preceding claims,

characterized in that

a plurality of images (10) is provided by means of the camera (4), the plane (13) of the roadway (12) is determined based on each of the provided images (10) and an averaged plane is determined from the determined planes (13).

9. Method according to any one of the preceding claims,

characterized in that

the plane (13) of the roadway (12) is determined such that respective distances of the subset of the pixels (p, p₀, pi , p₂, p₃) to the plane (13) with respect to a normal of the plane (13) are minimal.

10. Method according to any one of claims 1 to 8,

characterized in that

the plane (13) of the roadway (12) is determined such that respective distances of the subset of the pixels (p, p₀, p₁ , p₂, P₃) to the plane (13) with respect to the height are minimal.

1 1 . Method according to any one of the preceding claims,

characterized in that

the plane (13) of the roadway (12) is determined based on the subset of the pixels (p, p₀, p₁ , p₂, P₃) by means of the least squares method.

12. Method according to any one of the preceding claims,

characterized in that

an orientation of the plane (13) of the roadway (12) located in front of the motor vehicle (1 ) in direction of travel is determined with respect to a longitudinal axis and/or a transverse axis and/or a vertical axis of the motor vehicle (1 ).

13. Driver assistance system (2) for a motor vehicle (1 ), which is adapted to perform a method according to any one of the preceding claims.

14. Motor vehicle (1 ) with a driver assistance system (2) according to claim 13.